You should confirm that the device is really running at 4Mhz. Perhaps it is running at 2. You often have to set up the OSCCON register. Check the datasheet.
Also, the interrupt is triggered right away beause you have not cleared the interrupt flag BEFORE enabling the interrupt (usually PIR.x, but again, check your device).