This may not be it, but the only thing I notice is: you have OSC_HS selected, but you are using an oscillator at 4 mhz. If you are using an oscillator instead of a resonator, this should be set to XT for 4 mhz. If you are using a resonator, did you see the note with the data sheet?

Note: When using resonators with frequencies
above 3.5 MHz, the use of HS mode,
rather than XT mode, is recommended.
HS mode may be used at any VDD for
which the controller is rated. If HS is
selected, it is possible that the gain of the
oscillator will overdrive the resonator.
Therefore, a series resistor should be
placed between the OSC2 pin and the
resonator. As a good starting point, the
recommended value of RS is 330Ω.