I got bit by this phenomena once on my first '18F' series parts after years of using '16F' parts.

Turns out I didn't have both pins of the oscillator set up correctly. One was set for OSC IN but the other was inadvertantly configured for GPIO. It worked fine on my desk for months during development but would inexplicably drop to the internal slow oscillator in the field.

Check the "Clock Sources and Oscillator Switching" section of your data sheet.

The following is for a different device than yours but illustrates the issue.

...includes a feature that allows the
device clock source to be switched from the main
oscillator to an alternate low-frequency clock source.
PIC18F2525/2620/4525/4620 devices offer two alternate
clock sources. When an alternate clock source is enabled,
the various power managed operating modes are
available.
Essentially, there are three clock sources for these
devices:
• Primary oscillators
• Secondary oscillators
• Internal oscillator block

...The secondary oscillators are those external sources
not connected to the OSC1 or OSC2 pins. These
sources may continue to operate even after the
controller is placed in a power managed mode.