When a PIC goes into sleep mode, the primarary oscillator is shut-down (internal or external).
So it shouldn't matter what oscillator you are using.

And in sleep mode, a 4520 should only draw 10uA or less. (about 1/1000th of what you are reading).

Are you sure you didn't leave the WDT ON?
Any interrupts enabled?

There are several things you can turn off to get lower current in sleep mode, but none of them take 6 ma.
<br>