Many of Microchips new offerings are 'micro-power'... that's mico-amps... ie they run on a drop of Lemon juice and two dissimilar metals (try it!)... I've powered an LCD and PIC with a potatoe, however, despite there being a ready and cheap supply on the high-street, they don't come in handy AA size uniform shapes to fit the Battery compartment. PICs can even start running in your programmer during the verify cycle - I've posted a report and warning about it in this very forum.

All the I/O's have protection diodes (go look at the PIC block schematics) and it's through those Diodes that your PIC is being powered.

It is BAD PRACTICE to design any microprocessor circuitry where in the application there is power appearing on the pins but not on Vdd. Redesign the way you approach your application to avoid this.