Change your test circuit as follows:

Put a diode between the cap and the battery disconnect point like the sample schematic at the beginning of this thread has. This will isolate the PIC power from the rest of the circuit so that the cap will power the PIC when the battery is disconnected.

Connect the LED cathodes to the I/O pins. Then connect the LED anodes to the battery disconnect point between the battery and the diode that was added above. By doing this, the LEDs won't draw power once the battery is disconnected.