from the data sheet, my best guess . i see no attempt to read portb before sleep

For enabled interrupt-on-change pins, the present valueis compared with the old value latched on the last read
of PORTB to determine which bits have changed or
mismatched the old value. The ‘mismatch’ outputs of
the last read are OR’d together to set the PORTB
Change Interrupt flag bit (RBIF) in the INTCON register.
This interrupt can wake the device from Sleep. The user,
in the Interrupt Service Routine, clears the interrupt by:
a) Any read or write of PORTB. This will end the
mismatch condition.
b) Clear the flag bit RBIF.
A mismatch condition will continue to set flag bit RBIF.
Reading or writing PORTB will end the mismatch
condition and allow flag bit RBIF to be cleared. The latch
holding the last read value is not affected by a MCLR nor
Brown-out Reset. After these Resets, the RBIF flag will
continue to be set if a mismatch is present.