I've been looking over your code for about 10 minutes and nothing is jumping out at me. My only thought is something flaky is going on during the power down stage where PORTB.4 goes high momentarily. You could try adding a debounce or a small pause at the beginning of the SWITCH routine and see if there is any change.

David