MAIN:
IF PORTB.3 = 0 then SWITCH_LED
goto main

SWITCH_LED:
pause 100 'Debounce
toggle PORTB.4
goto main

So what happens above when you keep your finger pressed on the button for longer than 100mS? Your LED will keep toggling every 100mS...

Try this...

MAIN:
IF PORTB.3 = 0 then SWITCH_LED
goto main

SWITCH_LED:
pause 100 'Debounce
toggle PORTB.4
While PortB.3=0:Wend ' Wait here for Button to be released
goto main