This is what I will bet is happening.

If you have not changed the *.inc file (configs) MCLR is on.

When A3 = 0 causes the LED to go high, then when the switch is pressed A3 goes to a hard LOW resetting the PIC. Something like that.

Yup, move over to a bread board to get started.