It sounds like your power suppy is "sagging" when you release the switch, because the cap on the PortD.1 is charging, and drawing current from either your MCLR circuit or your power supply.

Add some 10uF caps across the bypass caps on your PIC power pins and the problem should go away.

I learned years ago to use DS1233 reset generator chips on my MCLR pin to solve little issues like this. Never had a problem since.