Sometimes it only starts when I unplug the programming plug from the programmer.
I will assume you are not powering the chip with the PicKit?

Is the chip still in the EPIC when you use the PicKit?

Maybe a picture of your setup?