Set all pin to low, and then just change tris registers, instead of changing port and tris.
Also add 5.1V zener between resistor common wire and ground.