Do you have JP5 connected?
Without JP5 connected you will get strange results.
With JP5 connected A3 will always be high, then when the button is pressed A3 will go low.