I would try the following:

1/. Use the Fixed Voltage Reference (FVR) instead of Vdd as your ADC reference voltage.
2/. Add some capacitance - say 100nF - between the pot wiper and ground.

Cheers
Barry
VK2XBP