If you truely have filtered, noisless input on your ADC pin like you say, it won't be jumping about they way it does. Movement by one or two counts is possibly expected - but not by twenty or thirty.

As a start, increase your Sample Time and/or change your Clock Source. It's a good possibility that you are not letting the sampling Capacitor enough time to charge properly before you take your reading.

It's also a possibility you have set the wrong Justification for a WORD variable. Double-check this also.