Hi,
First of all. With a VRef of 4.096 you can not input a voltage HIGHER than that, so the 4.1V (although very close) isn't a valid test. With that said something is seriously wrong - either with the code, the setup, or the hardware.

And again, have you actually looked at the reference voltage AND at the input voltage with a scope to verify that they are clean? Simply measuring them with a voltmeter may not show the error.

Instead of using the pot across the reference voltage, try using a low impedance source, ie a variable DC supply, to feed the ADC and put a 10nF capacitor from the PIC pin to GND to act as a noise filter. See if that makes it any better. Also, take a couple of readings and average them to further clean any noise.

If the above doesn't help then try another channel on the ADC or try another PIC. It's possible, but not likely, that something got damaged when you overloaded the input.

/Henrik.