It's not ADC value reference that causes problems.
Here's the explanation.

Say ADC changes from 100 to 200 and "middle" point is being 150.
So to have zero at my system variable output (which I later display both on screen and write to EEPROM), I'm deducting 150 from the input ADC value, to get zero. In case when ADC reading increases, than all is good, final variable value changes from 0 to 50, that's good. But if ADC value gets below 150, then things get bad.