Another Silly Math Question!

[Ramius]

Hi All!
I have a draw full of LTC 1298 12 bit A/D converters. So at 5.0 volts DC on the input of a channel the A/D give the digital number 4096. Okay so lets say you do see 5.00 volts on the input it seems the basic formula would be INPUT*10 divided by 4096 * 5 which would give 50 or 5.00 volts. However, at an input of 3.71 volts you have an A/D input of 3040. 3040*10= 30400. Now divide by 4096 you get 7.42 and the PIC will only give you the 7 so that 7*5=35 and not the 3.71 volts. So how do you create a formula that is accurate? Do you need to take your A/D reading and divide it or ....???

Thanks in advance, Ed

[HenrikOlsson]

Hi Ed,
5V/4096=1.2207mV per count. You can use the ** operator to multiply by units of 1/65536, try something like Voltage = ADResult ** 8000.
Basically the ** operator multiplies by the value given and then divides the intermediate 32bit result by 65536 so doing ADResult ** 8000 would be the same as doing ADResult / 0.12207 if you could do that.

At 5V input you'll get a ADResult of 4096. 4096*8000/65536=500 representing 5.00V
At 3.71V input you'll get an ADResult of 3040. 3040*8000/65536=371 representing 3.71V

It's not perfect bu pretty close. How I came up with 8000? Because 500/4096 * 65536 = 8000

/Henrik.