Help with the conversion math

[HenrikOlsson]

Hi Ed,

The sensor nominally outputs 0.5V at 0 psi. With 5V as the reference the 10bit ADC will return a value of 102 (102.4 ideally) for a 0.5V input.
Becaues we are taking 40 readings and then dividing that by 10 we go from 10bit reslution to 12bit (4 times oversampling) so our 102.4 now becomes 102.4*40/10=409 which is where the 409 in the code comes from.

At full scale (7psi, 0.07psi whatever is is) the sensor nominally outputs 4.5V. With 5V as the reference the 10bit ADC will return a value of 921 (921.6 ideally) for a 4.5V input.
Because we are taking 40 readings and then dividing that by 10 we go from 10bit resoultion to 12bit (4 times oversampling) so our 921.6 now becaomes 921.6*40/10=3686.

So now we have a value ranging from 409 to 3686 for our 0 to 7 (or 0.07) psi. The first thing we need to do is remove the offset. Easy enough, just subtract 409 to get a value ranging from 0 to 3277 for our 0 to 7 (or 0.07psi).

The final step is to get that strange value into something reassembling the actual pressure and that's where the 547 numbler comes in.

The */ operator has be covered numerous times on the forum. It's like multiplying by units of 1/256. So 3277 * (1/256*547) = 7002.

So now we have a value ranging from 0 to 7002 for a pressure of 0 to 7 (or 0.07 or whatever) psi and all we need to do is to put a decimal point between the correct digitis when displaying the value.
For 0 to 7psi:
HSEROUT["Pressure: ", DEC ADResult/1000, ".", DEC2 ADResult//1000, "psi", 13]

For 0 to 0.07psi:
HSEROUT["Pressure: 0.0", DEC ADResult, "psi", 13]

Obviously, if your sensor outputs values other than the nominal (which of course is likely) the numbers needs to be tweaked to calibrate the readings.

/Henrik.

[Ramius]

Thank you Henrik!
You and the people here are incredibly smart! I cannot thank all of you enough for your support and patience in teaching me so much and guiding me out of the "fog" I tend to get into. The high pressure side of the sensor is being fed with a .094" tube so there is not a lot of pressure to sense which is why the sensor has such a small range. The low side of the sensor is "static" pressure so you can compensate for the depth and still be able to know the speed when sub is travelling through water. In Microcode Studios ICD is where I can see the readings from the sensor. So since I know from the ICD that at zero pressure I get 110 and not 102.4 should I only adjust the low value or should I also adjust the high value assuming that the sensor has basically a 4 volt span (.5v to 4.5v) or does the high value stay the same? That is, a range from .5368 volts to 4.5368 volts. Again my most sincere thanks.

Ed

[HenrikOlsson]

Hi Ed,
You may need to change both the offset and the gain value.
Just because the voltage at the low end is off by 37mV doesn't mean the high end is off by the same amount.

Numbers within () are example numbers.

Step 1, Take a bunch of readings at 0 pressure, write down the reported numbers, then calculate the average. (345)
Step 2, Take a bunch of readings at max pressure, write down the reported numbers then calculate the average. (3591)
Step 3, Figure out what you want the final range of values to be. 0-1000, 0-5000, 0-7000 etc? (0-1000)
Step 4, Calculate the "offset" and the "gain":
The "offset" is simply the value calculated in step 1. (345)
The "gain" is [whatever value you want max to be] divided by ( [value at step 2] minus [value at step 1] ) times 256 ie. 1000 / (3591 - 345) * 256 = 79

What's that 79?
Again, it's like multiplying by units of 1/256. So when our value returned by ADC routine is 3591 (max pressure) we first subtract the offset of 345 and then multiplies with 79/256, what's the result?
(3591-345) * 79 / 256 = 1001, not perfect but pretty close.


If you're using the routine I posted earlier, with the 4 times oversampling, then look at the result directly from the ADC (110 in your case), look at the result at the end of the routine. It it might be 438 or 441 and, as long as the sensor isn't super stable and "slient", it'll give you a better over all resolution.

/Henrik.

[Ramius]

One of the things I wanted to do is pubicly thank everyone especially Henrik! After finally receiving replacement sensors (I blew up the last 3 due to over pressure) I got a chance to play! One thing I noticed when firing up the code in Microcode Studio Plus and using the ICD was that the ICD display made room at the bottom to actually display the results! The HSEROUT line of code made eveything so much easier! Again many, many thanks!

Sincerely, Ed