Math Formula / Variable Problem! Help!

[Charles Linquis]

PBP does integer math only. You should rewrite the equation to multiply the A/D converter result by 5000 then divide by 64320 (or multiply by 500 and divide by 6432).

The result will be an integer, so you will lose the fractional part. However, you can get an extra digit of resolution if you modify the formula to something like:

FirstPart = (ADresult * 5000)/6432

HSEROUT [FirstPart/10,".",FirstPart//10]

[Archilochus]

Wouldn't multiplying the AD result by 5000 or 500 cause the variable to overflow?

[Charles Linquis]

Yes it would. Sorry, my error.

So, the best you could do is to multiply by 50 and divide by 643.
You could possibly get closer by using a different multiply/divide ratio.

Another option would be to use the DIV32 function.

[Melanie]

> Lets say I get a reading on the ADC of 11260.

Well, that in itself would be a problem... your 10-bit ADC shouldn't give you readings outside the range 0-1024.

Do the *5000 multiplication first, this in effect would cause an overflow from a word variable, but if immediately followed by a DIV32 (as Charles suggested) would result in an integer value.

So your input of 0-1024 multiplied by 5000 and divided by 64320 would result in a range of 0-79 which is very small and you lose a great deal of precision.

You can add TWO additional signigicant figures by multiplying by 50000 and dividing by 6432, this will then give you a range of 0-7960 which then by using the DIG function will allow you to extract the digits and insert a decimal point displaying your two decimal points to give you 0-79.60

Always work within a WORD variable but if you must spill out of it momentarily, immediately use the DIV32 function to come back.