-
Math overflow?
For some project I need to do some calculations, since debugging for this project is very hard because I have not really an output device on this project. I would like to know if this calculation would work smoothly:
Code:
Temp2 var byte
Temp3 var word
Temp3 = (MasterVolume * 128) / 100
Temp2 = Temp3 + ((Temp3 * (TrimLeft * 10)) / 500)
Mastervolume and trimleft having a max. value of 100
is this the right way for this calculation ? i already know that the result will not be 100% but this is not a problem.
-
Yes, it will work fine for Temp3,
But tempt2 will be approx 12 million. You will have to use a double on this. Melabs has a FP up to 32 bits...
http://www.melabs.com/resources/fp.htm
Dwayne
-
Or, you could just use DIV32. And, make sure Temp2 is a WORD.
Code:
Temp2 var word
Temp3 var word
Temp3 = (MasterVolume * 128) / 100
Temp2 = TrimLeft * 10
Temp2 = Temp2 * Temp3
Temp2 = DIV32 500
Temp2 = Temp2 + Temp3
HTH,
Darrel
