however
if you have a word var say numx
and x =4095
then 50*numx/100 the result is 81
the word var causes a intermediate stage overflow (bits 0-15 of 204750 =8142)
I still say integer math is tricky and results need to be double checked
however
if you have a word var say numx
and x =4095
then 50*numx/100 the result is 81
the word var causes a intermediate stage overflow (bits 0-15 of 204750 =8142)
I still say integer math is tricky and results need to be double checked
Steve Earl www.datageo.co.uk
poor typingWhat is the relationship between numx and x?
if you have a word var say numx
and numx =4095
then 50*numx/100 the result is 81
the word var causes a intermediate stage overflow (bits 0-15 of 204750 =8142)
I still say integer math is tricky and results need to be double checked
intermediate overflow
50*4095 =204750 max value for word 65535 therefore an overflow occurs and the result becomes 8142
Last edited by richard; - 3rd July 2014 at 10:44. Reason: more info
simply stated
if there is a var in a multilstage calculation then any intermediate results must be able to be stored in that var , its just the way it works.
try it yourself
the */ ,** ops are the only workaround
you might have missed the point
50*4095/100 is 2047 this works (all constants)
numx=4095
50*numx/100 is 81 this does not
numy=50
numy*4095/100 is 81 this does not
numz=100
50*4095/numz is 81 this does not
Now I understand what you are saying. It does seem odd to me that there is a 32 bit register for maths calculation which can not be used if there is a word or byte variable in the calculation because the word/byte variable could overflow. A bit more thinking is required on my part, I think.
Does this mean that using ** and */ uses 32 bit and * only 16 bit?
Last edited by EarlyBird2; - 3rd July 2014 at 11:48. Reason: more thinking
Steve Earl www.datageo.co.uk
Look at the Statement keyword "DIV32". It is possible to compute all of these statements:
numx=4095
50*numx/100
numy=50
numy*4095/100
numz=100
50*4095/numz
Dave Purola,
N8NTA
EN82fn
After more thought and a bit of direction to look at DIV32 by Dave.
Multiplication stores a 32 bit result internally but the normal divide only works on the lower 16 bits which causes the 'overflow' problem highlighted by Richard. DIV32 was created specifically to overcome the 16 bit division limitation and as Dave says is a solution here.
Scampy asked about */ and how that works. It works by removing the division in the calculation so there is no 16 bit divide, there is an 8 bit shift or a divide by 256 of the 32 bits whichever way one wishes to visualise the process. Knowing that */ will apply a division by 256 one has to take account of this in the program.
In this case
B_max = (4095/100)*maxbright
take 4095/100 and multiply by 256 giving 10483.2 which is rounded up to 10484. Resulting in this
B_max = maxbright*/10484 which results in an integer value of 2047 when maxbright is 50.
Last edited by EarlyBird2; - 4th July 2014 at 07:44.
Steve Earl www.datageo.co.uk
Bookmarks