View Full Version : What is wrong with this division?
tjkelly
- 5th March 2010, 00:01
I am compiling this with the long version of PBP and using an 18F2420.
Why do I get the result of the division as shown below?
I expected it to be 0xC455A
P var Long
current_voltage var Word
current_voltage = 49800
P = 78610 * current_voltage ; results is 0xE956C590
P = P / 4868 ; result is 0xFFFECEED in MPLAB watch
mackrackit
- 5th March 2010, 03:07
Looks like you need Extra_LONGS for that. :)
The multiplication over flows the LONG VAR.
Try
current_voltage = 49800
P = 7861 * current_voltage
P = P / 4868 * 10
ScaleRobotics
- 5th March 2010, 05:25
Nevermind.
tenaja
- 5th March 2010, 15:03
Looks like you need Extra_LONGS for that. :)
The multiplication over flows the LONG VAR.
Try
current_voltage = 49800
P = 7861 * current_voltage
P = P / 4868 * 10
Dave, I thought Longs were 4 bytes... ?
49800 x 7861 = 391477800 = $1755 7A28 ...which is 4 bytes...
Acetronics2
- 5th March 2010, 15:22
Hi, tenaja ...
I think you've been cheating ...
Dave is the best ...
LONGS are SIGNED ... ( Manual p.24 )
Alain
tenaja
- 5th March 2010, 15:31
Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
mackrackit
- 5th March 2010, 15:47
The OP had
49800 * 78610 = 3914778000
tenaja
- 5th March 2010, 16:29
Yes, but he also said it did not match the calculator output. My point was that the calculator proved it fits within the constraints of a Long.
mackrackit
- 5th March 2010, 16:31
A calculator gives
3914778000
LONGS can be
2147483647
Acetronics2
- 5th March 2010, 16:39
Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
in fact, the sign is bit 31 ...
so, ... better let's say +/- 3 3/4 bytes ... for maximum value
Alain
tjkelly
- 5th March 2010, 17:04
Yeah, what a dumb mistake on my part. I had forgotten about the sign bit being lost resulting in only 31 bits. Thanks.
tenaja
- 5th March 2010, 17:34
A calculator gives
3914778000
LONGS can be
2147483647
My bad. I missed the last zero when entering it in the calculator. Apparently I took the numbers from your code and mistook it as a quote.
Powered by vBulletin® Version 4.1.7 Copyright © 2025 vBulletin Solutions, Inc. All rights reserved.