What is wrong with this division?

Closed Thread
Results 1 to 12 of 12
  1. - 5th March 2010, 00:01 #1
    tjkelly's Avatar
    tjkelly
    tjkelly is offline Junior Member
    Join Date
    Jan 2010
    Posts
    19

    Default What is wrong with this division?

    I am compiling this with the long version of PBP and using an 18F2420.

    Why do I get the result of the division as shown below?
    I expected it to be 0xC455A

    Code:
    P var Long
current_voltage var Word	


current_voltage = 49800
P = 78610 * current_voltage	; results is 0xE956C590
P = P / 4868			; result is 0xFFFECEED in MPLAB watch
  2. - 5th March 2010, 03:07 #2
    mackrackit's Avatar
    mackrackit
    mackrackit is offline SuperMod
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Did you find this post helpful? Yes | No

    Default

    Looks like you need Extra_LONGS for that.
    The multiplication over flows the LONG VAR.

    Try
    Code:
    current_voltage = 49800
P = 7861 * current_voltage	
P = P / 4868 * 10
    Dave
    Always wear safety glasses while programming.
  3. - 5th March 2010, 05:25 #3
    ScaleRobotics's Avatar
    ScaleRobotics
    ScaleRobotics is offline Prolific Poster
    Join Date
    Feb 2006
    Location
    Gilroy, CA
    Posts
    1,530

    Did you find this post helpful? Yes | No

    Default

    Nevermind.
    Last edited by ScaleRobotics; - 5th March 2010 at 05:29.
  4. - 5th March 2010, 15:03 #4
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Quote Originally Posted by mackrackit View Post
    Looks like you need Extra_LONGS for that.
    The multiplication over flows the LONG VAR.

    Try
    Code:
    current_voltage = 49800
P = 7861 * current_voltage	
P = P / 4868 * 10
    Dave, I thought Longs were 4 bytes... ?
    49800 x 7861 = 391477800 = $1755 7A28 ...which is 4 bytes...
  5. - 5th March 2010, 15:22 #5
    Acetronics2's Avatar
    Acetronics2
    Acetronics2 is offline Fanatical Contributor
    Join Date
    May 2004
    Location
    NW France
    Posts
    3,648

    Did you find this post helpful? Yes | No

    Talking Rtf ....

    Hi, tenaja ...

    I think you've been cheating ...


    Dave is the best ...

    LONGS are SIGNED ... ( Manual p.24 )

    Alain
    ************************************************** ***********************
    Why insist on using 32 Bits when you're not even able to deal with the first 8 ones ??? ehhhhhh ...
    ************************************************** ***********************
    IF there is the word "Problem" in your question ...
    certainly the answer is " RTFM " or " RTFDataSheet " !!!
    *****************************************
  6. - 5th March 2010, 15:31 #6
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
  7. - 5th March 2010, 15:47 #7
    mackrackit's Avatar
    mackrackit
    mackrackit is offline SuperMod
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Did you find this post helpful? Yes | No

    Default

    The OP had
    49800 * 78610 = 3914778000
    Dave
    Always wear safety glasses while programming.
  8. - 5th March 2010, 16:29 #8
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Yes, but he also said it did not match the calculator output. My point was that the calculator proved it fits within the constraints of a Long.
  9. - 5th March 2010, 16:31 #9
    mackrackit's Avatar
    mackrackit
    mackrackit is offline SuperMod
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Did you find this post helpful? Yes | No

    Default

    A calculator gives
    3914778000
    LONGS can be
    2147483647
    Dave
    Always wear safety glasses while programming.
  10. - 5th March 2010, 16:39 #10
    Acetronics2's Avatar
    Acetronics2
    Acetronics2 is offline Fanatical Contributor
    Join Date
    May 2004
    Location
    NW France
    Posts
    3,648

    Did you find this post helpful? Yes | No

    Talking

    Quote Originally Posted by tenaja View Post
    Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
    in fact, the sign is bit 31 ...

    so, ... better let's say +/- 3 3/4 bytes ... for maximum value

    Alain
    ************************************************** ***********************
    Why insist on using 32 Bits when you're not even able to deal with the first 8 ones ??? ehhhhhh ...
    ************************************************** ***********************
    IF there is the word "Problem" in your question ...
    certainly the answer is " RTFM " or " RTFDataSheet " !!!
    *****************************************
  11. - 5th March 2010, 17:04 #11
    tjkelly's Avatar
    tjkelly
    tjkelly is offline Junior Member
    Join Date
    Jan 2010
    Posts
    19

    Did you find this post helpful? Yes | No

    Default

    Yeah, what a dumb mistake on my part. I had forgotten about the sign bit being lost resulting in only 31 bits. Thanks.
  12. - 5th March 2010, 17:34 #12
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Quote Originally Posted by mackrackit View Post
    A calculator gives
    3914778000
    LONGS can be
    2147483647
    My bad. I missed the last zero when entering it in the calculator. Apparently I took the numbers from your code and mistook it as a quote.
« Previous Thread | Next Thread »

Similar Threads

  1. LCD Showes Some Wrong Letters
    By sbobowski in forum General
    Replies: 2
    Last Post: - 23rd September 2008, 19:15
  2. What am I doing wrong?
    By Charles Linquis in forum mel PIC BASIC Pro
    Replies: 6
    Last Post: - 29th November 2007, 21:46
  3. ADCIN - AD settings are wrong
    By teverett in forum mel PIC BASIC Pro
    Replies: 5
    Last Post: - 4th December 2006, 16:32
  4. Whats wrong with this I2C code???? HELP?
    By yourmomOS in forum mel PIC BASIC Pro
    Replies: 7
    Last Post: - 13th July 2006, 15:18
  5. Decimals and Digits and Division Oh My!
    By Spindle in forum mel PIC BASIC Pro
    Replies: 1
    Last Post: - 23rd June 2005, 08:16

Members who have read this thread : 0

You do not have permission to view the list of names.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Forum Rules