What is wrong with this division?

Closed Thread
Results 1 to 12 of 12

Hybrid View

  1. - 5th March 2010, 15:22 #1
    Acetronics2's Avatar
    Acetronics2
    Acetronics2 is offline Fanatical Contributor
    Join Date
    May 2004
    Location
    NW France
    Posts
    3,653

    Did you find this post helpful? Yes | No

    Talking Rtf ....

    Hi, tenaja ...

    I think you've been cheating ...


    Dave is the best ...

    LONGS are SIGNED ... ( Manual p.24 )

    Alain
    ************************************************** ***********************
    Why insist on using 32 Bits when you're not even able to deal with the first 8 ones ??? ehhhhhh ...
    ************************************************** ***********************
    IF there is the word "Problem" in your question ...
    certainly the answer is " RTFM " or " RTFDataSheet " !!!
    *****************************************
  2. - 5th March 2010, 15:31 #2
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
  3. - 5th March 2010, 15:47 #3
    mackrackit's Avatar
    mackrackit
    mackrackit is offline SuperMod
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Did you find this post helpful? Yes | No

    Default

    The OP had
    49800 * 78610 = 3914778000
    Dave
    Always wear safety glasses while programming.
  4. - 5th March 2010, 16:29 #4
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Yes, but he also said it did not match the calculator output. My point was that the calculator proved it fits within the constraints of a Long.
  5. - 5th March 2010, 16:31 #5
    mackrackit's Avatar
    mackrackit
    mackrackit is offline SuperMod
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Did you find this post helpful? Yes | No

    Default

    A calculator gives
    3914778000
    LONGS can be
    2147483647
    Dave
    Always wear safety glasses while programming.
  6. - 5th March 2010, 17:34 #6
    tenaja's Avatar
    tenaja
    tenaja is offline Member
    Join Date
    Sep 2007
    Location
    USA, CA
    Posts
    271

    Did you find this post helpful? Yes | No

    Default

    Quote Originally Posted by mackrackit View Post
    A calculator gives
    3914778000
    LONGS can be
    2147483647
    My bad. I missed the last zero when entering it in the calculator. Apparently I took the numbers from your code and mistook it as a quote.
  7. - 5th March 2010, 16:39 #7
    Acetronics2's Avatar
    Acetronics2
    Acetronics2 is offline Fanatical Contributor
    Join Date
    May 2004
    Location
    NW France
    Posts
    3,653

    Did you find this post helpful? Yes | No

    Talking

    Quote Originally Posted by tenaja View Post
    Yeah, but it's still only 29 bytes. Does the "sign" take up 4 bytes?
    in fact, the sign is bit 31 ...

    so, ... better let's say +/- 3 3/4 bytes ... for maximum value

    Alain
    ************************************************** ***********************
    Why insist on using 32 Bits when you're not even able to deal with the first 8 ones ??? ehhhhhh ...
    ************************************************** ***********************
    IF there is the word "Problem" in your question ...
    certainly the answer is " RTFM " or " RTFDataSheet " !!!
    *****************************************
« Previous Thread | Next Thread »

Similar Threads

  1. LCD Showes Some Wrong Letters
    By sbobowski in forum General
    Replies: 2
    Last Post: - 23rd September 2008, 19:15
  2. What am I doing wrong?
    By Charles Linquis in forum mel PIC BASIC Pro
    Replies: 6
    Last Post: - 29th November 2007, 21:46
  3. ADCIN - AD settings are wrong
    By teverett in forum mel PIC BASIC Pro
    Replies: 5
    Last Post: - 4th December 2006, 16:32
  4. Whats wrong with this I2C code???? HELP?
    By yourmomOS in forum mel PIC BASIC Pro
    Replies: 7
    Last Post: - 13th July 2006, 15:18
  5. Decimals and Digits and Division Oh My!
    By Spindle in forum mel PIC BASIC Pro
    Replies: 1
    Last Post: - 23rd June 2005, 08:16

Members who have read this thread : 0

You do not have permission to view the list of names.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Forum Rules