Resolution integer math assistance?


Closed Thread
Results 1 to 4 of 4

Hybrid View

  1. #1
    Join Date
    Dec 2005
    Location
    Salt Lake City, Ut, USA
    Posts
    108


    Did you find this post helpful? Yes | No

    Thumbs up

    Thank you so much DT!

    I'm working on the rest of the code as we speak.
    I think I get the following...

    Ang = AngCount * 40909
    Ang = DIV32 10000

    DIV32 effectively "slides" the decimal point so the final number is 5 places max?
    I think I got tripped up thinking ALL math had to fit in 16 bits.

    Ang = AngCount * 40909 <<<< potentially way bigger than word sized

    My bad! You are so gracious for helping! Thank you a bunch.

    Chris

  2. #2
    Join Date
    Jul 2003
    Location
    Colorado Springs
    Posts
    4,959


    Did you find this post helpful? Yes | No

    Default

    No prob Chris,

    For a lot of things, you really don't need Floating Point math.
    When multiplying and dividing, the decimal point doesn't matter (until you get the final answer).

    For X * 0.040909 ...
    Just drop any leading 0's and keep anything that will fit with-in 16-bits (<=65535). Anything more, and you have to drop digits to the right.

    At the end, you'll remember that the decimal point shifted 6 places to get that.
    Which leaves 40909. Multiply that times the number you are converting.

    Since the number had 6 decimals, and you want 2 decimals left in the result ... then there are 4 decimals unaccounted for.

    Ang = DIV32 10000

    Dividing by 10,000 shifts it those 4 decimal places. 1 decimal for each 0.

    And yes, it's WAAYYY bigger than 16-bits.
    But the multiplication result is temporarily stored in PBP's system variables as a 32-bit number before using DIV32. So a larger than 16-bit variable is not required.

    hth,
    DT

Similar Threads

  1. High Resolution Timer & Speed Calculator
    By WOZZY-2010 in forum Code Examples
    Replies: 4
    Last Post: - 7th February 2010, 17:45
  2. Replies: 5
    Last Post: - 28th May 2008, 11:20
  3. PBPL Math...new math takes more cycles...Always?
    By skimask in forum mel PIC BASIC Pro
    Replies: 2
    Last Post: - 10th February 2008, 11:22
  4. Help!! Integer Math?
    By scottl in forum mel PIC BASIC Pro
    Replies: 4
    Last Post: - 2nd July 2006, 15:49
  5. Help!! Integer Math?
    By scottl in forum mel PIC BASIC Pro
    Replies: 1
    Last Post: - 29th June 2006, 14:47

Members who have read this thread : 0

You do not have permission to view the list of names.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts