Calculation Problem - value goes to zero after 65


Results 1 to 15 of 15

Threaded View

  1. #4
    Join Date
    Jul 2003
    Location
    Colorado Springs
    Posts
    4,959


    Did you find this post helpful? Yes | No

    Default

    That's good Ingvar, got me thinking.

    But, since we're going for accuracy too.

    1 Nautical Mile = 6076.11549 Feet
    1 Mile = 5280 Feet

    So the conversion is 1.150779449 (6076.11549/5280), and the number we need is [0.150779449 * 65536] or 9881.482. It's closer to 9881 than 9882

    But the biggest problem is with the integer nature of the calculation. Most of the added accuracy is lost in the Low Word of the ** multiplication..

    For instance, at 45.0 knots, the MPH is 51.78 MPH (45 * 1.150779449) which is closest to 51.8 MPH.
    However, either of our previous formula's will only show 51.7 MPH.

    Here's another way that does the same thing as the **, but also allows you to round the result according to the low Word of the multiplication.
    Code:
    Dummy = Dec_speed * 9881
    Dummy = R0                               ; Get High Word of Multiply result
    IF R2 > 32768 then Dummy = Dummy + 1     ; Round to nearest .1 MPH
    Dec_speed = Dec_speed + Dummy
    This will hold the result to +/- 1/2 Least Significant Digit.

    Best regards,
       Darrel
    Last edited by Darrel Taylor; - 28th June 2005 at 18:23.

Similar Threads

  1. Problem to compile my program
    By wagner in forum Off Topic
    Replies: 5
    Last Post: - 7th July 2008, 20:17
  2. USART Problem , but don't know where, in pc? or in PIC?
    By precision in forum mel PIC BASIC Pro
    Replies: 0
    Last Post: - 15th July 2007, 08:12
  3. Microcode Studio 18f2455 problem?????
    By volkan in forum mel PIC BASIC Pro
    Replies: 11
    Last Post: - 21st May 2007, 21:04
  4. calculation problem
    By nicolelawsc in forum mel PIC BASIC Pro
    Replies: 2
    Last Post: - 31st March 2006, 15:23
  5. 1 slave 1 master 1 MAX232 1 problem ?
    By SuB-ZeRo in forum mel PIC BASIC Pro
    Replies: 19
    Last Post: - 31st July 2005, 22:59

Members who have read this thread : 0

You do not have permission to view the list of names.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts