Calculation error within a for next loop


Closed Thread
Results 1 to 3 of 3

Hybrid View

  1. #1
    Join Date
    Nov 2005
    Location
    Bombay, India
    Posts
    970


    Did you find this post helpful? Yes | No

    Default Re: Calculation error within a for next loop

    Typical problem of word size and technique.

    Take i = 220
    Now, plug into the formula
    220-179 = 41
    41 * 1623 = 66543
    which ends up in the variable as 66543-65535(0FFFFH) = 1008

    Now, you do the rest of the math and end up with the 1%

    You need to re-factor the way you calculate the problem.

  2. #2
    Join Date
    Sep 2006
    Posts
    747


    Did you find this post helpful? Yes | No

    Default Re: Calculation error within a for next loop

    aaarg yep correct thank you

    K

Members who have read this thread : 0

You do not have permission to view the list of names.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts