Masking a number


Closed Thread
Results 1 to 7 of 7
  1. #1
    Join Date
    Jan 2013
    Posts
    64

    Default Masking a number

    Hi,

    I'm programming a PIC in Oshonsoft basic, and have come across a problem.

    When using SINGLE as a variable. When two numbers as a variable should be equal. e,g, a=0 b=0 If a=b then turn on LED. When these numbers should natch they don't. It's something to do with rounding up. Someone told me too quickly for me to understand, how to use MASKING, but I didn't quite follow.

    Can anyone explain how to do this in Basic please, or if possible Oshonsoft basic. I know this isn't an Oshonsoft forum.

    Camerart.

  2. #2
    Join Date
    May 2013
    Location
    australia
    Posts
    2,380


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    if a and b are int type variables and a=0 b=0 then If the statement a==b does not return a true result then that is a bug in Oshonsoft basic {unless Oshonsoft basic is weird}
    however if a and or b are of a float type then a true result is not guaranteed ( at least in c or c++ ) for equals type comparisons .
    digit masking of float data types is an invalid action , only int data types can be meaningfully masked

    since this is a pbp forum (where Oshonsoft basic vagaries not really known) you need to provide more info about your available variable types for a more detailed answer

  3. #3
    Join Date
    Jan 2013
    Posts
    64


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    Hi Richard,

    I tried it in Oshonsoft, using a calculation where 1 then = 0.995. when a and b (both = 1) are tried, the variable keeps rounding up and down, so I'll have to find how to get over the Oshonsoft problem.

    I'm not good at maths, so can you explain what 'digit masking of float types is an invalid action' please. I think that INT = a whole number with nothing after the decimal point??

    C.

  4. #4
    Join Date
    May 2013
    Location
    australia
    Posts
    2,380


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    pbp has no floating point type vars so a discussion about floating point arithmetic on this forum is probably inappropriate

    a value like 0.995 is a float type .
    google "comparing floating point numbers" for an explanation

  5. #5
    Join Date
    May 2013
    Location
    australia
    Posts
    2,380


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    for your second question
    I'm not good at maths, so can you explain what 'digit masking of float types is an invalid action' please
    I think you mean
    I'm not good at maths, so can you explain why 'digit masking of float types is an invalid action' please


    see this link for how a computer stores a floating point number (I would assume Oshonsoft uses ieee754 32 bit floats) masking floats is entirely inappropriate

    http://www.h-schmidt.net/FloatConverter/IEEE754.html

  6. #6
    Join Date
    Jan 2013
    Posts
    64


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    Ok, thanks. C.

  7. #7
    Join Date
    Jan 2013
    Posts
    64


    Did you find this post helpful? Yes | No

    Default Re: Masking a number

    Quote Originally Posted by richard View Post
    for your second question

    I think you mean
    I'm not good at maths, so can you explain why 'digit masking of float types is an invalid action' please


    see this link for how a computer stores a floating point number (I would assume Oshonsoft uses ieee754 32 bit floats) masking floats is entirely inappropriate

    http://www.h-schmidt.net/FloatConverter/IEEE754.html
    Hi R,

    It appears that I'm not good at English either

    Thanks, I'll check the link.

    C.

Similar Threads

  1. Bit masking rules
    By muddy0409 in forum General
    Replies: 2
    Last Post: - 10th March 2009, 17:48
  2. Bit Masking and Or?
    By dbachman in forum mel PIC BASIC Pro
    Replies: 4
    Last Post: - 24th February 2009, 17:19
  3. Is Number Odd?
    By T.Jackson in forum Off Topic
    Replies: 29
    Last Post: - 11th June 2008, 18:53
  4. Masking address bits
    By Demon in forum mel PIC BASIC
    Replies: 12
    Last Post: - 19th November 2005, 03:34

Members who have read this thread : 1

You do not have permission to view the list of names.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts