I assumed the following that either... You will have a Pull-Up resistor on your Button pin (button pulls pin down to 0v), or you will have switched-on the internal weak pull-up's for PortB. If that button pin is floating, you will have unpredicatable results.