When doing it like that you don't "where" in the cycle you are reading the inputs.
Since you're basically first reading one input then the other it's possible for "the other" input to change state from what it actually was when you read the first input.

Try something like
Code:
Temp = PortB
Direction = Temp.5 ^ Temp.4
This will insure that both bits are sampled at exactly the same time.

/Henrik.