Make PIC simulate binary up/down counter - how to calculate how fast can it count?

[richard]

some thoughts, in my experience decoding quadrature signals that way usually miscounts when small back forward movements and or freq direction
changes occur. plus you only get a quarter of the possible resolution , edge detection on both channels is a better method
i might add your method will be intolerant to any noise [contact bounce] too
you could try these ideas and maybe tweak a little more speed.

PORTB = counter && 255 is not really correct , && is a logical AND you need a BITWISE AND
PORTB = counter & 255
or
PORTB = counter.lowbyte
or since its an 8bit chip
PORTB = counter , would do but may get a warning about low byte value being used

since the count can only inc/dec by 1 count you can simplify to this

if pin_B = 1 then counter = counter + 1
if pin_B = 0 then counter = counter - 1
if counter.15 then counter = 359
if counter > 359 then counter = 0

bit9 can be simplified to this

bit9=counter.8


making

if pin_B = 1 then counter = counter + 1
if pin_B = 0 then counter = counter - 1
if counter.15 then counter = 359
if counter > 359 then counter = 0
bit9=counter.8
PORTB = counter.lowbyte