OK so this is how I am now setup ...
http://www.rentron.com/serkey16.htm
notice there are no pull-ups on the rows.
You don't see any pullup because they are using internal pullup on portB

See the instruction that activate the weak pullup in the code.

OPTION_REG.7 = 0 ' Enable PORTB pull-ups

If you use different port then you have to use external resistors.

Al.