Table 1.6 pg 11 shows POR/RST value of IODIRA and IODIRB is %11111111. NOT %00000000. Setting the dir to 1 makes the pin an input.

Maybe you were looking at the next 2 registers? IPOLA and IPOLB?