Hi,
Have you had a look in the PBP manual?

It says: The LOOKUP statement can be used to retreive values from a table of 8-bit constants.

So, for LOOKUP you have to stick to 8bits in the table. You can do that by either use prescaler of 16 or stick with a prescaler of 4 and multiply the retreived value by 4 before putting it in the dutycycle register. Obviosuly both options reduces the effective resolution.

If you need more resolution then take a look at the LOOKUP2 statement, it's also described in the manual.