Note that "DEFINE OSC" doesn't set or change the actual clock frequency. It only
tells PBP what to expect. The actual frequency is set by selecting a crystal,
changing the device configuration, setting registers in your program, or (most
commonly) a combination of all of these.

you need to set the OSCCON reg appropriately and the PLL config too to use the 32mhz clock