This should work.
Code:
    DEFINE OSC 8
    TRISC.2 = 1  ' CCP1 pin stays input intil PWM on
    TRISB.5 = 1  ' button input (external pull-up with switch to ground)
    Push1 var PORTB.5
    
    PR2 = 15 'Load Period Register
    CCPR1L = 8 'Set 50% duty cycle
    CCP1CON = %1100 'PWM mode
    T2CON.2 = 1 'Start TIMER2
    
Main:
  TRISC.2=Push1 ' if RB5=0 CCP1 pin is an output, PWM=on  
  GOTO Main     ' if RB5=1 CCP1 pin is an input, PWM=off
The PWM module is still operationg so you shouldn't have any glitches in the signal, and
you only need one button to control it.

When the CCP1 pin is switched to an input, the PWM signal is disconnected. When it's
flipped to an output it's back on.