Hi,
One problem I have is in my lookup table the there is value greater than 796.What happen to this values greater than 796?
You answered your own question in (1) from the Midrange manual:
(1) " If the PWM duty cycle value is longer than the PWM period, the CCPx pin will not be cleared. This allows a duty cycle of 100%".
Any dutycycle value above 796 will result in 100% dutycycle. In other words your SIN-output will be distorted/clipped.

is this two reason cause some part of my wave form not to have a PWM frequency of 5khz? having a long off time
Don't think so, I'd expect it to stay at 100% but I may be wrong. However this isn't correct:
Code:
    if stepcount =36 then stepcount =0
    if stepcount1 =36 then stepcount1 =12
You can't restart at the 12th byte, that will make the second output miss 1/3 of the cycle. You start at the 12th byte to get the correct phase angle offset but you must restart at 0 once it reaches 36.

/Henrik.