Since you are averaging then it does not make any difference if you do your calculation on the whole number or on the difference from the average. Flag if the difference is negative, so you will know that it will be a subtraction and work with the difference, that being smaller will never overflow your word.

Cheers

Al.