Hi,
If I'd venture a guess it would be that with a 360CPR encoder and 4x decoding the counter variable will "tick" 4 counts per degree but your math assumes the resolution is 0.1 degrees.

As a simple workaround try multiplying your count value by 2.5 before you enter your display routine.
Code:
' Here is were I struggle.....
enc_counter = enc_counter */ 640   ' Multiply by 2.5
'*********************************************************************************
lcdout $fe,$C0, SDEC2 enc_counter/10, ".", SDEC2 enc_counter//10     	'display enc_counter value
'*********************************************************************************
Obviously, the encoder only provides 0.25 degree resolution and no math can change that.

If it doesn't work properly then strip out the formating and display the raw value (pre or post multiplying with 2.5) and tell us between what values it ranges.

/Henrik.