The only time you should need to worry about bit 5 is when bit 6 = 1 (12 hour mode).

When bit 6 = 0 (24 hour mode) you do not have to remove or mess with bit 5. Bit 5 is part of the number [B]not[B] an am/pm bit.

24Hr mode

Time is 11:00 hours, hours = %00010001 notice bit 5 is a zero which means the 2nd 10s is clear, and bit 4 (1st 10s) is a one which is set.
Time is 21:00 hours, hours = %00100001 notice bit 5 is a one which means the 2nd 10s is set, and bit 4 (1st 10s) is zero which is clear.