Servo's need to be continuously updated about 50 times per second.
Those pauses are screwing with your servo causing it to lose it's mind.