Put a 1uF Electrolytic... or even a 10uF... across the 0.1 Capacitor.

It's a Battery Charging circuit - you don't need instantaneous results. If your filters time period is a few hundred mS then goodbye to any 500Hz noise! Make sure you haven't got any noise on your 5v line - that could be throwing +VRef as well.