How are you measuring the frequency? Its very unlikely that noise will be coming from batteries. Usually its the other way around, -power supplies have noise from their rectification of AC / regulation etc and batteries give a nice smooth output.