When I say swap the servos, I was meaning physically. If the up/down is used for the left/right, does the left/right action still work.

I I was wanting to see the 7805s in parallel.

But... I still think the problem is power.
The one 9volt 450ma source does not provide enough power for tho whole setup, I think.

That is 450 ma total, yes at 9 volt, start adding two or more 7805s to drop to 5 volt and try to run motors... the amps are just not available.

Do you have another source?