Sometimes I must laugh at myself. I know I read the third pot was to adjust the speed, but I still keep thinking about it as the more you move the joystick, the faster you go.

OK, re-read the original post and have my brain going the right direction.
grab the analog and determine if the X and Y are above or below center and set a direction pin to 1 or 0.
that is what threw me. center point can't determine direction, because if you are right of center and you need to move left, joystick will still be right of center. Direction needs to be determined by pot vaule being higher or lower then the last value.

So controlling it this way, I would not worry about "trimming the speed" and still use the third pot for Z. The speed will be controlled by how fast you move the joystick. Something to keep in mind here, with 6' of travel, moving 3' will take 3.6 seconds (@ max jog of 10"/sec). So for realtime control, you will have to move the joystick very slowly.

now for some math:
to move 72 inches you need to make 72/2.375= 30.315 rev of the motor
at 1440 pulses per rev thats 30.315*1440=43653.6 pulses.
now divide 43654/1024=42.63 pulse resolution.

So for each bit of the A/D value you have to output 42.63 pulses for the 6' travel.
and for the 4' axis you need to output 28.42 pulses.

I would approach it like this: pulse output will always be at 6000 pps. I would set up a timer to run so it takes 166uS to roll over. (@ 4Mhz clock, you could use an 8 bit timer preloaded with 90 for 166uS)
on each timer rollover, reload clock,check position and toggle if necessary.

Now for the pulse streams:

To make things much easier on the coding, I would make home be a corner so all my positions are in the positive X,Y direction.

Multiply the A/D by 42.63 for the 6' and 28.42 for the 4' axis. this is your target position in pulses. Current position is a counter which is keeping number of pulses sent from the time you said everything was zero (homed). If current pos = target pos then no toggle. If current pos<target then toggle with dir=1 If current pos > target pos then toggle with dir = 0.
(Of course dir may be backwards, all depends on your setup). If you decide to use the third pot as Z, just fill in the travel (in inches) above for the Z multiplier.

For your drift control and to accomdate for hand jitter, maybe have a control check to say, no movement unless A/D value changed by more then 2 points. of course that will decrease resolution, but right now thats all I can think of.

Of course you could just get the plugin Henrik is talking about, But to me there is value in having a way to move the machine with out the control. Of course, Mach will not know the machine is moving so you won't be able to use this as a jog device per say. I assume you will unplug the puter input to the drives and plug in your "jog Stick"