Adrian,

The placement of the capacitors isn't to critical at 4 MHz. Good design would tell you to keep the leads short. Are you certain of the values of the capacitors? The oscillator won't start if you accidentally use a value to much higher. 22pf is given as a good starting point. You might try 10 pf or so since the crystal does oscillate with no capacitors in the circuit. Look for solder bridges around the capacitors while you are at it.

Terry