The speaker is 8Ohms but I would like this circuit to handle both 8Ohm and 4Ohm speakers.

It's very possible there is some noise. I didn't have a 470nF cap to put in series with this resistor so I used the highest non-polarized one I have which is only 220nF. The datasheet does say that a smaller value may cause oscillations. I will be buying some 470nF caps and a few other values on my next order. In the mean time I will try to safely connect my scope to check for high frequency oscillations.

Thanks for you're suggestion of a 0.6W metal film resistor. I will buy a few of those and some higher values just incase / for the future. I would still like to know if there's some kind of calculation I could use to find the exact value though. I want the components to cope easily but I don't want them to be unnecessarily large.