in another hand, lower your crystal is, more you need to multiply it's value with the PLL... wich in some case may be a killer. PLL may and will drift with temperature change...more you multiply more it will drift.

As now i didn't experiment too much problem with 20MHZ but i doubt one day i'll have. A real 48MHZ osc should be better but will probably never work good on a bread-board. Nothing is perfect.