It was mainly a general question.

So what makes the difference is power consumption?

If power is not an issue then I could use the fastest crystal supported by the PIC and it wouldn't cause an issue?