Well it's not a com port issue, it's not even an issue, it's a hidden feature, and it's probably like that for reliability rather than for speed

If you want to speed up a bit the process, you may uncheck some Verify options in MicroCode Loader option.

This said, the COM setting in MCSP is for ICD, not bootloader.