Another thing to bear in mind is that if you plug the USB Serial Adapter into a different USB Port (either on the PC or a hub) it will get "reinstalled" as a completely different serial port hence the high com Port numbers some people get upto.

You also cant uninstall a USB Serial port if it isnt physcially connected but you can steal the Com Port number that it was occupying in Device Mangler