It's frustrating, but there are numerous ways to actually use a slower port once you know where it is.
USB drives should not take forever to transfer files. If yours does, hardware limits and system settings could be holding ...