Where your order actually spends its time
Retail latency discussion is almost entirely about internet speed, which is the least interesting segment of the path.
Break the journey into pieces: your machine deciding, your connection to the broker, the broker's own systems processing and risk-checking, the broker's link to the venue, and the venue's matching. For most retail setups the middle segment — what happens inside the broker. Is larger and more variable than the part you can buy your way out of with a better connection.
Which means upgrading your internet to shave milliseconds off a path where your broker's risk check takes considerably longer is optimizing the wrong segment.
Brokers themselves differentiate their gateways by speed tier, which tells you they consider that segment worth charging for. The framing worth keeping is that the broker is part of the latency picture rather than a neutral pipe.
Practical takeaway: if latency actually matters to your strategy, measure your own round trip end to end before you buy anything. If it doesn't matter, stop reading about it.