server selection should not be based solely on geographic proximity

[critzo]

Based on numerous support inquiries over the years, I'm adding this issue to discuss in upcoming refactoring of mlab-ns. This is a longer discussion, but I would like to see us approach it at our next off-site meeting.

Routing users to the closest geographically proximate server is a catch all method that ignores users' actual paths to content. Specifically it produces inaccurate measurements in countries where peering and transit markets are not well mature, or where the paths to most content are not in country, but abroad.

Consider the case of this user in Accra, Ghana, as described below:

I have been using both your website and the chrome extension to run the test and my point was that using the geographically closest server is not representative of how the Internet actually works. The latency to the server in Accra (~180ms) is currently higher than the latency to the server in Washington DC (~160ms) when Paris is only ~76ms.
Since the majority of users will not bother changing the server, the aggregated data for operators in guinea will obviously not be able capture the Internet users' actual experience. I would therefore advice to switch the default server from the closest geographically to the closest considering the number of network hops.

It would be great to discuss the how mlab-ns ought to function as we begin to think about refactoring. Some ideas:

• user is directed to the server with the lowest # of hops between then
• user is directed to the server with the lowest ping time

Additionally, I'd also think it relevant to discuss server placement with respect to where content on the internet lives. Our model fits as a proxy for where some content on the internet lives, but ignores CDNs for example.

[pboothe]

We should talk about this at the Jan retreat. This is a lot of work, but it is important.

[bassosimone]

One thing that comes to mind is to request for more than one server, connect to all of them in parallel, measure the latency, and choose the one with the best latency (it's like happy eyeballs, except that it's done for different servers, not protocols). OTOH, the problem that may occur with this suggestion is that J. Random User may be less likely to discover performance issues with another M-Lab pod (and hence with some place where some content is) because the best server in terms of latency is always used. To some extent, this is probably a client choice. That is, it depends on the user story: you are measuring the performance for what purpose?

[pboothe]

Right. It gets complicated, and is wrapped up with inferring user intent and need in a low-information context with consumers who are themselves not knowledgeable enough to know and with internet access and transit providers who may have overly-fitted themselves to locally-bad market conditions. Lowest-latency is also bad, because it means that access ISPs never get punished for having bad peering to the Internet.

I have a whole doc I'm working on, but the basic answer is: however complicated you think it is to open this can of worms, I would like to assure you it is far far worse than that. It's important to think about, but let's not fool ourselves into thinking this is a "tweak" kind of effort.