Latency tests don't work as intended (mirror auto-selection)

[nroi]

The latency tests are intended to select low-latency mirrors, so we would expect users from the USA to get mirrors from America, users from Australia to get users from Australia, etc. Apparently, this is not the case: See #28

My mirror auto-selection seems not to have worked too well :( Perhaps I can help because I'm a US user(?). I find I have mirrors from Sweden and Australia in my auto-selection list, as well as United States and Canada

[nroi]

The problem may be related to CDNs like cloudflare. If the connection is not made to the actual mirror, but just the CDN, then the latency will obviously be very low. This doesn't necessarily mean that this is an apt mirror for this particular user: Maybe the CDN was just able to deliver the core.db file relatively fast because it was already cached, but then turns out to be very slow when downloading a larger file that is not frequently accessed.

One option would be to simply exclude all mirrors that use cloudflare: https://www.cloudflare.com/ips/. In our application, the primary_ip function can be used to determine the IP of the remote server. However, we would first have to check how many mirrors we would be excluding this way.

[nroi]

Another thing to consider:

• Many mirrors don't support IPv6, but the current default setting is to exclude all those mirrors.
• We currently exclude all mirrors whose "score" is below 2.5 (the score provided by the JSON API at https://www.archlinux.org/mirrors/status/json/), but we haven't verified if this score is really a meaningful measure. At first glance, the score value seems to be biased towards European servers, which may be one of the reason why some users from the US or Australia get mirrors from Europe.

[nroi]

A few improvements have been made in this regard, so if you previously had issues, then please upgrade Flexo, delete the /var/cache/flexo/state/latency_test_results.json file and restart Flexo to see if things have improved. If they haven't, please comment.

Improvements so far:

• Cloudflare mirrors are excluded (see Exclude mirrors returning cloudflare pages from latency tests #31 for more details).
• Users can specify a country whitelist, check the allowed_countries setting in flexo.toml for more details.
• The mirrors_auto.ipv6 setting had its default value changed from true to false: This should be useful if there are very few mirrors with IPv6 support in your region.

I just tested this by running flexo on a server located in Australia, and on another server located in the US, and the results where as expected: The server from Australia only uses mirrors from Australia, and the server located in the US only uses mirrors located in the US and Canada.