Website: Use geoIP to configure download mirror when downloading apps from website

[JedMeister]

[update Feb 2021] The main issue has been worked around for now, so this issue is being repurposed to improve the current workaround.

---

As of Jan 2021 Chrome blocks "mixed content downloads". Because our mirror network is hosted via http, but out main site is https, this means that when using Chrome, the appliance downloads are (silently) blocked by default.

To workaround this, as now noted on the download page, Chrome users will need to right-click the "direct link" and select "Open link in new tab|window". It is unclear whether Edge or Safari users will be affected or not. Firefox users should be unaffected.

We hope to find a proper resolution to this but until then, I am adding a note to the download page for Chrome users.

[bomdia]

If you can't secure the mirror software itself, you can add a transparent proxy software like nginx and secure the connection here all download link to function with chrome can't be http if the page is https as you stated

[frankwas]

Hi @JedMeister . I have been using turnkeylinux images for years and it's saved me hours and hours of time so a big shout out from me to say thank you! I also wanted to offer you my services for free. I can host unlimited storage space with an SSL certificate for your mirror if you wanted just to say thanks for the great work. Let me know. I have a hosting company and can provide you with direct access to the mirror.turnkeylinux.org or whatever domain you would want on that. If you want more details or are interested, please reply let me know.

[JedMeister]

@bomdia - The existing mirror network are all provided by hosting companies or universities, so we have no direct control over them. We have geo-ip & round robin DNS that points mirror.turnkeylinux.org to whichever mirror IP seems the most relevant, none of which have a "real" turnkeylinux.org domain. So securing the current setup via HTTPS is a little problematic. Your proxy suggestion should work around the issue, although somewhat defeats the point of using the mirror network in the first place.

@frankwas - Thank you for your kind offer! TBH, I haven't yet put a whole lot of effort into addressing this issue (I've been trying to focus on a refresh of the library before I start work on developing v17.0) but I should get onto it...! As I noted above, the load is currently distributed (and somewhat optimised depending on location) so I'm not really sure exactly how we go from there, but regardless, I'd love to chat with you further regarding your offer. Could you please email me via jeremy AT turnkeylinux.org. Thanks.

[JedMeister]

I've tweaked the website downloads page so that it now links directly to the mirror providers' https URLs (sidestepping the round robin DNS for mirror.turnkeylinux.org for now). It randomly picks one of the high speed mirrors to download from and as it uses an https URL, the download now "just works".

This should solve the immediate problem, but I'd actually like to rejig this with some geoIP goodnesss too. Perhaps there is also some value in trying to gauge the user's connection bandwidth and use slower mirrors for slower connections?! OTH I imagine that most of our users would not notice the difference downloading from a 1Gbps mirror a 20Gbps. So perhaps I just add them all on there and go from there?

I'd be interested in any further thoughts...

Also @frankwas - I'd still love to chat when you get a chance! 😄

[bomdia]

Mmm as i know nginx is able to load balancing server with geo ip too (and optional upgrade http to s on the fly), i found this maybe can help you https://www.cleverti.com/blog/how-to-build-a-load-balancer-based-on-user-s-location-with-nginx-and-geoip2

[bomdia]

An optimal approach to the problem can be DNS load balancing for macro area and then nginx load balancing for individual mini zone (round robin or geoip)

[JedMeister]

@bomdia - Thanks for your input, they're cool ideas. FWIW Nginx's geoIP module is pretty cool. I run a (personal) local server and block non-local traffic (which is mostly bots trying to exploit common vulnerabilities) and it works really well.

Unfortunately though I don't have any control over any of the mirrors (obviously, we control the master mirror, but it doesn't have good enough bandwidth to host directly). I'm not really keen to set up a new additional server if I can avoid it (although I guess I could do this on the existing website server?!). Also, whilst I could get DNS records updated, I don't have direct control over that myself and it's easier if I can just do what I need to do via the website if possible...

At this point, I'm just thinking of using some custom PHP embedded within the website (the website is Drupal) to do the geo-ip to redirect to the most relevant mirror URL.

[bomdia]

Mmm i searched for existing drupal plugin and are all old and unsupported, and php is slow for this thing, if you already use nginx for website you can configure a location where the user is redirected to the correct mirror based on geo ip so than you link the special location and nginx do the correct redirect

[bomdia]

Something like .../mirror/abcddownload that nginx translate to mirror.domain/abcddownload

[JedMeister]

Great ideas @bomdia. Unfortunately though, our webserver isn't running Nginx. Although we have been considering migrating to it. Regardless though, we'll need to wait until we rebuild the webserver. It's on the backlog, but the backlog list is long...