Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the Maxmind GeoLite Legacy Database Discontinuance affect gdnsd? #161

Closed
ghub2015 opened this Issue Jan 31, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@ghub2015
Copy link

commented Jan 31, 2018

I was just reading this: https://support.maxmind.com/geolite-legacy-discontinuation-notice/

  1. I assume the workaround here is to use GeoLite2

Our GeoLite Legacy databases are being discontinued

We have decided to discontinue our GeoLite Legacy databases effective January 2, 2019. We will stop updating the databases before that date (see “General Information” below), and the databases will no longer be available for download on January 2, 2019. This was a difficult decision for us and we have created this page to provide more information and answer any immediate questions you may have.

  1. But, I was wondering if this part of the GeoLite2 changes may adversely affect gdnsd?

Important Notice for GeoLite2 Users

We have decided to discontinue providing “latitude” and “longitude” with the GeoLite2 City database. Once discontinued, any integration using these keys will no longer return the approximate latitude and longitude of the most granular geolocation returned for the IP queried: postal code, city, subdivision or country. Please take action to update your integration as needed.

Thanks in advance for any insights.

@ghub2015 ghub2015 changed the title Does the Maxmind GeoLite Legacy Database Discontinuance after gdnsd? Does the Maxmind GeoLite Legacy Database Discontinuance affect gdnsd? Jan 31, 2018

@blblack

This comment has been minimized.

Copy link
Member

commented Feb 6, 2018

Thanks for bringing it up, I hadn't noticed this!

For (1) yes, the answer is "use GeoLite2".

For (2), it's a bit trickier!

In code terms, I think we can handle this with at most some minor updates. I honestly haven't tested what the failure mode will be in that case (yet). The failure mode should be sane in the code sense (no crash), but I fear it could be "silent" (as in, loading up the new coordinate-free GeoLite2 City may just result in default/awful mapping of clients that used to map sanely, without much feedback to the administrator about what's going on). We'll probably want to find a way to make this user-friendlier and fail at city_auto_mode operation completely if the database is a coordinate-free "GeoLite2 City". Some documentation updates might be helpful as well.

Beyond making sure the documentation is informative, and that the code fails in an acceptable way, there's not a whole lot we can do other than offer advice in the case of gdnsd deployers who have been relying on auto-mapping via the free GeoLite2 City data. As far as I can tell, there's no plan to drop coordinate data from the commercial GeoIP2 City databases, just the free GeoLite2 City databases.

  • For now, freeze any automatic update of GeoLite2 data so that you don't lose the coordinate info you've already got. This is the simplest temporary solution, and may be an acceptable solution for quite a while, until this data gets too far behind on changes.
  • Switch to manual mapping. It's not all that difficult for a relatively-small count of datacenters, and it also gives you a chance to tweak a few things where geographic distance may not match up with network realities very well. Production example from Wikimedia
  • Purchase a commercial GeoIP2 City license.
  • Create a tool which re-applies coordinate data at the per-Country/State level from old GeoLite2 databases to new GeoLite2 updates to generate a local custom database. The GeoLite2 coordinate data isn't super-precise anyways, and GeoDNS doesn't need super-precise data (country-level is fine for most countries, and state-level in a few larger places). MaxMind offers open source tooling in Perl to read and write GeoIP2 databases. However, I don't think it would be appropriate in licensing or ethical senses for me to package up workaround tooling upstream here and/or host the copied-out country/state -level coordinate data.
@ghub2015

This comment has been minimized.

Copy link
Author

commented Feb 7, 2018

Hi Brandon, thank you for your response to this.

I have to admit I will have to chew on this a bit to figure out the impacts and logical next steps for my setup ... I have been using GeoLite, so I am going to have to do something with this ASAP.

  • BTW, the manual mapping approach (with occasional tweaking as necessary) does look interesting!

  • Also, I definitely agree with your premise regarding the "failure modes" -- it should be sane and informative, IMHO.

  • Let me know if there is a way I can help, even if it is just to help with testing

  • I am running gdnsd in numerous locations across 5 continents for a single zone that gets a decent amount of traffic from a few hundred domains.

Thank you

@blblack blblack closed this in 2415dab Feb 15, 2018

blblack added a commit that referenced this issue Feb 15, 2018

Warn in docs and code about 2019 GeoLite2 changes
All GeoIP2 databases will have some networks mapped to objects
that do not contain coordinates (satellite, anon-proxy, and some
unlabeled unknowns as well), and plugin map statements can overlay
fixed and coordinate-based mapping to varying degrees, so there's
not even a reliable measurement we could use like "X% of
coordinate lookups must succeed" to try to figure this out
precisely and automatically.

So, while it would have been nicer to detect this based on the
data, it's not realistic in terms of complexity, cost, or
accuracy.  Our only reasonable alternative here is to rely on the
database's own metadata description abd build epoch times.

Fixes #161

(cherry picked from commit 2415dab)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.