GeoIp2 replace GeoIP

[mmorel-35]

GeoIP .dat file will not be used anymore, the application shall handle .csv or .mmdb files

We will be discontinuing updates to the GeoLite Legacy databases as of April 1, 2018. You will still be able to download the April 2018 release until January 2, 2019. GeoLite Legacy users will need to update their integrations in order to switch to the free GeoLite2 or commercial GeoIP databases by April 2018.

For more information, please visit our Support Center.

In addition, in 2019, latitude and longitude coordinates in the GeoLite2 databases will be removed.* Latitude and longitude coordinates will continue to be provided in GeoIP2 databases. Please check back for updates.

[kinmenRP]

any news about that?

[neilgierman]

I have started working on one solution. Creating a GeoIP2 plugin that is based off the GeoIP plugin. Feel free to check it out. I have tested it and it works using the GeoLite2-Country database for looking up both IP4v and IPv6 addresses. Comments are welcome.

https://github.com/neilgierman/awstats/tree/GeoIP2

[kinmenRP]

Good. Unfortunately I can't test it, I still use an older version installed from packages manager on a production site, but I hope to see the new working plugin soon. Tnx for your efforts

[neilgierman]

Yes. I have the 1.5 version of GeoIP installed from the production repositories in CentoOS7. I installed the GenIP2 version of geoipupdate from https://github.com/maxmind/geoipupdate which installed to /usr/local/bin, and the mmdb files are downloaded to /usr/local/share/GeoIP so it doesn't conflict with the 1.x versions installed from production repositories.

The one challenge I saw is there is not a GeoIP2 version of the Perl module any of the CentOS supported repositories (EPEL, Atomic, etc), so I had to install the required module directly from CPAN.

[kinmenRP]

I mean, I have awstats 7.6. I've cpan install GeoIP2 but, where's geoip2.pm? It's not in /usr/share/awstats/plugins/

[neilgierman]

To manually install, you would need to copy geoip2.pm (and geoip2_city.pm if you want) into /usr/share/awstats/plugins and also you would need to patch /usr/share/awstats/wwwroot/cgi-bin/awstats.pl with the other changes I made to call the new plugin. Finally you would need to modify your conf file to load the geoip2 plugin(s) pointing to their respective mmdb files instead of the existing geoip plugins (examples are in awstats.model.conf).

Please keep in mind that I have done my testing only on a complete source package based off 7.7 (HEAD of the develop branch). I don't know if there would be other issues if you manually patched a 7.6 installation with my changes.

[kinmenRP]

Yes, but the problem is that after doing cpan install GeoIP2 I can't find geoip2.pm anywhere... Do you mean I have to copy it from /usr/local/share/perl/5.22.1/GeoIP2.pm ?

[neilgierman]

I think there may be some confusion. The CPAN install is only to install the ability for Perl programs to query the new GeoIP2 mmdb databases. The geoip2.pm that I created in my code is an awstats plugin to call the CPAN module with specific instructions on how to get information out of the mmdb files. The flow is awstats.pl (my modified version) -> geoip2.pm (my file) -> CPAN Module (in the default Perl INC path) -> mmdb database.

[kinmenRP]

You're right, I'm in confusion... Sorry some years have passed since the last time I've put my hands on awstats :-P I've copied your awstats.pl into my cgi folder, modified my awstats.conf as described. Already have the new db in the GeoIP folder, now let's see what will happen on the next update... Thanks for your patience

[kinmenRP]

It "works", but the generated page stops at the host info; I see the countries' graph, but no list of countries & cities (excepted one host that shows: The IP address you provided (xxxxxxx) is not a valid IPv4 or IPv6 address at /usr/local/share/perl/5.22.1/MaxMind/DB/Reader/Role/Reader.pm line 39). Also all the sections like visited urls, the errors, browsers, etc. have disappeared: I see that the above pages are generated, but not attached to the "main" page. Any clue about what happened?

[neilgierman]

Yes. I see the issue. It seems (I am still learning awstats) that in the ShowHost section, the resolved DNS name is sent instead of the IP address. I have updated that section of geoip2.pm to do the resolution from name back to IP since the GeoIP2 API only supports IPv4 and IPv6 addresses, not hostnames. I'm sure there is a more efficient way to do this from dnscache but it should at least get you unblocked.

[kinmenRP]

I don't see that error anymore now. However I've just found out that the geoip2 plugin for cities doesn't work, doesn't generate any html file (error 404, file is really missing inside the dir).
By the way I'm still missing all the bottom section of the main file (just as reference, all the part under the cake graph of the Hosts)

[neilgierman]

Yes. Probably best to not load the geoip2_city plugin for now. I checked it in only to save my work but not read for testing just yet. Trying to clean up the country plugin first before I get back to the city plugin.

[kinmenRP]

Ok, I've disabled it

[sherpya]

in the meanwhile, why non making a script that converts geolite2 db in the legacy format?