Create auto-updater & auto-downloader for GeoIP databases #3456

Closed
diosmosis opened this Issue Oct 18, 2012 · 15 comments

4 participants

@diosmosis
Piwik Open Source Analytics member

One of the improvements in: #3442

The goal of this ticket is to create an admin function that will automatically download a GeoIP database after Maxmind updates it.

  • It should support free & commercial databases.
  • There should be a way to download a free database to get started using GeoIP as quickly as possible.
    • The download should display a progress bar.
  • This download should take place during installation if possible.
@halfdan
Piwik Open Source Analytics member

Can't that just be done via a Scheduled Task?

For the GeoLite City database the task would only have to run on the first Wednesday of each month. (The database is updated on the first Tuesday of each month - http://www.maxmind.com/en/geolite)

That of course leaves out the commercial database.

On a related note: Maxmind offers a Web API for Geolocation that could potentially be added as a provider (http://www.maxmind.com/en/web_services), although I think it would hurt the performance of tracking when calling the Maxmind API for each request.

@hpvd

Replying to halfdan:

On a related note: Maxmind offers a Web API for Geolocation that could potentially be added as a provider (http://www.maxmind.com/en/web_services), although I think it would hurt the performance of tracking when calling the Maxmind API for each request.

yes using web api would be very interesting. Some words on this topic are also posted in ticket #3442

@diosmosis
Piwik Open Source Analytics member

(In [7550]) Fixes #3456, added updating scheduled task for GeoIP databases w/ manager UI and easy-install for GeoLiteCity.

Notes:

  • Added new PEAR lib Archive_Tar and new Unzip implmentations for .tar.gz, .tar.bz2 & .gz files.
  • Modified Http class to allow use of Range HTTP header.
  • Added ability to download file in chunks to Http class.
  • Moved GeoIP admin page styles to separate CSS file.
  • Allowed monthly scheduled tasks to specify day of week to run on.
@diosmosis
Piwik Open Source Analytics member

(In [7576]) Refs #3456, add extra security for GeoIP auto-updater/downloader & fix a couple bugs in the code. Also made the UX a tiny bit more robust.

@diosmosis
Piwik Open Source Analytics member

(In [7577]) Refs #3456, various UX improvements to GeoIP updater/auto-downloader.

@diosmosis
Piwik Open Source Analytics member

(In [7578]) Refs #3456, remove downloaded files on all exceptions, even sanity check.

@diosmosis
Piwik Open Source Analytics member

In 4169d90: Refs #3456, add redundant checks of GeoIP database wellness to GeoIP auto updater.

@diosmosis
Piwik Open Source Analytics member

In 91098c7: Refs #3456, only save one broken file when doing redundant checks & add unit test for checks.

@anonymous-piwik-user

In a2e7a87: Refs #3456, fixing build.

@anonymous-piwik-user

In 62da3a2: Refs #3456, fixing build.

@anonymous-piwik-user

In 5aa918f: this should help the build refs #3456

@anonymous-piwik-user

In 8905b08: Refs #3456, fixing build.

@anonymous-piwik-user

In 8adc8cb: Refs #3456, fixing build.

@anonymous-piwik-user

In 7f9332d: this should help the build refs #3456

@diosmosis
Piwik Open Source Analytics member

In 34d65f2: Refs #3456, if geoip updater has not been run, display inline help saying so in geoip updater setup.

@diosmosis diosmosis added this to the 1.10 - Piwik 1.10 milestone Jul 8, 2014
@diosmosis diosmosis self-assigned this Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment