Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maxmind IP lookup database cannot be downloaded anonymously anymore #8275

Closed
escopecz opened this issue Dec 31, 2019 · 16 comments
Closed

Maxmind IP lookup database cannot be downloaded anonymously anymore #8275

escopecz opened this issue Dec 31, 2019 · 16 comments
Labels
bug

Comments

@escopecz
Copy link
Sponsor Member

@escopecz escopecz commented Dec 31, 2019

Bug Description

Mautic uses Maxmind dataset as the default IP lookup database. It cannot be downloaded directly since yesterday. Users must register to get a license key. We'll have to update the Maxmind config option to allow users to set the key.

Starting December 30, 2019, we will be requiring users of our GeoLite2 databases to register for a MaxMind account and obtain a license key in order to download GeoLite2 databases.

https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

Q A
Mautic version Any
PHP version Any
Browser Any
@npracht npracht added the bug label Jan 6, 2020
@marioguima
Copy link

@marioguima marioguima commented Jan 6, 2020

While we don't have the automatic update I solved it like this

Let's go step by step

  1. Create a maxmind account
    just click here
    https://www.maxmind.com/en/geolite2/signup

  2. Create a password
    For this you need to click on the link you will receive in your email (they will send you an email, just click on the link where it is written)
    "you must first create a password here" you will click on the word "here" in your email message to create a password

  3. You will be logged into your maxmind.com account by entering your email and the password you just created

  4. Sign in to your account by clicking the icon at the top.
    Will open a screen with a menu on the left side
    Click on the "Download Files" option

  5. Download the file that has the ID "GeoLite2-City" (to date was on the third line)
    Click on GZIP
    (will download a file named GeoLite2-City_20191231.tar.gz)
    note that 20191231 will change every week because this is the last ips list update date
    This list is updated every Tuesday.

  6. Use an unzip (I use 7zip)
    it will open another .tar file
    (GeoLite2-City_20191231.tar)

  7. Extract again with 7zip
    Will create a folder (GeoLite2-City_20191231)

  8. Send the file GeoLite2-City.mmdb (which is inside the folder) to the server where your mautic is installed.
    Put the file in the folder
    YOUR-MAUTIC / app / cache / ip_data
    Where YOUR-MAUTIC is the folder where you installed mautic

Ready :)

After that (if you want to check if it's up to date)
Go to settings in your mautic and see that the list of ips will have the update time

@quangmai911
Copy link

@quangmai911 quangmai911 commented Jan 11, 2020

Hi,

I followed your helpful tips. However, I am still stucked at the last step. Put in the folder. I put the file exactly where the path shown with File Manager in Cpanel and grand the permission for read/write etc... Put the issue can not solve yet. Any ideas please. Thank you so much.

Quang

  1. Send the file GeoLite2-City.mmdb (which is inside the folder) to the server where your mautic is installed.
    Put the file in the folder
    YOUR-MAUTIC / app / cache / ip_data
    Where YOUR-MAUTIC is the folder where you installed mautic

@ekkeguembel
Copy link
Sponsor

@ekkeguembel ekkeguembel commented Jan 11, 2020

Disable the cronjob that tries to update ("app/console mautic:iplookup:download"), or it will overwrite (delete) the manual upload next time it's executed.

@quangmai911
Copy link

@quangmai911 quangmai911 commented Jan 12, 2020

"app/console mautic:iplookup:download
Thanks for the reply.
I attached the photos for your references. What I did so far.

  • Disable cronjob
  • download & upload through cpanel the latest GeoIP file included license from my download account page from maxmind website.
  • grant the file with read/write/modification
  • try to fetch it again, still error.
    Any other ideas? Thanks so much
    screencapture-maxmind-en-accounts-161904-license-key
    screencapture cronjob
    screencapture frontend
    screencapture-email-muenvironmental-org-nz-s-config-edit-2020-01-12-10_28_33

@atulcj
Copy link

@atulcj atulcj commented Jan 15, 2020

Starting with @marioguima's comment (above), I discovered "Automatic Updates for GeoIP2 and GeoIP Legacy Databases".

  1. Installed the GeoIP Update program as described on the page
  2. Created an account and obtained license keys
  3. Created a geoipudate configuration file as per the "partially pre-filled configuration file"
  4. Then using the command "sudo -u daemon geoipupdate -f /usr/local/etc/GeoIP.conf -d /path/to/my/mautic/install/app/cache/ip_data downloaded the geoip database
  5. Confirmed that this worked ok
  6. Replaced my geoip update cron with the above command to periodically update the database

Fingers crossed, hopefully this should help me till the team finds a solution

@atulcj
Copy link

@atulcj atulcj commented Jan 16, 2020

"app/console mautic:iplookup:download
Thanks for the reply.
I attached the photos for your references. What I did so far.

  • Disable cronjob
  • download & upload through cpanel the latest GeoIP file included license from my download account page from maxmind website.
  • grant the file with read/write/modification
  • try to fetch it again, still error.
    Any other ideas? Thanks so much
    screencapture-maxmind-en-accounts-161904-license-key
    screencapture cronjob
    screencapture frontend
    screencapture-email-muenvironmental-org-nz-s-config-edit-2020-01-12-10_28_33

@quangmai911 - In your screenshot the mmdb file seems to be 0kb

@YosuCadilla
Copy link

@YosuCadilla YosuCadilla commented Jan 19, 2020

Hi guys, and thank you @atulcj for the solution, after trying different options, your steps helped me solve this issue.

I have written a step by step guide for less technical people to be able to fix this ASAP:

https://mauteam.org/mautic/mautic-admins/solved-maxmind-geolite2-database-not-updating/

@lukassykora
Copy link
Contributor

@lukassykora lukassykora commented Jan 20, 2020

Solved in this PR: #8337. Then you can add license key in Mautic configuration.

@escopecz escopecz added the ready-to-test label Jan 20, 2020
@luguenth
Copy link
Contributor

@luguenth luguenth commented Jan 20, 2020

Whoops, while I did my PR, @lukassykora published his 💩

@YosuCadilla
Copy link

@YosuCadilla YosuCadilla commented Jan 20, 2020

Well, it's great to see everyone is chipping in in their own ways!

@dennisameling
Copy link
Member

@dennisameling dennisameling commented Jan 28, 2020

Just merged #8337. This issue will remain open for now for the reasons I described in #8337 (comment). Further follow-up will take place at a later stage, either in this issue or in a new issue/forum thread. Thanks everyone for the contributions so far! 😄

@dennisameling dennisameling removed the ready-to-test label Jan 28, 2020
@khaoua
Copy link

@khaoua khaoua commented Feb 25, 2020

@YosuCadilla If a user procedure exist via PUTTy that would great

@YosuCadilla
Copy link

@YosuCadilla YosuCadilla commented Mar 3, 2020

@khaoua
Copy link

@khaoua khaoua commented Mar 4, 2020

@YosuCadilla ; thanks a lot, I had a look, but I struggle to find where this can be done via PUTTy in windows10, it looks like it is only a unix/linux process with sudo commands, I might be wrong, I'm sure you can let me know which part can taken in your post for a process via PUTTy, can you let me know, that would be great ;-)

@RCheesley
Copy link
Sponsor Member

@RCheesley RCheesley commented Jul 29, 2020

@dennisameling in view of the comments on the PR which has been merged for some time I will close this old issue down and create separate issues for the license key being added to the CI tooling and providing a better message when running the cron without the license key in place.

@RCheesley
Copy link
Sponsor Member

@RCheesley RCheesley commented Sep 9, 2021

Anyone coming here from a web search please check the documentation here for how to set up Maxmind: https://docs.mautic.org/en/setup/maxmind-license

Make sure you use the correct syntax, which has changed from what was needed in the past.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug
Projects
None yet
Development

No branches or pull requests