Skip to content

Commit

Permalink
NTOPng MaxMind GeoLite2 DB download fix. Issue #9211
Browse files Browse the repository at this point in the history
  • Loading branch information
@vktg
vktg committed Apr 28, 2020
1 parent 8e025d2 commit 85cdc14
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 24 deletions.
2 changes: 1 addition & 1 deletion net/pfSense-pkg-ntopng/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

PORTNAME= pfSense-pkg-ntopng
PORTVERSION= 0.8.13
PORTREVISION= 3
PORTREVISION= 4
CATEGORIES= net
MASTER_SITES= # empty
DISTFILES= # empty
Expand Down
32 changes: 15 additions & 17 deletions net/pfSense-pkg-ntopng/files/usr/local/pkg/ntopng.inc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -238,19 +238,15 @@ function ntopng_create_datadir() {
mwexec("/usr/sbin/chown -R nobody:nobody /var/db/ntopng");
}

function ntopng_update_geoip() {
function ntopng_update_geoip($key) {
global $config;
$fetchcmd = "/usr/bin/fetch";
$geolite_city = "https://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz";
$geolite_city_v6 = "https://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz";
$geoip_asnum = "https://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz";
$geoip_asnum_v6 = "https://download.maxmind.com/download/geoip/database/asnum/GeoIPASNumv6.dat.gz";
$geolite_city = "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key={$key}&suffix=tar.gz";
$geoip_asnum = "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key={$key}&suffix=tar.gz";
$output_dir = "/usr/local/share/ntopng";

mwexec("{$fetchcmd} -o {$output_dir} -T 5 {$geolite_city}");
mwexec("{$fetchcmd} -o {$output_dir} -T 5 {$geolite_city_v6}");
mwexec("{$fetchcmd} -o {$output_dir} -T 5 {$geoip_asnum}");
mwexec("{$fetchcmd} -o {$output_dir} -T 5 {$geoip_asnum_v6}");
mwexec("{$fetchcmd} -o {$output_dir}/GeoLite2-City.tar.gz -T 5 \"{$geolite_city}\"");
mwexec("{$fetchcmd} -o {$output_dir}/GeoLite2-ASN.tar.gz -T 5 \"{$geoip_asnum}\"");

ntopng_fixup_geoip();

Expand All @@ -271,16 +267,14 @@ function ntopng_fixup_geoip() {

safe_mkdir($target_dir, 0755);

foreach(glob("{$source_dir}/Geo*.dat*") as $geofile) {
/* Decompress if needed. */
if (substr($geofile, -3, 3) == ".gz") {
// keep -f here, otherwise the files will not get updated
mwexec("/usr/bin/gzip -d -f " . escapeshellarg($geofile));
foreach(glob("{$source_dir}/Geo*.tar.gz") as $geofile) {
if (substr($geofile, -7, 7) == ".tar.gz") {
mwexec("tar -C {$source_dir} -f {$geofile} --strip 1 -xz '*.mmdb'");
}
}

/* Use a separate glob since the filenames could have changed since the last run */
foreach(glob("{$source_dir}/Geo*.dat*") as $geofile) {
foreach(glob("{$source_dir}/Geo*.mmdb*") as $geofile) {
$target_file = $target_dir . '/' . basename($geofile);
if (!file_exists($target_file)) {
symlink($geofile, $target_file);
Expand Down Expand Up @@ -341,8 +335,12 @@ function ntopng_validate_input($post, &$input_errors) {
}
}
}
if ($post['Submit'] == "Update GeoIP Data") {
ntopng_update_geoip();
if ($post['Submit'] == "Update GeoLite2 DB") {
if (empty($post['maxmind_key'])) {
$input_errors[] = "To download MaxMind GeoIP data, you need to provide a valid GeoLite2 DB license key.";
} else {
ntopng_update_geoip($post['maxmind_key']);
}
}
if ($post['Delete'] == "Delete Data") {
ntopng_flush_historical_data();
Expand Down
12 changes: 6 additions & 6 deletions net/pfSense-pkg-ntopng/files/usr/local/pkg/ntopng.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -182,16 +182,16 @@
<type>listtopic</type>
</field>
<field>
<fielddescr>GeoIP Data</fielddescr>
<fielddescr>GeoLite2 DB License Key</fielddescr>
<fieldname>maxmind_key</fieldname>
<description>
<![CDATA[
Additionally, GeoIP Data can provide location information about IP addresses.
<div class="infoblock blockopen">
This product includes GeoLite data created by MaxMind, available from <a href="http://www.maxmind.com" target="_blank">http://www.maxmind.com</a><br /></div><br />
<button class="btn btn-success btn-sm" name='Submit' id='update_geoip' type='submit' value='Update GeoIP Data'><i class="fa fa-refresh icon-embed-btn"></i>Update GeoIP Data</button>
Additionally, <b>GeoLite2 DB</b> can provide location information about IP addresses.<br />
To utilize the free MaxMind GeoLite2 GeoIP functionality, you must <a href="https://www.maxmind.com/en/geolite2/signup" target="_blank">register for a free MaxMind user account</a>.<br /></br />
<button class="btn btn-success btn-sm" name='Submit' id='update_geoip' type='submit' value='Update GeoLite2 DB'><i class="fa fa-refresh icon-embed-btn"></i>Update GeoLite2 DB</button>
]]>
</description>
<type>info</type>
<type>input</type>
</field>
<field>
<fielddescr>Delete ntopng Data</fielddescr>
Expand Down

2 comments on commit 85cdc14

@dalraf
Copy link

@dalraf dalraf commented on 85cdc14 Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Give this error in the update:
PHP ERROR: Type: 1, File: /usr/local/pkg/ntopng.inc, Line: 241, Message: Uncaught ArgumentCountError: Too few arguments to function ntopng_update_geoip(), 0 passed in /etc/inc/pkg-utils.inc(770) : eval()'d code on line 2 and exactly 1 expected in /usr/local/pkg/ntopng.inc:241
Stack trace:
#0 /etc/inc/pkg-utils.inc(770) : eval()'d code(2): ntopng_update_geoip()
#1 /etc/inc/pkg-utils.inc(770): eval()
#2 /etc/inc/pkg-utils.inc(858): eval_once('ntopng_php_inst...')
#3 /etc/rc.packages(74): install_package_xml('ntopng')
#4 {main}
thrownpkg-static: POST-INSTALL script failed

Cleaning up cache... done.
Success

@vktg
Copy link
Contributor Author

@vktg vktg commented on 85cdc14 Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, see #856

Please sign in to comment.