-
Notifications
You must be signed in to change notification settings - Fork 75
/
geoip
520 lines (404 loc) · 15 KB
/
geoip
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
#!perl -w
=head1 NAME
geoip - provide geographic information about mail senders.
=head1 SYNOPSIS
Use MaxMind's GeoIP databases and the GeoIP2 or Geo::IP perl modules to report
geographic information about incoming connections.
=head1 DESCRIPTION
Save geographic information about the sender in the following connection notes:
geoip_country - 2 char country code
geoip_country_name - english name of country
geoip_continent - 2 char continent code
geoip_city - english name of city
geoip_distance - distance in kilometers
geoip_asn - network number
And adds entries like this to your logs:
(connect) ident::geoip: NA, US, United States, 1319 km
(connect) ident::geoip: AS, IN, India, 13862 km
(connect) ident::geoip: fail: no results
(connect) ident::geoip: NA, CA, Canada, 2464 km
(connect) ident::geoip: NA, US, United States, 2318 km
(connect) ident::geoip: AS, PK, Pakistan, 12578 km
(connect) ident::geoip: AS, TJ, Tajikistan, 11965 km
(connect) ident::geoip: EU, AT, Austria, 8745 km
(connect) ident::geoip: AS, IR, Iran, Islamic Republic of, 12180 km
(connect) ident::geoip: EU, BY, Belarus, 9030 km
(connect) ident::geoip: AS, CN, China, 11254 km
(connect) ident::geoip: NA, PA, Panama, 3163 km
Calculating the distance has three prerequsites:
1. The MaxMind city database (free or subscription)
2. The Math::Complex perl module
3. The IP address of this mail server (see CONFIG)
Other plugins can utilize the geographic notes to alter the
connection, reject, greylist, etc.
=head1 CONFIG
The following options can be appended in this plugins config/plugins entry.
=head2 distance <IP Address>
Enables geodesic distance calculation. Will calculate the distance "as the
crow flies" from the remote mail server. Accepts a single argument, the IP
address to calculate the distance from. This will typically be the public
IP of your mail server.
ident/geoip [ distance 192.0.1.5 ]
Default: none. (no distance calculations)
=head2 too_far <distance in km>
Assign negative karma to connections further than this many km.
Default: none
=head2 db_dir </path/to/GeoIP>
The path to the GeoIP database directory.
ident/geoip [ db_dir /etc/GeoIP ]
Default: /usr/local/share/GeoIP
=head2 add_headers <true|false>
Add message headers with GeoIP data
ident/geoip [ add_headers (true|false) ]
Default: true
=head1 LIMITATIONS
The distance calculations are more concerned with being fast than accurate.
The MaxMind location data is collected from whois and is of limited accuracy.
MaxMind offers more accurate data for a fee.
For distance calculations, the earth is considered a perfect sphere. In
reality, it is not. Accuracy should be within 1%.
This plugin does not update the GeoIP databases. You may want to.
=head1 CHANGES
2014-06 - Matt Simerson - added GeoIP2 support
2012-06 - Matt Simerson - added GeoIP City support, continent, distance
2012-05 - Matt Simerson - added geoip_country_name note, added tests
=head1 SEE ALSO
MaxMind: http://www.maxmind.com/
Databases: http://geolite.maxmind.com/download/geoip/database
It may become worth adding support for Geo::IPfree, which uses another
data source: http://software77.net/geo-ip/
=head1 ACKNOWLEDGEMENTS
MaxMind - the packager and distributor of the free GeoIP data
Stevan Bajic, the DSPAM author, who suggested SNARE, which describes using
geodesic distance to determine spam probability. The research paper on SNARE
can be found here:
http://smartech.gatech.edu/bitstream/handle/1853/25135/GT-CSE-08-02.pdf
=cut
use strict;
use warnings;
use Qpsmtpd::Constants;
#use GeoIP2; # eval'ed in register()
#use Geo::IP; # eval loaded if GeoIP2 doesn't
#use Math::Trig; # eval'ed in set_distance_gc
sub register {
my ($self, $qp, @args) = @_;
$self->log(LOGERROR, "Bad arguments") if @args % 2;
$self->{_args} = {@args};
$self->{_args}{db_dir} ||= '/usr/local/share/GeoIP';
$self->load_geoip() or return;
my $enabled = $self->{_args}{add_headers};
$enabled = 'true' if ! defined $enabled;
return if $enabled =~ /false/i;
$self->register_hook( data_post => 'add_headers' );
}
sub load_geoip {
my ( $self ) = @_;
$self->load_geoip2() and return 1;
$self->load_geoip1() and return 1;
return 0;
}
sub load_geoip1 {
my $self = shift;
eval 'use Geo::IP';
if ($@) {
warn "could not load Geo::IP";
$self->log(LOGERROR, "could not load Geo::IP");
return;
}
$self->open_geoip_db();
# Note that opening the GeoIP DB only in register has caused problems before:
# https://github.com/smtpd/qpsmtpd/commit/29ea9516806e9a8ca6519fcf987dbd684793ebdd#plugins/ident/geoip
# Opening the DB anew for every connection is horribly inefficient.
# Instead, attempt to reopen upon connect if the DB connection fails.
$self->init_my_country_code();
$self->register_hook('connect', 'geoip_lookup');
return 1;
}
sub load_geoip2 {
my $self = shift;
eval 'use GeoIP2::Database::Reader';
if ($@) {
$self->log(LOGERROR, "could not load GeoIP2");
return;
}
warn "Using GeoIP2."
. " ASN data is not currently available using the GeoIP2 module!\n";
eval {
$self->{_geoip2_city} = GeoIP2::Database::Reader->new(
file => $self->{_args}{db_dir} . '/GeoLite2-City.mmdb',
);
};
if ($@) {
$self->log(LOGERROR, "unable to load GeoLite2-City.mmdb");
}
eval {
$self->{_geoip2_country} = GeoIP2::Database::Reader->new(
file => $self->{_args}{db_dir} . '/GeoLite2-Country.mmdb',
);
};
if ($@) {
$self->log(LOGERROR, "unable to load GeoLite2-Country.mmdb");
}
if ($self->{_geoip2_city} || $self->{_geoip2_country}) {
$self->register_hook('connect', 'geoip2_lookup');
return 1;
}
return;
}
sub add_headers {
my ( $self, $txn ) = @_;
for my $h (qw( Country Continent City ASN )) {
my $note = lc "geoip_$h";
next if ! $self->connection->notes($note);
$txn->header->delete("X-GeoIP-$h");
$txn->header->add( "X-GeoIP-$h", $self->connection->notes($note), 0 );
}
return DECLINED;
}
sub geoip2_lookup {
my $self = shift;
my $ip = $self->qp->connection->remote_ip;
return DECLINED if $self->is_localhost($ip);
if ($self->{_geoip2_city}) {
my $city_rec = $self->{_geoip2_city}->city(ip => $ip);
if ($city_rec) {
$self->qp->connection->notes('geoip_country', $city_rec->country->iso_code());
$self->qp->connection->notes('geoip_country_name', $city_rec->country->name());
$self->qp->connection->notes('geoip_continent', $city_rec->continent->code());
$self->qp->connection->notes('geoip_city', $city_rec->city->name());
$self->qp->connection->notes('geoip_asn', $city_rec->traits->autonomous_system_number());
return DECLINED;
}
}
if ($self->{_geoip2_country}) {
my $country_rec = $self->{_geoip2_country}->country(ip => $ip);
if ($country_rec) {
$self->qp->connection->notes('geoip_country', $country_rec->country->iso_code());
$self->qp->connection->notes('geoip_country_name', $country_rec->country->name());
$self->qp->connection->notes('geoip_continent', $country_rec->continent->code());
};
}
return DECLINED;
}
sub geoip_lookup {
my $self = shift;
return DECLINED if $self->is_localhost($self->qp->connection->remote_ip);
# reopen the DB if Geo::IP failed due to DB update
$self->open_geoip_db();
my $c_code = $self->set_country_code() or do {
$self->log(LOGINFO, "skip, no results");
return DECLINED;
};
$self->set_asn();
my $c_name = $self->set_country_name();
my ($city, $continent_code, $distance) = '';
if ($self->{_my_country_code}) {
$continent_code = $self->set_continent($c_code);
$city = $self->set_city_gc();
$distance = $self->set_distance_gc();
}
my @msg_parts;
if ($continent_code && $continent_code ne '--') {
push @msg_parts, $continent_code;
};
push @msg_parts, $c_code if $c_code;
#push @msg_parts, $c_name if $c_name;
push @msg_parts, $city if $city;
if ($distance) {
push @msg_parts, "\t$distance km";
if ($self->{_args}{too_far} && $distance > $self->{_args}{too_far}) {
$self->adjust_karma(-1);
}
}
$self->log(LOGINFO, join(", ", @msg_parts));
return DECLINED;
}
sub open_geoip_db {
my $self = shift;
# this might detect if the DB connection failed. If not, this is where
# to add more code to do it.
return if (defined $self->{_geoip_city} || defined $self->{_geoip});
# The methods for using GeoIP work differently for the City vs Country DB
# save the handles in different locations
my $db_dir = $self->{_args}{db_dir};
foreach my $db (qw/ GeoIPCity GeoLiteCity /) {
next if !-f "$db_dir/$db.dat";
$self->log(LOGDEBUG, "using db $db");
$self->{_geoip_city} = Geo::IP->open("$db_dir/$db.dat");
last if $self->{_geoip_city};
}
warn "Missing GeoIP City data!\n" if ! $self->{_geoip_city};
if (-f "$db_dir/GeoIPASNum.dat") {
$self->log(LOGDEBUG, "using GeoIPASNum");
$self->{GeoIPASNum} = Geo::IP->open("$db_dir/GeoIPASNum.dat");
}
warn "Missing GeoIP ASN data!\n" if ! $self->{GeoIPASNum};
if (-f "$db_dir/GeoIPASNumv6.dat") {
$self->log(LOGDEBUG, "using GeoIPASNumv6");
$self->{GeoIPASNumv6} = Geo::IP->open("$db_dir/GeoIPASNumv6.dat");
warn "Missing GeoIP ASN IPV6 data!\n" if ! $self->{GeoIPASNum};
}
# can't think of a good reason to load country if city data is present
if (!$self->{_geoip_city}) {
$self->log(LOGDEBUG, "using default db");
eval { $self->{_geoip} = Geo::IP->new(); }; # loads default Country DB
if (!$self->{_geoip}) {
my $err = $@ || 'Unknown error';
warn "Missing GeoIP Country data:$err\n";
}
}
}
sub init_my_country_code {
my $self = shift;
my $ip = $self->{_args}{distance} or return;
$self->{_my_country_code} = $self->get_country_code($ip);
}
sub set_country_code {
my $self = shift;
my $ip = $self->qp->connection->remote_ip;
my $code = $self->get_country_code($ip) or return;
$self->qp->connection->notes('geoip_country', $code);
return $code;
}
sub get_country_code {
my $self = shift;
my $ip = shift || $self->qp->connection->remote_ip;
if ($self->{_geoip_city}) {
return $self->get_country_code_gc($ip);
}
if ($self->{_geoip}) {
return $self->{_geoip}->country_code_by_addr($ip);
}
return undef;
}
sub get_country_code_gc {
my $self = shift;
my $ip = shift || $self->qp->connection->remote_ip;
$self->{_geoip_record} = $self->{_geoip_city}->record_by_addr($ip)
or return;
return $self->{_geoip_record}->country_code;
}
sub set_country_name {
my $self = shift;
my $ip = $self->qp->connection->remote_ip;
my $name = $self->get_country_name($ip) or return;
$self->qp->connection->notes('geoip_country_name', $name);
return $name;
}
sub get_country_name {
my $self = shift;
my $ip = shift || $self->qp->connection->remote_ip;
if ($self->{_geoip_city}) {
return $self->get_country_name_gc($ip);
}
if ($self->{_geoip}) {
return $self->{_geoip}->country_name_by_addr($ip);
}
return undef;
}
sub get_country_name_gc {
my $self = shift;
return if !$self->{_geoip_record};
return $self->{_geoip_record}->country_name();
}
sub set_continent {
my ($self, $country_code) = @_;
return if !$country_code;
my $continent = $self->get_continent($country_code) or return;
$self->qp->connection->notes('geoip_continent', $continent);
return $continent;
}
sub get_continent {
my ($self, $country_code) = @_;
return if !$country_code;
if ($self->{_geoip_city}) {
return $self->get_continent_gc();
}
if ($self->{_geoip}) {
return $self->{_geoip}->continent_code_by_country_code($country_code);
}
return undef;
}
sub get_continent_gc {
my $self = shift;
return if !$self->{_geoip_record};
return $self->{_geoip_record}->continent_code();
}
sub set_asn {
my ($self, $ip) = @_;
$ip ||= $self->qp->connection->remote_ip;
if ($self->is_ipv6($ip)) {
return $self->set_asn_ipv6($ip);
}
return if ! $self->{GeoIPASNum};
my $asn = $self->{GeoIPASNum}->name_by_addr($ip) or return;
if ('AS' eq substr($asn, 0, 2)) {
$asn = substr($asn, 2);
}
$self->qp->connection->notes('geoip_asn', $asn);
return $asn;
}
sub set_asn_ipv6 {
my ($self, $ip) = @_;
$ip ||= $self->qp->connection->remote_ip;
return if ! $self->{GeoIPASNumv6};
my $asn = $self->{GeoIPASNumv6}->name_by_addr_v6($ip) or return;
$self->qp->connection->notes('geoip_asn', $asn);
return $asn;
}
sub set_city_gc {
my $self = shift;
return if !$self->{_geoip_record};
my $city = $self->{_geoip_record}->city() or return;
$self->qp->connection->notes('geoip_city', $city);
return $city;
}
sub set_distance_gc {
my $self = shift;
return if !$self->{_geoip_record};
my ($self_lat, $self_lon) = $self->get_my_lat_lon() or return;
my ($sender_lat, $sender_lon) = $self->get_sender_lat_lon() or return;
eval 'use Math::Trig qw(great_circle_distance deg2rad)';
if ($@) {
$self->log(LOGERROR,
"can't calculate distance, Math::Trig not installed");
return;
}
# Notice the 90 - latitude: phi zero is at the North Pole.
sub NESW { deg2rad($_[0]), deg2rad(90 - $_[1]) }
my @me = NESW($self_lon, $self_lat);
my @sender = NESW($sender_lon, $sender_lat);
my $km = great_circle_distance(@me, @sender, 6378);
$km = sprintf("%.0f", $km);
$self->qp->connection->notes('geoip_distance', $km);
#$self->log( LOGINFO, "distance $km km");
return $km;
}
sub get_my_lat_lon {
my $self = shift;
return if !$self->{_geoip_city};
if ($self->{_latitude} && $self->{_longitude}) {
return $self->{_latitude}, $self->{_longitude}; # cached
}
my $ip = $self->{_args}{distance} or return;
my $record = $self->{_geoip_city}->record_by_addr($ip) or do {
$self->log(LOGERROR, "no record for my Geo::IP location");
return;
};
$self->{_latitude} = $record->latitude();
$self->{_longitude} = $record->longitude();
if (!$self->{_latitude} || !$self->{_longitude}) {
$self->log(LOGNOTICE, "could not get my lat/lon");
}
return $self->{_latitude}, $self->{_longitude};
}
sub get_sender_lat_lon {
my $self = shift;
my $lat = $self->{_geoip_record}->latitude();
my $lon = $self->{_geoip_record}->longitude();
if (!$lat || !$lon) {
$self->log(LOGNOTICE, "could not get sender lat/lon");
return;
}
return $lat, $lon;
}