Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
...
Checking mergeability… Don't worry, you can still create the pull request.
  • 5 commits
  • 8 files changed
  • 0 commit comments
  • 1 contributor
Commits on May 21, 2012
@msimerson msimerson SPF plugin: refactored, tests, new config option
added POD description of spfquery note

changed spf_deny -> reject  (and offered 4 more options, see POD for reject)
	backwards compatible with old config settings
	replicates qmail-smtpd SPF patch behavior

improved logging (again)

uses a stringy eval 'use Mail::SPF' in the register sub. If missing, warn and log the error, and don't register any hooks. This is much nicer error than the current, "*** Remote host closed connection unexpectedly." broken mail server that results from enabling the SPF plugin without Mail::SPF installed.

background: I noticed I was deferring valid emails with the SPF plugin at 'spf_deny 1', and without changing the code, there wasn't a way to change how ~all records were handled. This provides that flexibility.
51486d0
@msimerson msimerson greylisting: added upgrade note 15bf425
@msimerson msimerson added loglevel option for plugins
Plugins can now use a 'loglevel' argument in config/plugins entry

Includes user instructions prepended to docs/logging.pod

Already works for all plugins that use named arguments
efbaf2e
@msimerson msimerson domainkeys: only register hooks if Mail::DomainKeys is loadable 41550c2
@msimerson msimerson enable domainkeys plugin d5f15a7
View
2  Changes
@@ -1,6 +1,8 @@
Next Version
+ sender_permitted_from. see UPGRADING (Matt Simerson)
+
dspam plugin added (Matt Simerson)
p0f version 3 supported and new default. see UPGRADING (Matt Simerson)
View
16 UPGRADING
@@ -3,8 +3,20 @@ When upgrading from:
v 0.84 or below
-p0f plugin: now defaults to p0f v3
+GREYLISTING plugin:
-Upgrade p0f to version 3 or add 'version 2' to your p0f line in config/plugins. perldoc plugins/ident/p0f for more details.
+ 'mode' config argument is deprecated. Use reject and reject_type instead.
+
+ The greylisting DB format has changed to accommodate IPv6 addresses. (The DB key has colon ':' seperated fields, and IPv6 addresses are colon delimited). The new format converts the IPs into integers. There is a new config option named 'upgrade' that when enabled, updates all the records in your DB to the new format. Simply add 'upgrade 1' to the plugin entry in config/plugins, start up qpsmtpd once, make one connection. A log entry will be made, telling how many records were upgraded. Remove the upgrade option from your config.
+
+SPF plugin:
+
+ spf_deny setting deprecated. Use reject N setting instead, which provides administrators with more granular control over SPF. For backward compatibility, a spf_deny setting of 1 is mapped to 'reject 3' and a 'spf_deny 2' is mapped to 'reject 4'.
+
+
+P0F plugin:
+ defaults to p0f v3 (was v2).
+
+ Upgrade p0f to version 3 or add 'version 2' to your p0f line in config/plugins. perldoc plugins/ident/p0f for more details.
View
1  config.sample/plugins
@@ -47,6 +47,7 @@ auth/authdeny
rcpt_ok
check_basicheaders days 5 reject_type temp
+domainkeys
# content filters
virus/klez_filter
View
121 docs/logging.pod
@@ -1,7 +1,124 @@
#
-# read this with 'perldoc README.logging' ...
+# read this with 'perldoc docs/logging.pod'
#
+=head1 qpsmtpd logging; user documentation
+
+Qpsmtpd has a modular logging system. Here's a few things you need to know:
+
+ * The built-in logging prints log messages to STDERR.
+ * A variety of logging plugins is included, each with its own behavior.
+ * When a logging plugin is enabled, the built-in logging is disabled.
+ * plugins/logging/warn mimics the built-in logging.
+ * Multiple logging plugins can be enabled simultaneously.
+
+Read the POD within each logging plugin (perldoc plugins/logging/B<NAME>)
+to learn if it tickles your fancy.
+
+=head2 enabling plugins
+
+To enable logging plugins, edit the file I<config/logging> and uncomment the
+entries for the plugins you wish to use.
+
+=head2 logging level
+
+The 'master switch' for loglevel is I<config/loglevel>. Qpsmtpd and active
+plugins will output all messages that are less than or equal to the value
+specified. The log levels correspond to syslog levels:
+
+ LOGDEBUG = 7
+ LOGINFO = 6
+ LOGNOTICE = 5
+ LOGWARN = 4
+ LOGERROR = 3
+ LOGCRIT = 2
+ LOGALERT = 1
+ LOGEMERG = 0
+ LOGRADAR = 0
+
+Level 6, LOGINFO, is the level at which most servers should start logging. At
+level 6, each plugin should log one and occasionally two entries that
+summarize their activity. Here's a few sample lines:
+
+ (connect) ident::geoip: SA, Saudi Arabia
+ (connect) ident::p0f: Windows 7 or 8
+ (connect) earlytalker: pass: remote host said nothing spontaneous
+ (data_post) domainkeys: skip: unsigned
+ (data_post) spamassassin: pass, Spam, 21.7 < 100
+ (data_post) dspam: fail: agree, Spam, 1.00 c
+ 552 we agree, no spam please (#5.6.1)
+
+Three plugins fired during the SMTP connection phase and 3 more ran during the
+data_post phase. Each plugin emitted one entry stating their findings.
+
+If you aren't processing the logs, you can save some disk I/O by reducing the
+loglevel, so that the only messages logged are ones that indicate a human
+should be taking some corrective action.
+
+=head2 log location
+
+If qpsmtpd is started using the distributed run file (cd ~smtpd; ./run), then
+you will see the log entries printed to your terminal. This solution works
+great for initial setup and testing and is the simplest case.
+
+A typical way to run qpsmtpd is as a supervised process with daemontools. If
+daemontools is already set up, setting up qpsmtpd may be as simple as:
+
+C<ln -s /usr/home/smtpd /var/service/>
+
+If svcscan is running, the symlink will be detected and tcpserver will
+run the 'run' files in the ./ and ./log directories. Any log entries
+emitted will get handled per the instructions in log/run. The default
+location specified in log/run is log/main/current.
+
+=head2 plugin loglevel
+
+Most plugins support a loglevel argument after their config/plugins entry.
+The value can be a whole number (N) or a relative number (+/-N), where
+N is a whole number from 0-7. See the descriptions of each below.
+
+C<ident/p0f loglevel 5>
+
+C<ident/p0f loglevel -1>
+
+ATTN plugin authors: To support loglevel in your plugin, you must store the
+loglevel settings from the plugins/config entry $self->{_args}{loglevel}. A
+simple and recommended example is as follows:
+
+ sub register {
+ my ( $self, $qp ) = shift, shift;
+ $self->log(LOGERROR, "Bad arguments") if @_ % 2;
+ $self->{_args} = { @_ };
+ }
+
+=head3 whole number
+
+If loglevel is a whole number, then all log activity in the plugin is logged
+at that level, regardless of the level the plugin author selected. This can
+be easily understood with a couple examples:
+
+The master loglevel is set at 6 (INFO). The mail admin sets a plugin loglevel
+to 7 (DEBUG). No messages from that plugin are emitted because DEBUG log
+entries are not <= 6 (INFO).
+
+The master loglevel is 6 (INFO) and the plugin loglevel is set to 5 or 6. All
+log entries will be logged because 5 is <= 6.
+
+This behavior is very useful to plugin authors. While testing and monitoring
+a plugin, they can set the level of their plugin to log everything. To return
+to 'normal' logging, they just update their config/plugins entry.
+
+=head3 relative
+
+Relative loglevel arguments adjust the loglevel of each logging call within
+a plugin. A value of I<loglevel +1> would make every logging entry one level
+less severe, where a value of I<loglevel -1> would make every logging entry
+one level more severe.
+
+For example, if a plugin has a loglevel setting of -1 and that same plugin
+logged a LOGDEBUG, it would instead be a LOGINFO message. Relative values
+makes it easy to control the verbosity and/or severity of individual plugins.
+
=head1 qpsmtpd logging system; developer documentation
Qpsmtpd now (as of 0.30-dev) supports a plugable logging architecture, so
@@ -62,7 +179,7 @@ plugin (the system will not infinitely recurse in any case).
=item C<@log>
The remaining arguments are as passed by the caller, which may be a single
-term or may be a list of values. It is usually sufficient to call
+term or may be a list of values. It is usually sufficient to call
C<join(" ",@log)> to deal with these terms, but it is possible that some
plugin might pass additional arguments with signficance.
View
29 lib/Qpsmtpd/Plugin.pm
@@ -63,10 +63,35 @@ sub qp {
sub log {
my $self = shift;
- $self->{_qp}->varlog(shift, $self->{_hook}, $self->plugin_name, @_)
- unless defined $self->{_hook} and $self->{_hook} eq 'logging';
+ return if defined $self->{_hook} && $self->{_hook} eq 'logging';
+ my $level = $self->adjust_log_level( shift, $self->plugin_name );
+ $self->{_qp}->varlog($level, $self->{_hook}, $self->plugin_name, @_);
}
+sub adjust_log_level {
+ my ( $self, $cur_level, $plugin_name) = @_;
+
+ my $adj = $self->{_args}{loglevel} or return $cur_level;
+
+ return $adj if $adj =~ m/^[01234567]$/; # a raw syslog numeral
+
+ if ( $adj !~ /^[\+\-][\d]$/ ) {
+ $self->log( LOGERROR, $self-"invalid $plugin_name loglevel setting ($adj)" );
+ undef $self->{_args}{loglevel}; # only complain once per plugin
+ return $cur_level;
+ };
+
+ my $operator = substr($adj, 0, 1);
+ my $adjust = substr($adj, -1, 1);
+
+ my $new_level = $operator eq '+' ? $cur_level + $adjust : $cur_level - $adjust;
+
+ $new_level = 7 if $new_level > 7;
+ $new_level = 0 if $new_level < 0;
+
+ return $new_level;
+};
+
sub transaction {
# not sure if this will work in a non-forking or a threaded daemon
shift->qp->transaction;
View
17 plugins/domainkeys
@@ -68,7 +68,22 @@ sub init {
};
}
-sub hook_data_post {
+sub register {
+ my $self = shift;
+
+ for my $m ( qw/ Mail::DomainKeys::Message Mail::DomainKeys::Policy / ) {
+ eval "use $m";
+ if ( $@ ) {
+ warn "skip: plugin disabled, could not load $m\n";
+ $self->log(LOGERROR, "skip: plugin disabled, is $m installed?");
+ return;
+ };
+ };
+
+ $self->register_hook('data_post', 'data_post_handler');
+};
+
+sub data_post_handler {
my ($self, $transaction) = @_;
if ( ! $transaction->header->get('DomainKey-Signature') ) {
View
200 plugins/sender_permitted_from
@@ -12,20 +12,41 @@ Prevents email sender address spoofing by checking the SPF policy of the purport
Sender Policy Framework (SPF) is an e-mail validation system designed to prevent spam by addressing source address spoofing. SPF allows administrators to specify which hosts are allowed to send e-mail from a given domain by creating a specific SPF record in the public DNS. Mail exchangers then use the DNS to check that mail from a given domain is being sent by a host sanctioned by that domain's administrators. -- http://en.wikipedia.org/wiki/Sender_Policy_Framework
+The results of a SPF query are stored in a transaction note named 'spfquery';
+
=head1 CONFIGURATION
In config/plugins, add arguments to the sender_permitted_from line.
- sender_permitted_from spf_deny 1
+ sender_permitted_from reject 3
+
+=head2 reject
+
+Set to a value between 1 and 6 to enable the following SPF behaviors:
+
+ 1 annotate-only, add Received-SPF header, no rejections.
+ 2 defer on DNS failures. Assure there's always a meaningful SPF header.
+ 3 rejected if SPF record says 'fail'
+ 4 stricter reject. Also rejects 'softfail'
+ 5 reject 'neutral'
+ 6 reject if no SPF records, or a syntax error
+
+Most sites should start at level 3. It temporarily defers connections (4xx) that have soft SFP failures and only rejects (5xx) messages when the sending domains policy suggests it.
+
+SPF levels above 4 are for crusaders who don't mind rejecting some valid mail when the sending server administrator hasn't dotted his i's and crossed his t's. May the deities bless theirobsessive little hearts.
+
+=head1 SEE ALSO
-=head2 spf_deny
+ http://spf.pobox.com/
+ http://en.wikipedia.org/wiki/Sender_Policy_Framework
-Setting spf_deny to 0 will prevent emails from being rejected, even if they fail SPF checks. sfp_deny 1 is the default, and a reasonable setting. It temporarily defers connections (4xx) that have soft SFP failures and only rejects (5xx) messages when the sending domains policy suggests it. Settings spf_deny to 2 is more aggressive and will cause soft failures to be rejected permanently.
+=head1 ACKNOWLDGEMENTS
-See also http://spf.pobox.com/
+The reject options are modeled after, and aim to match the functionality of those found in the SPF patch for qmail-smtpd.
=head1 AUTHOR
+Matt Simerson - 2002 - increased policy options from 3 to 6
Matt Simerson - 2011 - rewrote using Mail::SPF
Matt Sergeant - 2003 - initial plugin
@@ -33,55 +54,57 @@ Matt Sergeant - 2003 - initial plugin
=cut
use strict;
-use Mail::SPF 2.000;
+use warnings;
+
+#use Mail::SPF 2.000; # eval'ed in ->register
use Qpsmtpd::Constants;
sub register {
- my ($self, $qp, @args) = @_;
- %{$self->{_args}} = @args;
+ my ($self, $qp, %args) = @_;
+ eval "use Mail::SPF";
+ if ( $@ ) {
+ warn "skip: plugin disabled, could not find Mail::SPF\n";
+ $self->log(LOGERROR, "skip: plugin disabled, is Mail::SPF installed?");
+ return;
+ };
+ $self->{_args} = { %args };
+ if ( $self->{_args}{spf_deny} ) {
+ $self->{_args}{reject} = 3 if $self->{_args}{spf_deny} == 1;
+ $self->{_args}{reject} = 4 if $self->{_args}{spf_deny} == 2;
+ };
+ if ( ! $self->{_args}{reject} && $self->qp->config('spfbehavior') ) {
+ $self->{_args}{reject} = $self->qp->config('spfbehavior');
+ };
}
sub hook_mail {
my ($self, $transaction, $sender, %param) = @_;
- my $format = $sender->format;
+ if ( ! $self->{_args}{reject} ) {
+ $self->log( LOGINFO, "skip: disabled in config" );
+ return (DECLINED);
+ };
+
+ my $format = $sender->format;
if ( $format eq '<>' || ! $sender->host || ! $sender->user ) {
- $self->log( LOGDEBUG, "pass: null sender" );
+ $self->log( LOGINFO, "skip: null sender" );
return (DECLINED, "SPF - null sender");
};
- my $client_ip = $self->qp->connection->remote_ip;
- my $from = $sender->user . '@' . lc($sender->host);
- my $helo = $self->qp->connection->hello_host;
-
- # If we are receiving from a relay permitted host, then we are probably
- # not the delivery system, and so we shouldn't check
- if ( $self->qp->connection->relay_client() ) {
- $self->log( LOGDEBUG, "pass: relaying permitted (connection)" );
- return (DECLINED, "SPF - relaying permitted")
+ if ( $self->is_relayclient() ) {
+ return (DECLINED, "SPF - relaying permitted");
};
- my @relay_clients = $self->qp->config("relayclients");
- my $more_relay_clients = $self->qp->config("morerelayclients", "map");
- my %relay_clients = map { $_ => 1 } @relay_clients;
- while ($client_ip) {
- if ( exists $relay_clients{$client_ip} ||
- exists $more_relay_clients->{$client_ip} ) {
- $self->log( LOGDEBUG, "pass: relaying permitted (config)" );
- return (DECLINED, "SPF - relaying permitted");
- };
- $client_ip =~ s/\d+\.?$//; # strip off another 8 bits
- }
-
- my $scope = $from ? 'mfrom' : 'helo';
- $client_ip = $self->qp->connection->remote_ip;
- my %req_params = (
- versions => [1, 2], # optional
- scope => $scope,
- ip_address => $client_ip,
+ my $client_ip = $self->qp->connection->remote_ip;
+ my $from = $sender->user . '@' . lc($sender->host);
+ my $helo = $self->qp->connection->hello_host;
+ my $scope = $from ? 'mfrom' : 'helo';
+ my %req_params = ( versions => [1, 2], # optional
+ scope => $scope,
+ ip_address => $client_ip,
);
- if ($scope =~ /mfrom|pra/) {
+ if ($scope =~ /^mfrom|pra$/) {
$req_params{identity} = $from;
$req_params{helo_identity} = $helo if $helo;
}
@@ -95,44 +118,63 @@ sub hook_mail {
my $result = $spf_server->process($request);
$transaction->notes('spfquery', $result);
- $transaction->notes('spfcode', $result->code);
- if ( $result->code eq 'pass' ) { # this test passed
- $self->log( LOGINFO, "pass" );
+ $self->log( LOGINFO, $result );
+
+ if ( $result->code eq 'pass' ) {
return (OK);
};
- $self->log( LOGINFO, "fail: " . $result );
return (DECLINED, "SPF - $result->code");
}
sub hook_rcpt {
my ($self, $transaction, $rcpt, %param) = @_;
- # special addresses don't get SPF-tested.
- return DECLINED
- if $rcpt
- and $rcpt->user
- and $rcpt->user =~ /^(?:postmaster|abuse|mailer-daemon|root)$/i;
+ return DECLINED if $self->is_special_recipient( $rcpt );
my $result = $transaction->notes('spfquery') or return DECLINED;
my $code = $result->code;
my $why = $result->local_explanation;
- my $deny = $self->{_args}{spf_deny};
+ my $reject = $self->{_args}{reject};
+
+ if ( ! $code ) {
+ return (DENYSOFT, "SPF - no response") if $reject >= 2;
+ return (DECLINED, "SPF - no response");
+ };
- return (DECLINED, "SPF - $code: $why") if $code eq "pass";
- return (DECLINED, "SPF - $code, $why") if !$deny;
- return (DENYSOFT, "SPF - $code: $why") if $code eq "error";
- return (DENY, "SPF - forgery: $why") if $code eq 'fail';
+ return (DECLINED, "SPF - $code: $why") if ! $reject;
- if ($code eq "softfail") {
- return (DENY, "SPF probable forgery: $why") if $deny > 1;
- return (DENYSOFT, "SPF probable forgery: $why");
+# SPF result codes: pass fail softfail neutral none error permerror temperror
+ if ( $code eq 'pass' ) { }
+ elsif ( $code eq 'fail' ) {
+ return (DENY, "SPF - forgery: $why") if $reject >= 3;
+ return (DENYSOFT, "SPF - $code: $why") if $reject >= 2;
+ }
+ elsif ( $code eq 'softfail' ) {
+ return (DENY, "SPF - forgery: $why") if $reject >= 4;
+ return (DENYSOFT, "SPF - $code: $why") if $reject >= 3;
+ }
+ elsif ( $code eq 'neutral' ) {
+ return (DENY, "SPF - forgery: $why") if $reject >= 5;
+ }
+ elsif ( $code eq 'none' ) {
+ return (DENY, "SPF - forgery: $why") if $reject >= 6;
+ }
+ elsif ( $code eq 'error' ) {
+ return (DENY, "SPF - $code: $why") if $reject >= 6;
+ return (DENYSOFT, "SPF - $code: $why") if $reject >= 2;
+ }
+ elsif ( $code eq 'permerror' ) {
+ return (DENY, "SPF - $code: $why") if $reject >= 6;
+ return (DENYSOFT, "SPF - $code: $why") if $reject >= 2;
+ }
+ elsif ( $code eq 'temperror' ) {
+ return (DENYSOFT, "SPF - $code: $why") if $reject >= 2;
}
$self->log(LOGDEBUG, "result for $rcpt->address was $code: $why");
-
- return (DECLINED, "SPF - $code, $why");
+ return (DECLINED, "SPF - $code: $why");
}
sub hook_data_post {
@@ -147,3 +189,49 @@ sub hook_data_post {
return DECLINED;
}
+sub is_relayclient {
+ my $self = shift;
+
+ # If we are receiving from a relay permitted host, then we are probably
+ # not the delivery system, and so we shouldn't check
+ if ( $self->qp->connection->relay_client() ) {
+ $self->log( LOGINFO, "skip: relaying permitted (relay_client)" );
+ return 1;
+ };
+
+ my $client_ip = $self->qp->connection->remote_ip;
+ my @relay_clients = $self->qp->config('relayclients');
+ my $more_relay_clients = $self->qp->config('morerelayclients', 'map');
+ my %relay_clients = map { $_ => 1 } @relay_clients;
+
+ while ($client_ip) {
+ if ( exists $relay_clients{$client_ip} ||
+ exists $more_relay_clients->{$client_ip} ) {
+ $self->log( LOGDEBUG, "skip: relaying permitted (config)" );
+ return 1;
+ };
+ $client_ip =~ s/\d+\.?$// or last; # strip off another 8 bits
+ }
+ return;
+};
+
+sub is_special_recipient {
+ my ($self, $rcpt) = @_;
+
+ if ( ! $rcpt ) {
+ $self->log(LOGINFO, "skip: missing recipient");
+ return 1;
+ };
+ if ( ! $rcpt->user ) {
+ $self->log(LOGINFO, "skip: missing user");
+ return 1;
+ };
+
+ # special addresses don't get SPF-tested.
+ if ( $rcpt->user =~ /^(?:postmaster|abuse|mailer-daemon|root)$/i ) {
+ $self->log(LOGINFO, "skip: special user (".$rcpt->user.")");
+ return 1;
+ };
+
+ return;
+};
View
50 t/plugin_tests/sender_permitted_from
@@ -0,0 +1,50 @@
+#!perl -w
+
+use strict;
+use warnings;
+
+use Qpsmtpd::Constants;
+
+my $r;
+
+sub register_tests {
+ my $self = shift;
+
+ eval 'use Mail::SPF';
+ return if $@;
+
+ $self->register_test('test_is_relayclient', 3);
+ $self->register_test('test_is_special_recipient', 5);
+}
+
+sub test_is_relayclient {
+ my $self = shift;
+
+ my $transaction = $self->qp->transaction;
+ ok( ! $self->is_relayclient( $transaction ),
+ "sender_permitted_from, is_relayclient -");
+
+ $self->qp->connection->relay_client(1);
+ ok( $self->is_relayclient( $transaction ),
+ "sender_permitted_from, is_relayclient +");
+
+ $self->qp->connection->relay_client(0);
+ $self->qp->connection->remote_ip('192.168.7.5');
+ my $client_ip = $self->qp->connection->remote_ip;
+ ok( $client_ip, "sender_permitted_from, relayclients ($client_ip)");
+};
+
+sub test_is_special_recipient {
+ my $self = shift;
+
+ my $transaction = $self->qp->transaction;
+ my $address = Qpsmtpd::Address->new('user@example.com');
+
+ ok( ! $self->is_special_recipient( $address ), "is_special_recipient -");
+
+ foreach my $user ( qw/ postmaster abuse mailer-daemon root / ) {
+ $address = Qpsmtpd::Address->new("$user\@example.com");
+ ok( $self->is_special_recipient( $address ), "is_special_recipient ($user)");
+ };
+};
+

No commit comments for this range

Something went wrong with that request. Please try again.