Skip to content

bringing in 0.94 #36

Merged
merged 376 commits into from Jan 11, 2014

2 participants

@msimerson
qpsmtpd member

No description provided.

msimerson and others added some commits Nov 14, 2012
@msimerson msimerson TcpServer, improve IPv6 support, by Michael Holzt 161834f
@msimerson msimerson SPF: more logging additions 6758195
Matt Simerson summarize: check more locations to discover QP dir e9cf061
Matt Simerson log/run: removed spurious space b0f5618
Matt Simerson dspam: better error message if dspam_bin is not found 3127f4d
Matt Simerson qmail_deliverable: test variable if defined before accessing 74d97d3
Matt Simerson spamassassin: further log message refinement 4928792
@msimerson msimerson registry: added auth_ prefixes, relay aliases 1cfd7df
@msimerson msimerson summarize: recognize tcpserver log entries e46c6e3
@msimerson msimerson clamdscan: default is scan always, even authenticated 96f49c9
@msimerson msimerson run: define PORT variable 6e731e8
@msimerson msimerson logs: improve ability to find logs 77c892d
@msimerson msimerson helo: added is_plain_ip to lenient checks
there's no excuse for a client to ever send a raw IP, and I have yet to see a valid client do it
ad558d5
@msimerson msimerson karma: added adjust_karma method
makes it easier to set karma in plugins
477c5a6
@msimerson msimerson added log/show_message, dropped .pl suffix
to be consistent with other QP scripts
be828a8
@msimerson msimerson config: replace domainkeys with dkim
dkim is the heir apparent
the Mail::DomainKeys perl module is deprecated (per it's author)
710a838
@msimerson msimerson karma: added error keyword to error log messages aa4e102
@msimerson msimerson registry: renamed clamd abb3 from cad to clm 809390b
@msimerson msimerson added missing semicolon 7eedea8
@msimerson msimerson log/summarize: added auth formats ba6a04e
@msimerson msimerson config/plugins: better defaults, additional entries 3145a37
@msimerson msimerson log/summarize: narrower column when no geoip city data present 17abbfe
@msimerson msimerson log/show_message: fixed QP dir detection 9ffdd14
@msimerson msimerson resolvable_fromhost: additional logging 335a71e
@msimerson msimerson earlytalker: lower karma for earlytalkers 376bd49
@msimerson msimerson allow messages with no body: Robin's patch df715db
@msimerson msimerson change loglevel from 9 to 6
more appropriate loglevel for users
a5eda10
@msimerson msimerson dnsbl rejections handled by naughty plugin 2b02f6b
@msimerson msimerson update plugin/headers config entry
use future/past instead of days:

-headers days 5 reject_type temp require From,Date
+headers reject 1 reject_type temp require From,Date future 2 past 15
deca40a
@msimerson msimerson removed duplicate naughty from config 000db0a
@msimerson msimerson added vpopmail_ext to qmail_deliverable plugin ab1b211
@msimerson msimerson spamassassin: added 'headers none' option
enables suppression of SA header insertion
c1694b2
@msimerson msimerson whitelist: added debug log message & std plugin
entries.
51f5c88
@msimerson msimerson spf: improved support for IPv6 clients b8baa4b
@msimerson msimerson TcpServer, improve IPv6 support, by Michael Holzt 61f7ea0
@msimerson msimerson SMTP.pm: fixed invalid operator
which produced this warning:
   Useless use of string eq in void context at lib/Qpsmtpd/SMTP.pm line 639.
4f181c5
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev
Conflicts:
	lib/Qpsmtpd/SMTP.pm
938f72d
@msimerson msimerson fix relayclient test after commit b8baa4b
which added example IPv6 IPs to the config file
f0c01a0
@msimerson msimerson SA: suppress undefined variable warnings 427e92e
@msimerson msimerson dnsbl: fixed plugin test failure 01b623d
@msimerson msimerson dnsbl test: don't cry about test failures that
depend on working network & DNS.
eb154f2
@msimerson msimerson SPF: use $conn->relay_client instead of duplicated
is_in_relayclients method. Expects relay plugin to have set relay_client, a reasonable assumption.
78cab52
@msimerson msimerson fixed test for commit 78cab52 c77e304
@msimerson msimerson SPF: removed test for removed is_in_relayclients() d75ce70
@msimerson msimerson headers: simplify required headers logic 60d0c8b
@msimerson msimerson dkim: added some missing POD text 81aa6a6
@msimerson msimerson arrange sample plugins by SMTP phase
and add comments to that effect, provides the uninitiated with clues
about which data each plugin actions upon
d0e47a9
@msimerson msimerson dspam: change reject 'agree' to .95 score b89272c
@msimerson msimerson several adjustments for tests f56c200
@msimerson msimerson run: added commented example for port 587 3355d5c
@msimerson msimerson MANIFEST: packaging update 5881267
@msimerson msimerson qmail_deliverable: reject null sender to ezmlm lis 1081461
@msimerson msimerson clamdscan: replace immunity check with naught test
immunity check was disabled by default, as it wasn't a good policy. OTOH, a naughty check is a sensible default, as we can skip processing on messages we already decided to reject.
bf7c663
@msimerson msimerson relay: better error handling and logging
detect failures in calls to Net::IP for relayclient entries that don't parse.
e9b582e
@msimerson msimerson dspam: improve logging and config error reporting 37cb63c
@msimerson msimerson replace all instances of split '' with split //
newer versions of perl don't accept split '' syntax any longer
df577ff
@msimerson msimerson helo: avoid undef warning when rDNS is invalid
specifically, when rDNS returns an invalid FQDN like 'null.', which doesn't have
a domain part.
77182ec
@msimerson msimerson updated more split '' syntax to split // 4a745d6
@msimerson msimerson log/summarize: improve formatting so vertical
columns are consistent, regardless of when the connection is ended.
aaaf69d
@msimerson msimerson v0.91 bump for release 5758412
@msimerson msimerson uribl plugin: added 'pass' prefix to log message d06eac3
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev 11874aa
@jokey2k jokey2k Update qpsmtpd-forkserver
Drop block as it breaks with Net::DNS and should be safe according to http://www.nntp.perl.org/group/perl.qpsmtpd/2012/12/msg9980.html
2661e08
@msimerson msimerson Merge pull request #1 from jokey2k/patch-1
Fix for Net::DNS break
1a9c074
@msimerson msimerson plugins/helo: added RFC 5321 notes 22d1603
@msimerson msimerson Merge branch 'master' of github.com:qpsmtpd-dev/qpsmtpd-dev
merge in Net::DNS patch
08357b3
@msimerson msimerson plugins/bogus_bounce: add Return-Path check
make sure return path is empty, per RFC 3834
c31074b
@msimerson msimerson p0f: added path to socket in error message
if p0f cannot connect, provide a more descriptive error message. Particularly useful for a p0f plugin developer that runs both p0f v2 and v3 at the same time.
0a6f23d
@msimerson msimerson karma_tool: release didn't. fixed.
also, preserve karma history when using karma_tool to capture/release
b37a046
@msimerson msimerson qmail_deliverable: remove fail prefix from SMTP er
prefix should only be logged, not emitted during SMTP
30b7662
@msimerson msimerson whitelist: added pass prefix to log entries a021234
@msimerson msimerson headers: added section # to RFC citation 548415e
@msimerson msimerson karma: general improvements
skip earlytalker checks for positive senders

limit negative karma senders to 1 concurrent connection (hosts_allow)
  added karma::hook_pre_connection, to make hosts_allow change possible

added karma score to log entries
57a2f68
@msimerson msimerson dspam: added use lib, removed some parens 537af7c
@msimerson msimerson helo: added comments a108629
@jokey2k jokey2k Sanitize spamd_sock path for perl taint mode f198157
@msimerson msimerson adjust_karma now increments properly 5f9aed1
@msimerson msimerson log/watch: raise default # of log lines to parse 170fdc9
@msimerson msimerson logs: suppress perl errors in summary output 03641b3
@msimerson msimerson karma_tool: optimized for speedy IP search, IPv6
fixed one IPv6 issue
7b804c7
@msimerson msimerson geoip: added too_far option 79a5c3d
@msimerson msimerson badrcptto: smite matches with -2 karma
useful for (reject=>naughty) + spam filter training
3b71f06
@msimerson msimerson dnsbl: smite blacklisted IPs with -1 karma d427f43
@msimerson msimerson dspam: be more conservative with karma awards
previous settings were reasonable for a well trained dspam. After starting with a fresh dspam, the settings were not optimal for the amount of naive that a default dspam is.
d37f875
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev 1a7f2c2
@msimerson msimerson Merge pull request #2 from jokey2k/patch-2
Sanitize spamd_sock path for perl taint mode
a116926
@msimerson msimerson Merge branch 'master' of github.com:qpsmtpd-dev/qpsmtpd-dev 08da0fe
@msimerson msimerson spamassassin: assign karma for autolearn message
also removed 'use lib', to be consistent with most other plugins
and improved grammar
c4fc2ec
@msimerson msimerson earlytalker: if we skip for +karma, log it
and remove IP from log (not IPv6 optimal)
279a43f
@msimerson msimerson headers: smite poorly behaved senders with -karma e01843f
@msimerson msimerson helo: smite senders that fail the selected tests
and made log entries more terse
6a41d1e
@msimerson msimerson hosts_allow: allow +karma senders +3 concurrents
this is really useful if you set max-per-ip to <= 3.
1dfa55c
@msimerson msimerson p0f: added smite_os, assign -karma by OS aaa2241
@msimerson msimerson relay: give +2 karma boost to relay IPs 309fdbe
@msimerson msimerson whitelist: add +5 karma to whitelisted IPs a639fc7
@msimerson msimerson karma: be a bit more conservative
require at least -2 karma before smiting
also, add +1 karma to senders with karma_history > 10
a5b3cc3
@msimerson msimerson naughty: improve POD 4e3b338
@msimerson msimerson qm_deliverable: added reject option, karma smite
award senders -1 karma to senders to invalid addresses
a5856d2
@msimerson msimerson fcrdns: new plugin for Forward Confirmed rDNS 12f1de2
@msimerson msimerson badmailfrom: fix reject message typo 31609e3
msimerson added some commits Apr 21, 2013
@msimerson msimerson dmarc: added relaxed alignment tests b59000c
@msimerson msimerson Plugin: override dns_timeout by passing in a value 8122fcf
@msimerson msimerson dkim: when signing, use signing domain when we
finding the signing key in a different directory than the sending (eg: example.com instead of www.example.com.)
278399f
@msimerson msimerson dmarc: weed out SPF records from initial search
use a variable instead of array to count list (not using RR address after all)
8f834d5
@msimerson msimerson moved tls plugin to the top of the config
it must be listed before other connection plugins for port 465
place it up there just in case
b7ce45a
@msimerson msimerson auth_chkpw: added pass|fail prefix to log msgs 7d88c51
@msimerson msimerson tls: added pass|fail prefix to a couple log msgs 7199743
@msimerson msimerson dkim: reduce INFO logging to once per connect f1aa848
@msimerson msimerson Makefile.PL: gzip -9, and clean up test db
and a perltidy
8b95e90
@msimerson msimerson log2sql: populate plugins table from registry.txt
much easier for local customizations.
moved SQL connection settings to config/log2sql
78ac01d
@msimerson msimerson qmail_deliverable: smite null sender to email list f63c029
@msimerson msimerson docs/logging: corrected example register() syntax 050aa4b
@msimerson msimerson added modules required by several of the plugins
and imported bin/install_deps.pl, preparing for a future where QP is almost easy to install
eab1d5e
@msimerson msimerson bump RAM from 150 to 200MB
DKIM message signing needs more RAM
78e7a0c
@msimerson msimerson split is_immune into itself + is_naughty
is_immune tests designates to plugins they should always skip processing.

That's typical for naughty connections, but this change provides the ability to handly naughty connections differently than (whitelisted/relayclients/known good) senders.
3d7d43e
@msimerson msimerson summarize shows a narrower screen by default.
passing in -l for when your term windows is more than 200 chars wide will show more detail
f41df6e
@msimerson msimerson dmarc: improving and updating POD 88e6ce6
@msimerson msimerson SPF: arrage flow so if a pass result is possible,
we will get it and set the note for DMARC plugin
5aafca3
@msimerson msimerson dmarc: added support for DMARC policy pct=NNN b4ee962
@msimerson msimerson SPF: added more precise disposition logs, so that
postprocess can determine if a SPF failure caused a rejection
981bdf5
@msimerson msimerson docs/logging: added description of log prefixes 6947c4f
@msimerson msimerson docs/logging: added description of log prefixes f7a5970
@msimerson msimerson distinguish rejecting versus tolerated failures 736e3b6
@msimerson msimerson karma: limit rcpts to 1 for senders with neg karma b3ca4e3
@msimerson msimerson rcpt_ok: do immunity checks earlier, so that
disposition logs don't indicate failure for authenticated senders
e3d8a70
@msimerson msimerson run: increase RAM from 200 to 300MB (dkim)
still seeing (infrequent) "too large" errors validating DKIM signatures
3d6f23f
@msimerson msimerson dmarc test: comments in the public list was
allowing certain org domain searches to fail (plus.google.com, b/c a google.com email address was in the public list). Now I anchor the searches to the start of the line. This test also catches edge cases like co.uk, which isn't listed, but a wildcard *.uk is.
8823de5
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev 5f27a1e
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev
Conflicts:
	Makefile.PL
632b42e
@msimerson msimerson see if removing Mail::SPF makes Travis happy 96c27d4
@msimerson msimerson .travis.yml: added perl 5.16 b7a00a3
@msimerson msimerson try disabling Time::TAI64, update MANIFEST d02fbd2
@msimerson msimerson Makefile.PL: disable Geo::IP module ad08e7b
@msimerson msimerson Makefile.PL: comment out Mail::Spamassassin af3d795
@msimerson msimerson Makefile.PL: reenable Time::TAI64 2a192c4
@msimerson msimerson Makefile.PL: added clean { *.bak } 9e204aa
@msimerson msimerson dmarc: added subdomain policy handling f854736
@msimerson msimerson install_deps: handle comments in Makefile.PL 5ca971d
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev fc83226
@msimerson msimerson Qpsmtpd.pm: split config args on /\s+/, was / / c3c5643
@msimerson msimerson Qpsmtpd: untaint config data passed to plugins
if QP passes in tainted data, such as a hostname that subsequently gets used to open a connection using IO::Socket, the plugin die because the information is tainted. Fix it once here, instead of in each plugin.
8e9e292
@msimerson msimerson auth_vpopmaild: added taint checking to responses 887e3ca
@msimerson msimerson replace run with separate run for the 2 common
deployment methods. Rather than having to edit the run file, it's much easier to rename the run file.
Moved qpsmtpd* into bin/
0b8e4f5
@msimerson msimerson revert movement of qp bins to bin/
plugin dir, config dir, spool dir, all have different logic about where/how to find their config. The logic needs some untangling and unification before attempting this again.
dfe8c1c
@msimerson msimerson Makefile.PL: added more disabled dependencies
DBI: commented out, but included for documentation's sake
80bb4c9
@msimerson msimerson summarize: strip out unprintable chars cebf995
@msimerson msimerson Makefile.PL, added comments, stating where the
disabled plugins are used
e8effdd
@msimerson msimerson summarize: move parts of main while loop to subs
and added POD
341865d
@msimerson msimerson updated Changes 51645b8
@msimerson msimerson SMTP.pm: reduce auth details from Received header. 5eab739
@msimerson msimerson MANIFEST: updated with run.* files 9d74793
@msimerson msimerson added daemontools, ucspi-tcp to install list ebe72f6
@msimerson msimerson spf enabled in config/plugins by default
the plugin will detect if Mail::SPF is missing and not register it's hooks
e7b7277
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd-dev 1731542
@msimerson msimerson raised default max msg size in clamdscan from 128k
added max_size on config, so it's likely to get noticed, since even 1M is probably too low for most sites. This should likely default to the same as databytes?
cc6ab49
@msimerson msimerson summarize: fix syntax error cf5f1bb
@msimerson msimerson is_naughty is a setter now too 9c095ab
@msimerson msimerson store envelope from and to in connection notes 5538824
@msimerson msimerson headers: assign zeroes to avoid undef errors 2a61746
@msimerson msimerson Makefile.PL: added commented Math::Complex c80bcf8
@msimerson msimerson added Authentication-Results header, with provider
dkim, dmarc, fcrdns (iprev), spf, and smtp-auth
3973f9a
@msimerson msimerson tested and working Authentication-Results
changed the method of saving results. Instead of appending to/from a header, plugins save results to a connection note.

Qpsmtpd::SMTP.pm has a new method that inserts the Authentication-Results header
The smtp-auth information has been removed from the Received header

Authentication-Results providing plugins have been updated to store results in connection note
a6b563a
@msimerson msimerson define positioning of Authentication-Results header e32154e
@msimerson msimerson remove plaintext UPGRADING (.pod added by Ask) c330517
@msimerson msimerson removed a diff block from docs/config.pod 5b3f616
@msimerson msimerson TcpServer: optimize DNS lookups for PTR
a. don't use search path (/etc/resolv.conf)
b. explicitely specify PTR in query request
b085388
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd
Conflicts:
	lib/Qpsmtpd/TcpServer.pm
	packaging/rpm/qpsmtpd.spec.in
1cc7563
@msimerson msimerson move Auth-Results header to Original-Auth-Results
this was in a sub, commented out as a TODO to delete them. Instead of deleting, move the Authentication-Results header on incoming messages to the Original-A-R.
a26d46e
@msimerson msimerson STATUS: removed -dev comments 7a9ae2c
@msimerson msimerson Changes: updated with 0.93 changes 2a12acc
@msimerson msimerson remove plaintext UPGRADING (.pod added by Ask) 14d5bad
@msimerson msimerson removed a diff block from docs/config.pod 6b4b714
@msimerson msimerson TcpServer: optimize DNS lookups for PTR
a. don't use search path (/etc/resolv.conf)
b. explicitely specify PTR in query request
ddb613f
@msimerson msimerson move Auth-Results header to Original-Auth-Results
this was in a sub, commented out as a TODO to delete them. Instead of deleting, move the Authentication-Results header on incoming messages to the Original-A-R.
19115cd
@msimerson msimerson STATUS: removed -dev comments 04634fe
@msimerson msimerson Changes: updated with 0.93 changes f78da4b
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd 6ea12f0
@msimerson msimerson Plugin.pm: made is_naughty is now a getter too 2416d1e
@msimerson msimerson log/summarize: set undefined strings as empty str
avoids undef warnings
012a7a4
@msimerson msimerson dmarc integrated with Mail::DMARC
reimplemented dmarc module to use Mail::DMARC
updated SPF plugin to save SPF results in dmarc_spf note
update dkim to store DKIM results in dkim_result & dkim_verifier notes
c202d3e
@msimerson msimerson added dmarc (run SPF & DKIM) first comment 7a855d4
@msimerson msimerson dspam: remove hard coded default in train_ methods 725a8d1
@msimerson msimerson headers: added POD descripting each header 96dfb08
@msimerson msimerson geoip: added named array for invalid args
so it passes Perl::Critic tests
a4695ce
@msimerson msimerson updated DMARC plugin tests
disabled for now, b/c they tested methods which no longer exist in new plugin
a19b7de
@msimerson msimerson Qpsmtpd: version bump to 0.94 fd4cc6f
@msimerson msimerson tls: reduced importants of an info message
from WARN to INFO
9f88e37
@msimerson msimerson karma: added penalty for spammy TLDs 02da55e
@msimerson msimerson helo: add karma penalty for no HELO hostname bcc6ada
@msimerson msimerson anglebrackets: increase penalty, prefix log msgs 4531648
@msimerson msimerson naughty: legibility improvement 2d4f4a2
@msimerson msimerson updated Changes with some 0.94 commits 0e0cda6
@msimerson msimerson Merge branch 'master' of github.com:msimerson/qpsmtpd 81bf413
@msimerson msimerson clamdscan: add support for remote TCP/IP clamd
previous version only worked when clamd was running on the same machine and had access to the spool file. This version also works with a remote clamd.
3353578
@msimerson msimerson Merge branch 'master' of git://github.com/smtpd/qpsmtpd
Conflicts:
	Changes
8952644
@msimerson msimerson headers: use a more descriptive variable name 4d1b9ff
@msimerson msimerson dmarc: skip processing for null sender 3a47dd2
@msimerson msimerson domainkeys: fixed doc typo 0fee545
@msimerson msimerson merged commit 0c41c01 into smtpd:master Jan 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.