DEMYLTIFY: a milter library in Common Lisp
This is a Common Lisp library to implement milters. A milter is a Sendmail filter (hence the contraction); a daemon program that extends and augments the Sendmail functionality and implements features that are not provided by Sendmail itself, such as spam filtering, virus protection, mail archiving, mailing lists etc. Matter of fact, much of the logic behind Sendmail routing and access control could, in fact, be off loaded to a milter or a composition of milters.
Milters are usually C programs linked to the libmilter library, which comes with Sendmail. Interfacing to such library is not always an option, especially for many Lisp systems.
The libmilter library implements the milter protocol, the (de)serialisation of the data and the multi-threading. This is what demyltify does as well, in a more lispy style.
Milters written with demyltify don’t need to be multi-thread. It’s up
to you whether, in the
ON-CONNECTION callback, to spawn a new
thread, fork a new process, or simply do nothing special to handle
How it works
The program calls
START-MILTER passing a port number and a function.
The milter library binds a socket to that port and waits for Sendmail
For each connection, the milter library calls the callback function
that was provided to
START-MILTER, passing a socket. The callback,
in turn, must call
SERVER-LOOP with the context object that usually
will contain further state data and milter options such as the
On each event received from Sendmail, the library calls the relevant handler (method). Each event method accepts an event object and a context object, and returns an action object.
To install and compile
- link demyltify.asd into your ASDF system directory
- start your Lisp
(asdf:oos 'asdf:load-op :demyltify)
To use this library, all you have to do is:
- write your own context class inheriting from
- specialise the
HANDLE-EVENTmethods on your
MILTER-CONTEXTclass for all the events you care about (the default definition will simply let any mail through)
The default options negotiation method will signal an error condition if the MTA doesn’t fully support the milter prerequisites. This is a sensible behaviour considering that, if the MTA doesn’t match the performed actions and required events of the milter, there is very little the milter can do about it; it will simply not work.
HANDLE-EVENT methods must return an action symbol or object which
will be sent to the MTA. The action without arguments are specified
as keywords. Those are
:CONTINUEget on with the next event
:ACCEPTaccept the message
:REJECTbounce the message
:DISCARDsilently ignore the message
:PROGRESShang on, the milter is performing some lengthy computation
:TEMPORARY-FAILUREthe message can’t be processed by the milter because of a temporary problem
This library is mostly stateless, so the program, if needs to, is
responsible to save its state in the context object. To do that you
are supposed to write your own context class which inherits from
MILTER-CONTEXT and pass it to
The lifetime of a context object is the same as the Sendmail
connection. The user program has to make sure that it resets whatever
state, in the context, that is message-specific, at every message
boundary. Usually good places are the
START-MILTER is a procedure that never exits under normal
circumstances. It enters a loop serving MTA connections on the
specified socket. It is appropriate for the
ON-CONNECTION function to
fork or fire a new thread. You don’t need to use
START-MILTER, if you
want to write your own server function, go ahead, but for most
practical purposes it does what you need to connect to Sendmail.
The event handlers
The event handlers are CLOS methods specialised on the event type and the context.
;; here we add up the byte count per message (defmethod handle-event ((e event-body) (ctx my-context)) (incf (ctx-byte-count ctx) (length (event-body-data e))) keep-going) ;; at the beginning of each message we reset the counter (defmethod handle-event ((e event-mail) (ctx my-context)) (setf (ctx-byte-count ctx) 0) keep-going)
The events a milter can handle are:
EVENT-ABORTwhen Sendmail aborts the current message (others may follow)
EVENT-BODYa chunk of the message body (passed the headers)
EVENT-CONNECTwhen a client MTA connects to our Sendmail
EVENT-DATAmarks the beginning of the message body
EVENT-DISCONNECTSendmail wishes to disconnect but it will connect again later
EVENT-END-OF-HEADERSto signal the end of the email’s headers part
EVENT-END-OF-MESSAGEat the end of a message body
EVENT-HEADERfor each email header
EVENT-HELLOwhen Sendmail sees a HELO from its client
EVENT-MAILwhen Sendmail receives a MAIL command from its client
EVENT-QUITwhen Sendmail asks the milter to lay down and die
EVENT-RECIPIENTfor each recipient on the email envelope
EVENT-UNKOWNinvalid SMTP command from Sendmail’s client
Internally the milter library handles the following events. In normal circumstances you shouldn’t bother with them:
EVENT-DEFINE-MACROdefinition of symbolic values that supplement other events
EVENT-OPTIONSnegotiation of event and actions between Sendmail and the milter
A context class derives from a
MILTER-CONTEXT like this:
;; we specialise the context to add the byte count per message (defclass my-context (milter-context) ((byte-count :accessor ctx-byte-count)))
To start the milter you simply call
start-milter and you pass the
internet port and the connection callback. The callback will be
called with a socket as argument and, in turn, it should call
server-loop passing a milter context of your choice. Example:
(defun start-milter-loop (socket) (be context (make-instance 'my-context :socket socket :events '(:mail :body) :actions '(:add-header)) (server-loop context))) (defun start-my-milter () (let ((*log-file* #P"mymilter.log")) (start-milter 20025 #'start-milter-loop)))
Sendmail before some events passes some additional data to the milter.
This data is in form of values associated to a symbolic name
(macro) such as
_ (the connection host),
rcpt_host, etc. An association list, at the end of the day.
A milter may access these values with the
passing the current context and the macro name as a string. Example:
(let ((host (get-macro ctx "_"))) (format t "Got connection from ~A~%" host))
EVENT-RECIPIENT handler method it might be used like this:
(defmethod handle-event ((event event-recipient) (ctx my-context)) (push (make-recipient :address (extract-mail-address (event-recipient-address event)) :mailer (get-macro ctx "rcpt_mailer") :host (get-macro ctx "rcpt_host")) (ctx-my-recipients ctx)) :continue)
To install a milter in Sendmail, in /etc/mail/sendmail.mc, you have to add a line like these:
INPUT_MAIL_FILTER(`filter1', `S=unix:/var/run/demyltify.socket, F=T') INPUT_MAIL_FILTER(`filter2', `S=inet:20025@localhost, F=T')
and compile the .mc into a .cf file:
cd /etc/mail make make install restart
Then make sure you use the same address in the call of
(start-milter #P"/var/run/demyltify.socket" #'my-connect-callback) (start-milter 20025 #'my-start-milter-loop)
F=T flag tells Sendmail to treat milter-related errors (ie milter
not listening or crashing) as temporary. Read the Sendmail’s
cf/README file if you need further details.
Sendmail does not start the milters. You have to do that yourself at boot time (anyhow, before Sendmail needs them to process a message).
Some sample code is in the examples directory:
- simple.lisp is a milter that counts bytes in messages
- threaded.lisp is the threaded version of simple.lisp
- forked.lisp is the multi-process version of simple.lisp
The following pages could be useful to understand what a milter is and what it does:
There is also a version of this library for Clojure, which is available on GitHub at http://github.com/fourtytoo/demyjtify
This work is based on an informal description of the undocumented Sendmail-milter protocol. This code may therefore be outdated right now, as the Sendmail folks don’t want you to mess with their protocol. They rather want you to use their pthread-based libmilter library in C. Although, in practice, it’s unlikely that this code will be invalidated by the next few Sendmail versions, you never know.
This code has been tested on SBCL, CMUCL and CLISP. Porting to other Lisp systems should be fairly easy.
Credit should be given to Todd Vierling (firstname.lastname@example.org, email@example.com) for documenting the MTA/milter protocol and writing the first implementation in Perl.
Copyright © 2004-2015 Walter C. Pelissero <firstname.lastname@example.org>
Distributed under the GNU Lesser General Public License either version 2 or (at your option) any later version.