lfarm is a Common Lisp library for distributing work across machines using the lparallel API.
In lparallel a kernel was defined as abstract entity that schedules and executes tasks. lparallel implements it with a thread pool, while in lfarm it is implemented with a set of servers that execute tasks.
;; Create two servers bound to ports 11111 and 22222. (ql:quickload :lfarm-server) (lfarm-server:start-server "127.0.0.1" 11111 :background t) (lfarm-server:start-server "127.0.0.1" 22222 :background t) ;; Connect to the servers. `lfarm' is a package nickname for `lfarm-client'. (ql:quickload :lfarm-client) (setf lfarm:*kernel* (lfarm:make-kernel '(("127.0.0.1" 11111) ("127.0.0.1" 22222)))) ;; Use the lparallel API. (defpackage :example (:use :cl :lfarm)) (in-package :example) (let ((channel (make-channel))) (submit-task channel #'+ 3 4) (receive-result channel)) ;; => 7 (let ((f (future (+ 3 4)))) (force f)) ;; => 7 (plet ((x (+ 3 4)) (y (+ 5 6))) (+ x y)) ;; => 18 (pmapcar '1+ #(1 2 3)) ; => (2 3 4) (pmapcar #'1+ #(1 2 3)) ; => (2 3 4) (preduce '+ #(1 2 3)) ; => 6 (pmap-reduce '1+ '+ #(1 2 3)) ; => 9
Although the servers in this example are local, lfarm servers may run in separate Lisp instances on remote machines.
There are some restrictions on a task slated for remote execution. A task must be
- a lambda form, or
- a function that exists on the remote servers, or
- a function defined with
deftask is just like
defun except the function definition is
recorded. (A Lisp implementation may record a function definition, but
is not required to do so.)
(deftask add (x y) (+ x y)) (let ((channel (make-channel))) (submit-task channel #'add 3 4) (receive-result channel)) ;; => 7
submit-task notices that
add was defined with
converts it to a named lambda before submitting it to a server.
add remotely use
broadcast-task, which executes a given
task on all servers.
(broadcast-task (lambda () (defun add (x y) (+ x y))))
Or more likely
add would be part of a system that is loaded on all
(broadcast-task #'ql:quickload :my-stuff)
Limited support for closures is available on SBCL, CCL, LispWorks, and
Allegro. Lexical variables and symbol macrolets are captured, but
lexical functions (
labels) are not.
Tasks are not macroexpanded in order to ensure portability across clients and servers.
lfarm-client system defines the
lfarm-client package which has
lfarm. It exports the lparallel kernel
API with the following differences.
- tasks have the aforementioned restrictions placed upon them
- the addition of
make-kernelexpects addresses, and lacks the
task-handler-binddoes not exist
*kernel-spin-count*exist but have no effect
submit-taskis a macro that wraps
submit-task*(see the Details section)
- the addition of
broadcast-taskwhich similarly wraps
task-execution-erroris signaled when a task fails on a remote server, instead of the actual error (which may not have local meaning)
lfarm-admin provide the following functions.
lfarm-server:start-server host port &key background name-- Start a server instance listening at
backgroundis true then spawn the server in a separate thread named
lfarm-admin:ping host port &key timeout-- Send a ping to the lfarm server at
port. Keep trying to make contact for
timeoutseconds, or if
timeoutis nil then try forever. Default is 3 seconds. Returns true if successful and nil otherwise.
lfarm-admin:end-server host port-- End the server at
port. This only stops new connections from being made. Connections in progress are unaffected.
The purpose of an lfarm server is to execute arbitrary code, so it is highly advised to enable some form of security. lfarm directly supports Kerberos (or Active Directory) authentication. Alternatively, SSH tunnels may be used.
Security with SSH tunneling
;; On the remote machine (ql:quickload :lfarm-server) (lfarm-server:start-server "127.0.0.1" 33333)
To create a tunnel,
# On the local machine $ ssh -f -L 33333:127.0.0.1:33333 <remote-address> -N
The remote server should now be accessible locally.
;; On the local machine (ql:quickload :lfarm-admin) (lfarm-admin:ping "127.0.0.1" 33333) ;=> T
Of course there is still local security to consider, as local users on both ends have access to the server. If this is a concern then a packet filtering tool such as iptables may be used.
Security with Kerberos/GSSAPI
lfarm-gss system provides support for GSSAPI authentication. The
:auth argument to
lfarm-client:make-kernel accepts an instance of
When creating a server, the class
the initialization keyword
:service-name. This value is indicats
which service type should be used when requesting a ticket for the
remote service. The default is
lfarm. In other words, if an attempt
is done to connect to the server at
server.example.com, the service
principal will be
When creating a kernel (client), the class
accepts the initialization keyword
:allowed-users which specifies a
list of all users that are allowed to connect to the server. Each
element should be a string representing the principal name (including
realm) of the user that is allowed to connect. For example:
If a more complex authorization mechanism is needed which is not
covered by the simple user list as described above, you can subclass
gss-auth-server class and then implement the method
lfarm-gss:name-accepted on your new class. This generic function
takes two arguments, the authentication object and the name to be
verified, and should return non-NIL if the user is allowed to connect.
Note that the name is an instance of
cl-gss:name, and you need to
call the function
cl-gss:name-to-string on it to extract the actual
The server needs to have access to the service principal in a keytab file. How to create the keytab file depends on your Kerberos server implementation:
For Heimdal: http://www.h5l.org/manual/HEAD/info/heimdal/keytabs.html (don't forget to add the
-kflag to specify the file to which the key should be written)
For Active Directory: http://technet.microsoft.com/en-us/library/bb742433.aspx
Once you have the keytab file, you have to make sure that it is
loaded. The easiest way to do this is to simply call
CL-GSS:KRB5-REGISTER-ACCEPTOR-IDENTITY and pass in the name of the
The other way is to make sure the environment variable
is set to the path of the keytab file and that it is readable by the
lfarm server instance.
If the keytab file has not been loaded, the server will fail to authenticate and you will get a security error when the client attempts to connect to the server.
That covers perhaps all you need to know about lfarm. Those who are curious may read on (or not).
Serialization is done with cl-store. It uses a portable serialization format, allowing lfarm clients and servers to run on different Lisp implementations.
A symbol is deserialized on the remote server with its home package intact. If the server encounters a symbol whose package does not exist, an empty version of the package is automatically generated.
The lfarm client is obstinate with regards to connections: if there is
a connection error then it tries to reconnect, and will continue
trying. We may therefore restart servers while using the same kernel
instance, or call
make-kernel before any servers exist (the call
will block until they do).
Note it is possible for a task to be executed twice (or more). If a connection error occurs in the time interval after a task has been submitted and before its result has been received, the client will attempt to submit the task again.
submit-task is a function, but in lfarm it is a macro
that provides syntactic sugar for the function
(submit-task channel #'+ 3 4) ;; =macroexpand=> (SUBMIT-TASK* CHANNEL '+ 3 4) (submit-task channel (lambda (x) (1+ x)) 3) ;; =macroexpand=> (SUBMIT-TASK* CHANNEL '(LAMBDA (X) (1+ X)) 3)
submit-task may alter the task argument before giving it to
submit-task*, which expects a symbol or a lambda form. Sharp-quote
is replaced with quote, and a lambda form gets quoted. This provides a
lparallel:submit-task and relieves us from having to
'(lambda ...) and
'f in place of
(lambda ...) and
Verbose logging is enabled by binding
:info (default is
:error). The log stream is
lfarm-common:*log-stream* (default is
The lfarm test suite assumes a working ssh executable is present and
that passwordless authorization has been set up for "ssh localhost".
To run it load the
lfarm-test system and call
which may be given some configuration options. Unrecognized Lisp
implementations will require configuration (namely, specifying the
lisp executable and the command-line switch to eval a form). Tests
also assume that Quicklisp has been installed (but not necessarily
loaded), although configuration may remove this assumption.
The client has an internal lparallel kernel in which each worker thread manages a connection to an assigned remote server, one worker per server. When a worker connects to a server, the server enters a task execution loop wherein a form is deserialized, maybe compiled, and funcalled; repeat. A server may serve multiple clients.
Though an async backend is possible, this threaded implementation was chosen because it was easy and portable.
Opportunities for optimization in the realm of remote task queues and remote task stealing have been callously ignored. Task queues are local.
James M. Lawrence email@example.com
Kerberos support by Elias Martenson firstname.lastname@example.org