test-grid-agent:agent is a lisp object able
to manage test execution by subordinate lisp
implementations (executables) and submit test
results to server.
It is created with function
and has 3 required configuration properties:
lisps- Paths to the lisp implementations that should be used to run tests.
preferred-lisp- The lisp implementation used when it is necessary to perform an auxiliary task requiring a separate lisp process, for example downloading libraries to be tested. It is therefore desirable to specify here a lisp implementation known to work reliably on your platform.
user-email- Your email so that we know who is contributing the test results and can contact you. The email is also published in the test results reports so that library authors or other interested parties can contact you with questions about your platform. If you are strongly opposed to publish your email, you can specify just some nickname here.
test-grid-agent:main runs the agent.
It is necessary to perform
git pull on agent sources
There are template scripts demonstrating how to load, configure and run agent by a single command.
Please use CCL - it is the development platform and the only lisp known to run agent successfully.
So, the steps:
git clone git://github.com/cl-test-grid/cl-test-grid.git
cp run-agent.sh.sample run-agent.sh; chmod +x run-agent.sh
cp run-agent.sample.lisp run-agent.lisp
- Edit the run-agent.sh (edit one line - the path to CCL).
- Edit the run-agent.lisp (paths to the lisp implementations, your email)
Next time all you need is to just invoke
./run-agent.sh. It will update the
cl-test-grid from git, run tests and upload the results.
Agent keeps log files in the cl-test-grid/work-dir/agent/logs/, where you can control what it has done.
Example crontab record to run agent at 10 o'clock every day:
# minute hour day_of_month month day_of_week command 0 10 * * * cd /home/testgrid/cl-test-grid/ && ./run-agent.sh
Details of what agent actually does
Simplified, the agent mode of operation may be represented by the following pseudo code:
(let ((current-quicklisp (update-quicklisp))) (loop for lisp in my-lisp-implementations (when (not (tested-already lisp current-quicklisp)) (let ((results-dir (complete-test-run lisp (or (find-unfinished-test-run lisp current-quicklisp) (make-new-test-run lisp current-quicklisp))))) (submit-results results-dir) (remember-tested lisp current-quicklisp) (cl-fad:delete-directory-and-files results-dir)))))
As you can see, the agent submits test results after completing full test set on a single lisp implementation.
The code, including the internal implementation
complete-test-run is organized so that agent can
be interrupted (computer rebooted or hibernated,
agent process killed). When started again, it continues
the work from the point of interruption.
Testing single lisp implementation may take from 1-2 hours up to 10 hours or more (for ABCL - ABCL has long startup time, which becomes significant in our use case as we run every test suite or ASDF system compilation in a fresh lisp process).
Caveat of killing the agent: if you killed the agent process (without rebooting the machine), the subordinate process running current testsuite or compiling current ASDF system remains alive. Typically it takes less than a minute for it to finish, but sometimes it may take longer (the testsuite or library compilation may require longer time; or, in the worst case, test suite may hang). If you start agent again, it spawns new test running process, which can interfere with the old one via file system (.fasl files, output logs). Therefore it's better to give the old child process time to finish before starting the agent again.
Parallel execution of multiple agents
Agent operates sequentially.
During its work, agent keeps its working data in a directory specified by the configuration property
work-dir- Defaults to the <cl-test-grid source code root>/work-dir/agent/
The agent takes measures to ensure there is only one agent instance using this working directory.
This is achieved by using a TCP port as an inter-process lock. When started, agent tries to open a socket on the port. If it is successful, the agent continues. If the port is busy, the agent deduces there is another agent instance running, logs a warning and exists.
The port number is specified by the configuration property
singleton-lock-portdefaults to 7685.
If you want to run several agent processes and distribute testing work between them, you can assign each agent different set of lisp implementations and give each agent different working directory and lock port.
Feel free to contact us if you have any questions or difficulties (see the mailing list address below).
Source Code Guide
The key components, from bottom up:
lisp-exeRepresents lisp implementation executable and allows to start child lisp process to execute some lisp code. Supports timeouts: if the child lisp process exceeds the specified timeout, it is killed and corresponding condition is signalled.
- On top of it the function
complete-test-runis built. It runs tests on the specified
lisp-exeand quicklisp installation, and stores test results in a directory. If the testing process is interrupted and started again with the same output directory, it continues from the point of interruption.
submit-test-run-resultssubmits log files and test results from test run output directory to online storage.
- Finally, on top of these components
agentobject is build. It maintains private quicklisp installation in working directory and updates it to the latest version every time when started. It also remembers what lisp implementations were tested on what quicklisp version. This information kept in
persistence- a simple plist saved in file. Also agent checks the configuration parameters provided by user and ensures only one agent instance is run at time, using a TCP port, as described above. The agent public API is specified in the package.lisp.
The source files naming convention proc-*.lisp is used for files executed
by child lisp processes started via