Skip to content
This repository

This branch is 0 commits ahead and 0 commits behind master

..
Octocat-spinner-32 README.md run-agent.sample.lisp now installs Quicklisp for the main tg-agent pr… March 06, 2014
Octocat-spinner-32 agent.lisp agent: log quicklisp update process into <work-dir>/agent/quicklisp-u… February 12, 2014
Octocat-spinner-32 api-compatible-p.lisp Polish agent API a little. July 26, 2012
Octocat-spinner-32 as-singleton-agent.lisp singleton-lock-port is a public property of agent. Improve agent logg… August 07, 2012
Octocat-spinner-32 complete-test-run.lisp Minor code improvement in tg-agent::fmt-time and tg-agent::pretty-fmt… February 12, 2014
Octocat-spinner-32 generate-id.lisp Fixed typos and wording in comments. July 25, 2012
Octocat-spinner-32 killproctree.sh Implementing process tree kill for Unix-like platfomrs: improve agent… November 08, 2013
Octocat-spinner-32 lisp-exe-ccl.lisp improve phrasing in a comment September 21, 2012
Octocat-spinner-32 lisp-exe.lisp lisp-exe:lispworks : add (load-all-patches) for non-Windows too January 21, 2014
Octocat-spinner-32 package.lisp agent/package.lisp - move the deprecated functions to the bottom of t… March 06, 2014
Octocat-spinner-32 persistence.lisp Splitting the monolitic module test-grid into: testsuites, agent, blo… August 01, 2012
Octocat-spinner-32 proc-common-asdf.lisp Adopt our custom asdf-output-translations to the ASDF requirement des… April 04, 2013
Octocat-spinner-32 proc-common.lisp agent: bind *trace-output* to log files when logging ASFD system load… February 12, 2014
Octocat-spinner-32 proc-implementation-identifier.lisp tg-agent::implementation-identifier - copy/paste from ASDF again, to … November 07, 2013
Octocat-spinner-32 proc-list-quicklisp-projects.lisp The list of projects to test should be reteieved from the quicklisp b… September 17, 2012
Octocat-spinner-32 proc-run-libtest.lisp agent: Fixed a typo in the test log header: ASDF version*: => ASDF ve… February 13, 2014
Octocat-spinner-32 proc-test-loading.lisp agent: Fixed a typo in the test log header: ASDF version*: => ASDF ve… February 13, 2014
Octocat-spinner-32 proc-update-quicklisp.lisp agent/proc-update-quicklisp.lisp - fix typo in the commented out sect… March 07, 2014
Octocat-spinner-32 project-lister.lisp tiny refactoring November 30, 2013
Octocat-spinner-32 quicklisp.lisp Update the the Quicklisp bootstrap file agent/quicklisp.lisp to the l… March 06, 2014
Octocat-spinner-32 require-quicklisp.lisp run-agent.sample.lisp now installs Quicklisp for the main tg-agent pr… March 06, 2014
Octocat-spinner-32 send-notification.lisp Send email notifications through the Heroku server. February 14, 2013
Octocat-spinner-32 submit-results.lisp Send email notifications through the Heroku server. February 14, 2013
Octocat-spinner-32 with-response-file.lisp Get rid of *RESPONSE-FILE-TEMP-DIR* and use LISP-EXE:*TEMP-DIR* instead January 17, 2014
README.md

Running Tests

test-grid-agent:agent is a lisp object able to manage test execution by subordinate lisp implementations (executables) and submit test results to server.

It is created with function test-grid-agent:make-agent and has 3 required configuration properties:

  • lisps - Paths to the lisp implementations that should be used to run tests.

  • preferred-lisp - The lisp implementation used when it is necessary to perform an auxiliary task requiring a separate lisp process, for example downloading libraries to be tested. It is therefore desirable to specify here a lisp implementation known to work reliably on your platform.

  • user-email - Your email so that we know who is contributing the test results and can contact you. The email is also published in the test results reports so that library authors or other interested parties can contact you with questions about your platform. If you are strongly opposed to publish your email, you can specify just some nickname here.

Function test-grid-agent:main runs the agent.

It is necessary to perform git pull on agent sources often.

There are template scripts demonstrating how to load, configure and run agent by a single command.

Please use CCL - it is the development platform and the only lisp known to run agent successfully.

So, the steps:

  1. git clone git://github.com/cl-test-grid/cl-test-grid.git
  2. cd cl-test-grid
  3. cp run-agent.sh.sample run-agent.sh; chmod +x run-agent.sh
  4. cp run-agent.sample.lisp run-agent.lisp
  5. Edit the run-agent.sh (edit one line - the path to CCL).
  6. Edit the run-agent.lisp (paths to the lisp implementations, your email)
  7. ./run-agent.sh

Next time all you need is to just invoke ./run-agent.sh. It will update the cl-test-grid from git, run tests and upload the results.

Agent keeps log files in the cl-test-grid/work-dir/agent/logs/, where you can control what it has done.

Example crontab record to run agent at 10 o'clock every day:

  # minute hour day_of_month month day_of_week command
  0 10 * * * cd /home/testgrid/cl-test-grid/ && ./run-agent.sh

Details of what agent actually does

Simplified, the agent mode of operation may be represented by the following pseudo code:

   (let ((current-quicklisp (update-quicklisp)))
     (loop for lisp in my-lisp-implementations
        (when (not (tested-already lisp current-quicklisp))
           (let ((results-dir (complete-test-run lisp (or (find-unfinished-test-run lisp current-quicklisp)
                                                          (make-new-test-run lisp current-quicklisp)))))
              (submit-results results-dir)
              (remember-tested lisp current-quicklisp)
              (cl-fad:delete-directory-and-files results-dir)))))

As you can see, the agent submits test results after completing full test set on a single lisp implementation.

The code, including the internal implementation of complete-test-run is organized so that agent can be interrupted (computer rebooted or hibernated, agent process killed). When started again, it continues the work from the point of interruption.

Testing single lisp implementation may take from 1-2 hours up to 10 hours or more (for ABCL - ABCL has long startup time, which becomes significant in our use case as we run every test suite or ASDF system compilation in a fresh lisp process).

Caveat of killing the agent: if you killed the agent process (without rebooting the machine), the subordinate process running current testsuite or compiling current ASDF system remains alive. Typically it takes less than a minute for it to finish, but sometimes it may take longer (the testsuite or library compilation may require longer time; or, in the worst case, test suite may hang). If you start agent again, it spawns new test running process, which can interfere with the old one via file system (.fasl files, output logs). Therefore it's better to give the old child process time to finish before starting the agent again.

Parallel execution of multiple agents

Agent operates sequentially.

During its work, agent keeps its working data in a directory specified by the configuration property

  • work-dir - Defaults to the <cl-test-grid source code root>/work-dir/agent/

The agent takes measures to ensure there is only one agent instance using this working directory.

This is achieved by using a TCP port as an inter-process lock. When started, agent tries to open a socket on the port. If it is successful, the agent continues. If the port is busy, the agent deduces there is another agent instance running, logs a warning and exists.

The port number is specified by the configuration property

  • singleton-lock-port defaults to 7685.

If you want to run several agent processes and distribute testing work between them, you can assign each agent different set of lisp implementations and give each agent different working directory and lock port.

Getting assistance

Feel free to contact us if you have any questions or difficulties (see the mailing list address below).

Source Code Guide

The key components, from bottom up:

  • lisp-exe Represents lisp implementation executable and allows to start child lisp process to execute some lisp code. Supports timeouts: if the child lisp process exceeds the specified timeout, it is killed and corresponding condition is signalled.
  • On top of it the function complete-test-run is built. It runs tests on the specified lisp-exe and quicklisp installation, and stores test results in a directory. If the testing process is interrupted and started again with the same output directory, it continues from the point of interruption.
  • submit-test-run-results submits log files and test results from test run output directory to online storage.
  • Finally, on top of these components agent object is build. It maintains private quicklisp installation in working directory and updates it to the latest version every time when started. It also remembers what lisp implementations were tested on what quicklisp version. This information kept in persistence - a simple plist saved in file. Also agent checks the configuration parameters provided by user and ensures only one agent instance is run at time, using a TCP port, as described above. The agent public API is specified in the package.lisp.

The source files naming convention proc-*.lisp is used for files executed by child lisp processes started via lisp-exe.

Something went wrong with that request. Please try again.