Reference

ThomasHermann edited this page Sep 7, 2014 · 16 revisions

Functions and macros exported by lisp-unit are listed on this page.

Functions for managing tests

(define-test name
  [[documentation | tags*]]
 exp1 exp2 ...)

This macro defines a test called name with the expressions specified, in the package specified by the value of *package* in effect when define-test is executed. The expressions are assembled into runnable code whenever needed by run-tests. Hence you can define or redefine macros without reloading tests using those macros.

A documentation string can be defined for the test. The test can also be tagged and the tags used to run subsets of the tests. The following is an example of define-test with documentation and tags. Functions for managing and using tags are documented below.

(define-test foo
  "This is the documentation."
  (:tag :tag1 :tag2 symtag)
  exp1 exp2 ...)
(list-tests [package])

This function returns the names of all tests that have been defined for package. Use *package* if package is not specified.

(test-code name [package])

This function returns the code stored for the test name in package. Use *package* if package is not specified.

(remove-tests '(name1 name2 ...) [package])
(remove-tests :all [package])
(remove-tests)

This function removes the listed tests for the given package. All tests are removed with no arguments or if the keyword :all is provided instead of a list of names. If no package is given, the value of *package* is used.

(run-tests '(name1 name2 ...) [package])
(run-tests :all [package])
(run-tests)

This function runs the listed tests and reports the a summary of the results. All tests are run with no arguments or if the keyword :all is provided instead of a list. The package used is the value of *package* in effect when the macro is expanded. run-tests returns a test-results object containing the detailed test results. Test results are described below.

(use-debugger [flag])

By default, errors that occur while running tests are simply counted and ignored. You can change this behavior by calling use-debugger with one of three possible flag values: t (the default) means your Lisp's normal error handling routines will be invoked when errors occur; :ask means you will be asked what to do when an error occurs, and nil means errors are counted and ignored, i.e., the standard behavior.

Functions for managing tags

(list-tags [package])

This function returns the names of all tags that have been defined for package. Use *package* if package is not specified.

(tagged-tests '(tag1 tag2 ...) [package])
(tagged-tests :all [package])
(tagged-tests)

This function returns the tests associated with the listed tags. All tagged tests are returned with no arguments or if the keyword :all is provided instead of a list of tags. Use *package* if package is not specified.

(remove-tags '(tag1 tag2 ...) [package])
(remove-tags :all [package])
(remove-tags)

This function removes the listed tags for the given package. All tags are removed with no arguments or if the keyword :all is provided instead of a list of tags. If no package is given, the value of *package* is used.

(run-tags '(tag1 tag2 ...) [package])
(run-tags :all [package])
(run-tags)

This function runs the tests associated with the listed tags and reports the a summary of the results. All tags are run with no arguments or if the keyword :all is provided instead of a list. The package used is the value of *package* in effect when the macro is expanded. run-tags returns a test-results object containing the detailed test results. Test results are described below.

Test Results Database

Details of the unit tests are recorded in a test-results-db object. These details can be retrieved using the following functions.

(test-names <test-results-db>)

Return a list of the tests that were evaluated to generate the results.

(print-failures <test-results-db>)

Print the details of the failed assertions. The failure details can also be printed as the tests are run by setting *print-failures* to T.

(failed-tests <test-results-db>)

Return a list of the tests that contained failed assertions. This command can be combined with run-tests to focus on the failed tests.

(print-errors <test-results-db>)

Print the details of the assertions with errors. The error details can also be printed as the tests are run by setting *print-errors* to T.

(error-tests <test-results-db>)

Return a list of the tests that contained assertions with execution errors. This function can be combined with run-tests and *print-errors* to examine only tests that contained execution errors.

(missing-tests <test-results-db>)

Return a list of the tests that were not found in the test database.

(summarize-results <test-results-db>)

Print a summary of the results in the test-results object.

Signaling Test Completion

(signal-results [flag])
(results <test-run-complete condition>)

If signal-results is set to true, a test-run-complete condition is signaled when all of the tests specified by run-tests or run-tags are completed. This condition contains the test results database that is accessed using the results accessor.

Print parameters

*print-summary*
*print-failures*
*print-errors*

All of the lisp-unit print parameters are NIL by default. When set to true, *print-summary* will print a pass, fail, and error summary for each individual test when the tests are run. Similarly, setting *print-failures* and *print-errors* to true prints failures and executions errors for each individual assertion, respectively. In addition, if any of the print parameters are set equal to true, the individual test summaries are printed.

Forms for assertions

All of the assertion forms are macros. They tally a failure if the associated predication returns false. Assertions can be made about return values, printed output, macro expansions, and even expected errors. Assertion form arguments are evaluated in the local lexical environment.

All assertion forms allow you to include additional expressions at the end of the form. These expressions and their values will be printed only when the test fails.

Return values are unspecified for all assertion forms.

(assert-eq value form [form1 form2 ...])
(assert-eql value form [form1 form2 ...])
(assert-equal value form [form1 form2 ...])
(assert-equalp value form [form1 form2 ...])
(assert-equality predicate value form [form1 form2 ...])

These macros tally a failure if value is not equal to the result returned by form, using the specified equality predicate. In general, assert-equal is used for most tests. Example use of assert-equality:

      (assert-equality #'set-equal '(a b c) (unique-atoms '((b c) a ((b a) c))))
(assert-true test [form1 form2 ...])
(assert-false test [form1 form2 ...])

assert-true tallies a failure if test returns false. assert-false tallies a failure if test returns true.

(assert-prints "output" form [form1 form2 ...])

This macro tallies a failure if form does not print to standard output stream output equal to the given string, ignoring differences in beginning and ending newlines and spaces.

(assert-expands expansion form [form1 form2 ...])

This macro tallies a failure if (macroexpand-1 form) does not produce a value equal to expansion.

(assert-error condition-type form [form1 form2 ...])

This macro tallies a failure if form does not signal an error that is equal to or a subtype of condition-type. Use error to refer to any kind of error. See condition types in the Common Lisp Hyperspec for other possible names. For example,

(assert-error 'arithmetic-error (foo 0))

would assert that foo is supposed to signal an arithmetic error when passed zero.

Utility predicates

Several predicate functions are exported that are often useful in writing tests with assert-equality.

(logically-equal value1 value2)

This predicate returns true of the two values are either both true, i.e., non-NIL, or both false.

(set-equal list1 list2 [:test])

This predicate returns true the first list is a subset of the second and vice versa. :test can be used to specify an equality predicate. The default is eql.