Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 783 lines (617 sloc) 25.689 kb
169290b @richhickey made copyright notices uniform
richhickey authored
1 ; Copyright (c) Rich Hickey. All rights reserved.
2 ; The use and distribution terms for this software are covered by the
3 ; Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
4 ; which can be found in the file epl-v10.html at the root of this distribution.
5 ; By using this software in any fashion, you are agreeing to be bound by
6 ; the terms of this license.
7 ; You must not remove this notice, or any other, from this software.
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
8
169290b @richhickey made copyright notices uniform
richhickey authored
9 ;;; test.clj: test framework for Clojure
10
11 ;; by Stuart Sierra
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
12 ;; March 28, 2009
13
14 ;; Thanks to Chas Emerick, Allen Rohner, and Stuart Halloway for
15 ;; contributions and suggestions.
16
17 (ns
7879383 @richhickey replace #^s with ^s
richhickey authored
18 ^{:author "Stuart Sierra, with contributions and suggestions by
76e7c43 @stuartsierra Add :doc and :author metadata to core namespaces; fixes #216, fixes #217
stuartsierra authored
19 Chas Emerick, Allen Rohner, and Stuart Halloway",
20 :doc "A unit testing framework.
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
21
22 ASSERTIONS
23
24 The core of the library is the \"is\" macro, which lets you make
25 assertions of any arbitrary expression:
26
27 (is (= 4 (+ 2 2)))
28 (is (instance? Integer 256))
29 (is (.startsWith \"abcde\" \"ab\"))
30
31 You can type an \"is\" expression directly at the REPL, which will
32 print a message if it fails.
33
34 user> (is (= 5 (+ 2 2)))
35
36 FAIL in (:1)
37 expected: (= 5 (+ 2 2))
38 actual: (not (= 5 4))
39 false
40
41 The \"expected:\" line shows you the original expression, and the
42 \"actual:\" shows you what actually happened. In this case, it
43 shows that (+ 2 2) returned 4, which is not = to 5. Finally, the
44 \"false\" on the last line is the value returned from the
45 expression. The \"is\" macro always returns the result of the
46 inner expression.
47
48 There are two special assertions for testing exceptions. The
49 \"(is (thrown? c ...))\" form tests if an exception of class c is
50 thrown:
51
52 (is (thrown? ArithmeticException (/ 1 0)))
53
54 \"(is (thrown-with-msg? c re ...))\" does the same thing and also
55 tests that the message on the exception matches the regular
56 expression re:
57
58 (is (thrown-with-msg? ArithmeticException #\"Divide by zero\"
59 (/ 1 0)))
60
61 DOCUMENTING TESTS
62
63 \"is\" takes an optional second argument, a string describing the
64 assertion. This message will be included in the error report.
65
66 (is (= 5 (+ 2 2)) \"Crazy arithmetic\")
67
68 In addition, you can document groups of assertions with the
69 \"testing\" macro, which takes a string followed by any number of
70 assertions. The string will be included in failure reports.
71 Calls to \"testing\" may be nested, and all of the strings will be
72 joined together with spaces in the final report, in a style
73 similar to RSpec <http://rspec.info/>
74
75 (testing \"Arithmetic\"
76 (testing \"with positive integers\"
77 (is (= 4 (+ 2 2)))
78 (is (= 7 (+ 3 4))))
79 (testing \"with negative integers\"
80 (is (= -4 (+ -2 -2)))
81 (is (= -1 (+ 3 -4)))))
82
83 Note that, unlike RSpec, the \"testing\" macro may only be used
84 INSIDE a \"deftest\" or \"with-test\" form (see below).
85
86
87 DEFINING TESTS
88
89 There are two ways to define tests. The \"with-test\" macro takes
90 a defn or def form as its first argument, followed by any number
91 of assertions. The tests will be stored as metadata on the
92 definition.
93
94 (with-test
95 (defn my-function [x y]
96 (+ x y))
97 (is (= 4 (my-function 2 2)))
98 (is (= 7 (my-function 3 4))))
99
100 As of Clojure SVN rev. 1221, this does not work with defmacro.
101 See http://code.google.com/p/clojure/issues/detail?id=51
102
103 The other way lets you define tests separately from the rest of
104 your code, even in a different namespace:
105
106 (deftest addition
107 (is (= 4 (+ 2 2)))
108 (is (= 7 (+ 3 4))))
109
110 (deftest subtraction
111 (is (= 1 (- 4 3)))
112 (is (= 3 (- 7 4))))
113
114 This creates functions named \"addition\" and \"subtraction\", which
115 can be called like any other function. Therefore, tests can be
116 grouped and composed, in a style similar to the test framework in
117 Peter Seibel's \"Practical Common Lisp\"
118 <http://www.gigamonkeys.com/book/practical-building-a-unit-test-framework.html>
119
120 (deftest arithmetic
121 (addition)
122 (subtraction))
123
124 The names of the nested tests will be joined in a list, like
125 \"(arithmetic addition)\", in failure reports. You can use nested
126 tests to set up a context shared by several tests.
127
128
129 RUNNING TESTS
130
131 Run tests with the function \"(run-tests namespaces...)\":
132
133 (run-tests 'your.namespace 'some.other.namespace)
134
135 If you don't specify any namespaces, the current namespace is
136 used. To run all tests in all namespaces, use \"(run-all-tests)\".
137
138 By default, these functions will search for all tests defined in
139 a namespace and run them in an undefined order. However, if you
140 are composing tests, as in the \"arithmetic\" example above, you
141 probably do not want the \"addition\" and \"subtraction\" tests run
142 separately. In that case, you must define a special function
143 named \"test-ns-hook\" that runs your tests in the correct order:
144
145 (defn test-ns-hook []
146 (arithmetic))
147
6b63563 @stuartsierra Document mutual exclusion of fixtures and test-ns-hook; refs #235
stuartsierra authored
148 Note: test-ns-hook prevents execution of fixtures (see below).
149
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
150
151 OMITTING TESTS FROM PRODUCTION CODE
152
153 You can bind the variable \"*load-tests*\" to false when loading or
154 compiling code in production. This will prevent any tests from
155 being created by \"with-test\" or \"deftest\".
156
157
6b63563 @stuartsierra Document mutual exclusion of fixtures and test-ns-hook; refs #235
stuartsierra authored
158 FIXTURES
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
159
160 Fixtures allow you to run code before and after tests, to set up
161 the context in which tests should be run.
162
163 A fixture is just a function that calls another function passed as
164 an argument. It looks like this:
165
166 (defn my-fixture [f]
167 Perform setup, establish bindings, whatever.
168 (f) Then call the function we were passed.
169 Tear-down / clean-up code here.
170 )
171
172 Fixtures are attached to namespaces in one of two ways. \"each\"
173 fixtures are run repeatedly, once for each test function created
174 with \"deftest\" or \"with-test\". \"each\" fixtures are useful for
175 establishing a consistent before/after state for each test, like
176 clearing out database tables.
177
178 \"each\" fixtures can be attached to the current namespace like this:
179 (use-fixtures :each fixture1 fixture2 ...)
180 The fixture1, fixture2 are just functions like the example above.
181 They can also be anonymous functions, like this:
182 (use-fixtures :each (fn [f] setup... (f) cleanup...))
183
184 The other kind of fixture, a \"once\" fixture, is only run once,
185 around ALL the tests in the namespace. \"once\" fixtures are useful
186 for tasks that only need to be performed once, like establishing
187 database connections, or for time-consuming tasks.
188
189 Attach \"once\" fixtures to the current namespace like this:
190 (use-fixtures :once fixture1 fixture2 ...)
191
6b63563 @stuartsierra Document mutual exclusion of fixtures and test-ns-hook; refs #235
stuartsierra authored
192 Note: Fixtures and test-ns-hook are mutually incompatible. If you
193 are using test-ns-hook, fixture functions will *never* be run.
194
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
195
196 SAVING TEST OUTPUT TO A FILE
197
198 All the test reporting functions write to the var *test-out*. By
199 default, this is the same as *out*, but you can rebind it to any
200 PrintWriter. For example, it could be a file opened with
fbe0183 @stuarthalloway update references to contrib to point to new names in clojure
stuarthalloway authored
201 clojure.java.io/writer.
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
202
203
204 EXTENDING TEST-IS (ADVANCED)
205
206 You can extend the behavior of the \"is\" macro by defining new
207 methods for the \"assert-expr\" multimethod. These methods are
208 called during expansion of the \"is\" macro, so they should return
209 quoted forms to be evaluated.
210
211 You can plug in your own test-reporting framework by rebinding
212 the \"report\" function: (report event)
213
214 The 'event' argument is a map. It will always have a :type key,
215 whose value will be a keyword signaling the type of event being
216 reported. Standard events with :type value of :pass, :fail, and
217 :error are called when an assertion passes, fails, and throws an
218 exception, respectively. In that case, the event will also have
219 the following keys:
220
221 :expected The form that was expected to be true
222 :actual A form representing what actually occurred
223 :message The string message given as an argument to 'is'
224
225 The \"testing\" strings will be a list in \"*testing-contexts*\", and
226 the vars being tested will be a list in \"*testing-vars*\".
227
228 Your \"report\" function should wrap any printing calls in the
229 \"with-test-out\" macro, which rebinds *out* to the current value
230 of *test-out*.
231
232 For additional event types, see the examples in the code.
233 "}
a12092c @stuarthalloway gtic package renamings:
stuarthalloway authored
234 clojure.test
235 (:require [clojure.template :as temp]
236 [clojure.stacktrace :as stack]))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
237
238 ;; Nothing is marked "private" here, so you can rebind things to plug
239 ;; in your own testing or reporting frameworks.
240
241
242 ;;; USER-MODIFIABLE GLOBALS
243
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
244 (defonce ^:dynamic
7879383 @richhickey replace #^s with ^s
richhickey authored
245 ^{:doc "True by default. If set to false, no test functions will
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
246 be created by deftest, set-test, or with-test. Use this to omit
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
247 tests when compiling or loading production code."
248 :added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
249 *load-tests* true)
250
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
251 (def ^:dynamic
7879383 @richhickey replace #^s with ^s
richhickey authored
252 ^{:doc "The maximum depth of stack traces to print when an Exception
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
253 is thrown during a test. Defaults to nil, which means print the
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
254 complete stack trace."
255 :added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
256 *stack-trace-depth* nil)
257
258
259 ;;; GLOBALS USED BY THE REPORTING FUNCTIONS
260
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
261 (def ^:dynamic *report-counters* nil) ; bound to a ref of a map in test-ns
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
262
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
263 (def ^:dynamic *initial-report-counters* ; used to initialize *report-counters*
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
264 {:test 0, :pass 0, :fail 0, :error 0})
265
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
266 (def ^:dynamic *testing-vars* (list)) ; bound to hierarchy of vars being tested
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
267
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
268 (def ^:dynamic *testing-contexts* (list)) ; bound to hierarchy of "testing" strings
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
269
b9b1a09 @richhickey require dynamically rebindable vars be explicitly declared dynamic, via ...
richhickey authored
270 (def ^:dynamic *test-out* *out*) ; PrintWriter for test reporting output
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
271
272 (defmacro with-test-out
273 "Runs body with *out* bound to the value of *test-out*."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
274 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
275 [& body]
276 `(binding [*out* *test-out*]
277 ~@body))
278
279 ;;; UTILITIES FOR REPORTING FUNCTIONS
280
281 (defn file-position
282 "Returns a vector [filename line-number] for the nth call up the
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
283 stack.
284
285 Deprecated in 1.2: The information needed for test reporting is
286 now on :file and :line keys in the result map."
287 {:added "1.1"
288 :deprecated "1.2"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
289 [n]
7879383 @richhickey replace #^s with ^s
richhickey authored
290 (let [^StackTraceElement s (nth (.getStackTrace (new java.lang.Throwable)) n)]
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
291 [(.getFileName s) (.getLineNumber s)]))
292
293 (defn testing-vars-str
294 "Returns a string representation of the current test. Renders names
295 in *testing-vars* as a list, then the source file and line of
296 current assertion."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
297 {:added "1.1"}
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
298 [m]
299 (let [{:keys [file line]} m]
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
300 (str
301 ;; Uncomment to include namespace in failure report:
302 ;;(ns-name (:ns (meta (first *testing-vars*)))) "/ "
303 (reverse (map #(:name (meta %)) *testing-vars*))
304 " (" file ":" line ")")))
305
306 (defn testing-contexts-str
307 "Returns a string representation of the current test context. Joins
308 strings in *testing-contexts* with spaces."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
309 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
310 []
311 (apply str (interpose " " (reverse *testing-contexts*))))
312
313 (defn inc-report-counter
314 "Increments the named counter in *report-counters*, a ref to a map.
315 Does nothing if *report-counters* is nil."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
316 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
317 [name]
318 (when *report-counters*
319 (dosync (commute *report-counters* assoc name
320 (inc (or (*report-counters* name) 0))))))
321
322 ;;; TEST RESULT REPORTING
323
324 (defmulti
7879383 @richhickey replace #^s with ^s
richhickey authored
325 ^{:doc "Generic reporting function, may be overridden to plug in
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
326 different report formats (e.g., TAP, JUnit). Assertions such as
327 'is' call 'report' to indicate results. The argument given to
328 'report' will be a map with a :type key. See the documentation at
329 the top of test_is.clj for more information on the types of
4d08439 @richhickey direct linking of var calls, inlining of self calls
richhickey authored
330 arguments for 'report'."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
331 :dynamic true
332 :added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
333 report :type)
334
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
335 (defn- file-and-line
c32511a @jafingerhut CLJ-1102: Improve handling of empty stack traces returned by .getStackTr...
jafingerhut authored
336 [^Throwable exception depth]
337 (let [stacktrace (.getStackTrace exception)]
338 (if (< depth (count stacktrace))
339 (let [^StackTraceElement s (nth stacktrace depth)]
340 {:file (.getFileName s) :line (.getLineNumber s)})
341 {:file nil :line nil})))
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
342
343 (defn do-report
344 "Add file and line information to a test result and call report.
345 If you are writing a custom assert-expr method, call this function
346 to pass test results to report."
347 {:added "1.2"}
348 [m]
349 (report
350 (case
351 (:type m)
352 :fail (merge (file-and-line (new java.lang.Throwable) 1) m)
353 :error (merge (file-and-line (:actual m) 0) m)
354 m)))
355
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
356 (defmethod report :default [m]
357 (with-test-out (prn m)))
358
359 (defmethod report :pass [m]
360 (with-test-out (inc-report-counter :pass)))
361
362 (defmethod report :fail [m]
363 (with-test-out
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
364 (inc-report-counter :fail)
365 (println "\nFAIL in" (testing-vars-str m))
366 (when (seq *testing-contexts*) (println (testing-contexts-str)))
367 (when-let [message (:message m)] (println message))
368 (println "expected:" (pr-str (:expected m)))
369 (println " actual:" (pr-str (:actual m)))))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
370
371 (defmethod report :error [m]
372 (with-test-out
373 (inc-report-counter :error)
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
374 (println "\nERROR in" (testing-vars-str m))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
375 (when (seq *testing-contexts*) (println (testing-contexts-str)))
376 (when-let [message (:message m)] (println message))
377 (println "expected:" (pr-str (:expected m)))
378 (print " actual: ")
379 (let [actual (:actual m)]
380 (if (instance? Throwable actual)
381 (stack/print-cause-trace actual *stack-trace-depth*)
382 (prn actual)))))
383
384 (defmethod report :summary [m]
385 (with-test-out
386 (println "\nRan" (:test m) "tests containing"
387 (+ (:pass m) (:fail m) (:error m)) "assertions.")
388 (println (:fail m) "failures," (:error m) "errors.")))
389
390 (defmethod report :begin-test-ns [m]
391 (with-test-out
392 (println "\nTesting" (ns-name (:ns m)))))
393
394 ;; Ignore these message types:
395 (defmethod report :end-test-ns [m])
396 (defmethod report :begin-test-var [m])
397 (defmethod report :end-test-var [m])
398
399
400
401 ;;; UTILITIES FOR ASSERTIONS
402
403 (defn get-possibly-unbound-var
404 "Like var-get but returns nil if the var is unbound."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
405 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
406 [v]
407 (try (var-get v)
408 (catch IllegalStateException e
409 nil)))
410
411 (defn function?
412 "Returns true if argument is a function or a symbol that resolves to
413 a function (not a macro)."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
414 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
415 [x]
416 (if (symbol? x)
417 (when-let [v (resolve x)]
418 (when-let [value (get-possibly-unbound-var v)]
419 (and (fn? value)
420 (not (:macro (meta v))))))
421 (fn? x)))
422
423 (defn assert-predicate
424 "Returns generic assertion code for any functional predicate. The
425 'expected' argument to 'report' will contains the original form, the
426 'actual' argument will contain the form with all its sub-forms
427 evaluated. If the predicate returns false, the 'actual' form will
428 be wrapped in (not...)."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
429 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
430 [msg form]
431 (let [args (rest form)
432 pred (first form)]
433 `(let [values# (list ~@args)
434 result# (apply ~pred values#)]
435 (if result#
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
436 (do-report {:type :pass, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
437 :expected '~form, :actual (cons ~pred values#)})
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
438 (do-report {:type :fail, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
439 :expected '~form, :actual (list '~'not (cons '~pred values#))}))
440 result#)))
441
442 (defn assert-any
443 "Returns generic assertion code for any test, including macros, Java
444 method calls, or isolated symbols."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
445 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
446 [msg form]
447 `(let [value# ~form]
448 (if value#
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
449 (do-report {:type :pass, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
450 :expected '~form, :actual value#})
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
451 (do-report {:type :fail, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
452 :expected '~form, :actual value#}))
453 value#))
454
455
456
457 ;;; ASSERTION METHODS
458
459 ;; You don't call these, but you can add methods to extend the 'is'
460 ;; macro. These define different kinds of tests, based on the first
461 ;; symbol in the test expression.
462
463 (defmulti assert-expr
464 (fn [msg form]
465 (cond
466 (nil? form) :always-fail
467 (seq? form) (first form)
468 :else :default)))
469
470 (defmethod assert-expr :always-fail [msg form]
471 ;; nil test: always fail
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
472 `(do-report {:type :fail, :message ~msg}))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
473
474 (defmethod assert-expr :default [msg form]
475 (if (and (sequential? form) (function? (first form)))
476 (assert-predicate msg form)
477 (assert-any msg form)))
478
479 (defmethod assert-expr 'instance? [msg form]
480 ;; Test if x is an instance of y.
481 `(let [klass# ~(nth form 1)
482 object# ~(nth form 2)]
483 (let [result# (instance? klass# object#)]
484 (if result#
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
485 (do-report {:type :pass, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
486 :expected '~form, :actual (class object#)})
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
487 (do-report {:type :fail, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
488 :expected '~form, :actual (class object#)}))
489 result#)))
490
491 (defmethod assert-expr 'thrown? [msg form]
492 ;; (is (thrown? c expr))
493 ;; Asserts that evaluating expr throws an exception of class c.
494 ;; Returns the exception thrown.
495 (let [klass (second form)
496 body (nthnext form 2)]
497 `(try ~@body
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
498 (do-report {:type :fail, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
499 :expected '~form, :actual nil})
500 (catch ~klass e#
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
501 (do-report {:type :pass, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
502 :expected '~form, :actual e#})
503 e#))))
504
505 (defmethod assert-expr 'thrown-with-msg? [msg form]
506 ;; (is (thrown-with-msg? c re expr))
507 ;; Asserts that evaluating expr throws an exception of class c.
508 ;; Also asserts that the message string of the exception matches
634b3d5 @stuartsierra Use re-find (not re-matches) in thrown-with-msg?; fixes #204
stuartsierra authored
509 ;; (with re-find) the regular expression re.
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
510 (let [klass (nth form 1)
511 re (nth form 2)
512 body (nthnext form 3)]
513 `(try ~@body
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
514 (do-report {:type :fail, :message ~msg, :expected '~form, :actual nil})
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
515 (catch ~klass e#
516 (let [m# (.getMessage e#)]
634b3d5 @stuartsierra Use re-find (not re-matches) in thrown-with-msg?; fixes #204
stuartsierra authored
517 (if (re-find ~re m#)
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
518 (do-report {:type :pass, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
519 :expected '~form, :actual e#})
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
520 (do-report {:type :fail, :message ~msg,
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
521 :expected '~form, :actual e#})))
522 e#))))
523
524
525 (defmacro try-expr
526 "Used by the 'is' macro to catch unexpected exceptions.
527 You don't call this."
59b6566 @stuarthalloway added 1.0, test all public doc-ed fns have :added
stuarthalloway authored
528 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
529 [msg form]
530 `(try ~(assert-expr msg form)
531 (catch Throwable t#
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
532 (do-report {:type :error, :message ~msg,
533 :expected '~form, :actual t#}))))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
534
535
536
537 ;;; ASSERTION MACROS
538
539 ;; You use these in your tests.
540
541 (defmacro is
542 "Generic assertion macro. 'form' is any predicate test.
543 'msg' is an optional message to attach to the assertion.
544
545 Example: (is (= 4 (+ 2 2)) \"Two plus two should be 4\")
546
547 Special forms:
548
549 (is (thrown? c body)) checks that an instance of c is thrown from
550 body, fails if not; then returns the thing thrown.
551
552 (is (thrown-with-msg? c re body)) checks that an instance of c is
553 thrown AND that the message on the exception matches (with
634b3d5 @stuartsierra Use re-find (not re-matches) in thrown-with-msg?; fixes #204
stuartsierra authored
554 re-find) the regular expression re."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
555 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
556 ([form] `(is ~form nil))
557 ([form msg] `(try-expr ~msg ~form)))
558
559 (defmacro are
560 "Checks multiple assertions with a template expression.
a12092c @stuarthalloway gtic package renamings:
stuarthalloway authored
561 See clojure.template/do-template for an explanation of
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
562 templates.
563
564 Example: (are [x y] (= x y)
565 2 (+ 1 1)
566 4 (* 2 2))
567 Expands to:
568 (do (is (= 2 (+ 1 1)))
569 (is (= 4 (* 2 2))))
570
571 Note: This breaks some reporting features, such as line numbers."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
572 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
573 [argv expr & args]
fa927fd @tsdh Fix CLJ-931: Syntactically broken clojure.test/are tests succeed.
tsdh authored
574 (if (or
575 ;; (are [] true) is meaningless but ok
576 (and (empty? argv) (empty? args))
577 ;; Catch wrong number of args
578 (and (pos? (count argv))
579 (pos? (count args))
580 (zero? (mod (count args) (count argv)))))
581 `(temp/do-template ~argv (is ~expr) ~@args)
582 (throw (IllegalArgumentException. "The number of args doesn't match are's argv."))))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
583
584 (defmacro testing
585 "Adds a new string to the list of testing contexts. May be nested,
586 but must occur inside a test function (deftest)."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
587 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
588 [string & body]
589 `(binding [*testing-contexts* (conj *testing-contexts* ~string)]
590 ~@body))
591
592
593
594 ;;; DEFINING TESTS
595
596 (defmacro with-test
597 "Takes any definition form (that returns a Var) as the first argument.
598 Remaining body goes in the :test metadata function for that Var.
599
600 When *load-tests* is false, only evaluates the definition, ignoring
601 the tests."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
602 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
603 [definition & body]
604 (if *load-tests*
605 `(doto ~definition (alter-meta! assoc :test (fn [] ~@body)))
606 definition))
607
608
609 (defmacro deftest
610 "Defines a test function with no arguments. Test functions may call
611 other tests, so tests may be composed. If you compose tests, you
612 should also define a function named test-ns-hook; run-tests will
613 call test-ns-hook instead of testing all vars.
614
615 Note: Actually, the test body goes in the :test metadata on the var,
616 and the real function (the value of the var) calls test-var on
617 itself.
618
619 When *load-tests* is false, deftest is ignored."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
620 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
621 [name & body]
622 (when *load-tests*
7779c99 @technomancy Preserve test var metadata within deftest. Fixes #201.
technomancy authored
623 `(def ~(vary-meta name assoc :test `(fn [] ~@body))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
624 (fn [] (test-var (var ~name))))))
625
626 (defmacro deftest-
627 "Like deftest but creates a private var."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
628 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
629 [name & body]
630 (when *load-tests*
7779c99 @technomancy Preserve test var metadata within deftest. Fixes #201.
technomancy authored
631 `(def ~(vary-meta name assoc :test `(fn [] ~@body) :private true)
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
632 (fn [] (test-var (var ~name))))))
633
634
635 (defmacro set-test
636 "Experimental.
637 Sets :test metadata of the named var to a fn with the given body.
638 The var must already exist. Does not modify the value of the var.
639
640 When *load-tests* is false, set-test is ignored."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
641 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
642 [name & body]
643 (when *load-tests*
644 `(alter-meta! (var ~name) assoc :test (fn [] ~@body))))
645
646
647
648 ;;; DEFINING FIXTURES
649
650 (defn- add-ns-meta
651 "Adds elements in coll to the current namespace metadata as the
652 value of key."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
653 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
654 [key coll]
f5f2943 @technomancy Don't repeatedly compose on calls to use-fixtures. Fixes #194.
technomancy authored
655 (alter-meta! *ns* assoc key coll))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
656
f5f2943 @technomancy Don't repeatedly compose on calls to use-fixtures. Fixes #194.
technomancy authored
657 (defmulti use-fixtures
658 "Wrap test runs in a fixture function to perform setup and
659 teardown. Using a fixture-type of :each wraps every test
660 individually, while:once wraps the whole run in a single function."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
661 {:added "1.1"}
f5f2943 @technomancy Don't repeatedly compose on calls to use-fixtures. Fixes #194.
technomancy authored
662 (fn [fixture-type & args] fixture-type))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
663
664 (defmethod use-fixtures :each [fixture-type & args]
665 (add-ns-meta ::each-fixtures args))
666
667 (defmethod use-fixtures :once [fixture-type & args]
668 (add-ns-meta ::once-fixtures args))
669
670 (defn- default-fixture
671 "The default, empty, fixture function. Just calls its argument."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
672 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
673 [f]
674 (f))
675
676 (defn compose-fixtures
677 "Composes two fixture functions, creating a new fixture function
678 that combines their behavior."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
679 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
680 [f1 f2]
681 (fn [g] (f1 (fn [] (f2 g)))))
682
683 (defn join-fixtures
684 "Composes a collection of fixtures, in order. Always returns a valid
685 fixture function, even if the collection is empty."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
686 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
687 [fixtures]
688 (reduce compose-fixtures default-fixture fixtures))
689
690
691
692
693 ;;; RUNNING TESTS: LOW-LEVEL FUNCTIONS
694
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
695 (defn test-var
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
696 "If v has a function in its :test metadata, calls that function,
697 with *testing-vars* bound to (conj *testing-vars* v)."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
698 {:dynamic true, :added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
699 [v]
700 (when-let [t (:test (meta v))]
701 (binding [*testing-vars* (conj *testing-vars* v)]
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
702 (do-report {:type :begin-test-var, :var v})
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
703 (inc-report-counter :test)
704 (try (t)
705 (catch Throwable e
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
706 (do-report {:type :error, :message "Uncaught exception, not in assertion."
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
707 :expected nil, :actual e})))
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
708 (do-report {:type :end-test-var, :var v}))))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
709
fa5483a @Raynes CLJ-866: Provide a function for running specific tests with fixtures.
Raynes authored
710 (defn test-vars
711 "Groups vars by their namespace and runs test-vars on them with
712 appropriate fixtures applied."
919a710 @tobias CLJ-1352: Don't run :each fixtures for vars with no :test metadata
tobias authored
713 {:added "1.6"}
fa5483a @Raynes CLJ-866: Provide a function for running specific tests with fixtures.
Raynes authored
714 [vars]
715 (doseq [[ns vars] (group-by (comp :ns meta) vars)]
716 (let [once-fixture-fn (join-fixtures (::once-fixtures (meta ns)))
717 each-fixture-fn (join-fixtures (::each-fixtures (meta ns)))]
718 (once-fixture-fn
719 (fn []
720 (doseq [v vars]
919a710 @tobias CLJ-1352: Don't run :each fixtures for vars with no :test metadata
tobias authored
721 (when (:test (meta v))
722 (each-fixture-fn (fn [] (test-var v))))))))))
fa5483a @Raynes CLJ-866: Provide a function for running specific tests with fixtures.
Raynes authored
723
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
724 (defn test-all-vars
fa5483a @Raynes CLJ-866: Provide a function for running specific tests with fixtures.
Raynes authored
725 "Calls test-vars on every var interned in the namespace, with fixtures."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
726 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
727 [ns]
fa5483a @Raynes CLJ-866: Provide a function for running specific tests with fixtures.
Raynes authored
728 (test-vars (vals (ns-interns ns))))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
729
730 (defn test-ns
731 "If the namespace defines a function named test-ns-hook, calls that.
732 Otherwise, calls test-all-vars on the namespace. 'ns' is a
733 namespace object or a symbol.
734
735 Internally binds *report-counters* to a ref initialized to
5fcdc6e @cldwalker fix typos in docstrings and docs
cldwalker authored
736 *initial-report-counters*. Returns the final, dereferenced state of
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
737 *report-counters*."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
738 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
739 [ns]
740 (binding [*report-counters* (ref *initial-report-counters*)]
741 (let [ns-obj (the-ns ns)]
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
742 (do-report {:type :begin-test-ns, :ns ns-obj})
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
743 ;; If the namespace has a test-ns-hook function, call that:
744 (if-let [v (find-var (symbol (str (ns-name ns-obj)) "test-ns-hook"))]
745 ((var-get v))
746 ;; Otherwise, just test every var in the namespace.
747 (test-all-vars ns-obj))
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
748 (do-report {:type :end-test-ns, :ns ns-obj}))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
749 @*report-counters*))
750
751
752
753 ;;; RUNNING TESTS: HIGH-LEVEL FUNCTIONS
754
755 (defn run-tests
756 "Runs all tests in the given namespaces; prints results.
6073890 @stuartsierra clojure.test/run-tests: return summary, add successful?; refs #193
stuartsierra authored
757 Defaults to current namespace if none given. Returns a map
758 summarizing test results."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
759 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
760 ([] (run-tests *ns*))
761 ([& namespaces]
6073890 @stuartsierra clojure.test/run-tests: return summary, add successful?; refs #193
stuartsierra authored
762 (let [summary (assoc (apply merge-with + (map test-ns namespaces))
763 :type :summary)]
d4239d0 @stuarthalloway #377 test now reports file/line for failures in repl or Ant build
stuarthalloway authored
764 (do-report summary)
6073890 @stuartsierra clojure.test/run-tests: return summary, add successful?; refs #193
stuartsierra authored
765 summary)))
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
766
767 (defn run-all-tests
768 "Runs all tests in all namespaces; prints results.
769 Optional argument is a regular expression; only namespaces with
770 names matching the regular expression (with re-matches) will be
771 tested."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
772 {:added "1.1"}
d5f4c4c @stuarthalloway gtic work-in-progress:
stuarthalloway authored
773 ([] (apply run-tests (all-ns)))
774 ([re] (apply run-tests (filter #(re-matches re (name (ns-name %))) (all-ns)))))
6073890 @stuartsierra clojure.test/run-tests: return summary, add successful?; refs #193
stuartsierra authored
775
776 (defn successful?
777 "Returns true if the given test summary indicates all tests
778 were successful, false otherwise."
c1c3916 @stuarthalloway metadata for :added
stuarthalloway authored
779 {:added "1.1"}
6073890 @stuartsierra clojure.test/run-tests: return summary, add successful?; refs #193
stuartsierra authored
780 [summary]
52e630f @richhickey fix NPE if summary missing :error or :fail keys
richhickey authored
781 (and (zero? (:fail summary 0))
782 (zero? (:error summary 0))))
Something went wrong with that request. Please try again.