Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 280 lines (208 sloc) 12.151 kB
dbbc73c @pholser working on README
authored
1 # junit-quickcheck: QuickCheck-style parameter suppliers for JUnit theories
2
8286f5f @pholser README 80 columns
authored
3 junit-quickcheck is a library that supplies [JUnit](http://junit.org)
6766ff9 @pholser README changes
authored
4 [theories](http://groups.csail.mit.edu/pag/pubs/test-theory-demo-oopsla2007.pdf) with random values with which to test
5 the validity of the theories.
dbbc73c @pholser working on README
authored
6
7 ### Background
8
6766ff9 @pholser README changes
authored
9 The [Haskell](http://haskell.org) library [QuickCheck](http://www.cse.chalmers.se/~rjmh/QuickCheck/manual.html)
10 allows programmers to specify properties of a function that should hold true for some large set of possible arguments
11 to the function, then executes the function using lots of random arguments to see whether the property holds up.
dbbc73c @pholser working on README
authored
12
e6fbace @pholser more README work
authored
13 #### Theories
14
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
15 JUnit's answer to function properties is the notion of _theories_. Programmers write parameterized tests marked as
6766ff9 @pholser README changes
authored
16 theories, run using a special test runner:
dbbc73c @pholser working on README
authored
17
18 @RunWith(Theories.class)
19 public class Accounts {
20 @Theory
21 public void withdrawingReducesBalance(Money originalBalance, Money withdrawalAmount) {
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
22 assumeThat(originalBalance, greaterThan(0));
23 assumeThat(withdrawalAmount, allOf(greaterThan(0), lessThan(originalBalance));
24
dbbc73c @pholser working on README
authored
25 Account account = new Account(originalBalance);
26
27 account.withdraw(withdrawalAmount);
28
29 assertEquals(originalBalance.minus(withdrawalAmount), account.balance());
30 }
31 }
32
e6fbace @pholser more README work
authored
33 #### So?
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
34
6766ff9 @pholser README changes
authored
35 TDD/BDD builds up designs example by example. The resulting test suites give programmers confidence that their code
36 works for the examples they thought of. Theories offer a means to express statements about code that should hold for
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
37 an entire domain of inputs, not just a handful of examples, and to validate those statements against lots of randomly
6766ff9 @pholser README changes
authored
38 generated inputs.
e6fbace @pholser more README work
authored
39
40 ### Using junit-quickcheck
41
6766ff9 @pholser README changes
authored
42 Create theories as you normally would with JUnit. If you want JUnit to exercise the theory with lots of randomly
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
43 generated values for a theory parameter, mark the theory parameter with `@ForAll`:
e6fbace @pholser more README work
authored
44
45 @RunWith(Theories.class)
46 public class SymmetricKeyCryptography {
47 @Theory
901c94b @pholser Slight README amendment for longer example code lines.
authored
48 public void decryptReversesEncrypt(@ForAll String plaintext, @ForAll Key key) throws Exception {
e6fbace @pholser more README work
authored
49 Crypto crypto = new Crypto();
9186fac @pholser more s/extractor/generator/
authored
50
901c94b @pholser Slight README amendment for longer example code lines.
authored
51 byte[] ciphertext = crypto.encrypt(plaintext.getBytes("US-ASCII", key));
9186fac @pholser more s/extractor/generator/
authored
52
901c94b @pholser Slight README amendment for longer example code lines.
authored
53 assertEquals(plaintext, new String(crypto.decrypt(ciphertext, key)));
e6fbace @pholser more README work
authored
54 }
55 }
56
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
57 #### Sample size
58
6766ff9 @pholser README changes
authored
59 By default, 100 random values will be generated for a parameter marked `@ForAll`. Use the `sampleSize` attribute of
60 `@ForAll` to change the number of generated values.
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
61
62 #### Supported types
e6fbace @pholser more README work
authored
63
6766ff9 @pholser README changes
authored
64 Out of the box, junit-quickcheck can generate random values for theory parameters of the following types:
e6fbace @pholser more README work
authored
65
66 * all Java primitives and primitive wrappers
67 * `java.math.Big(Decimal|Integer)`
1415aab @pholser adding java.util.Date as an OOTB supported type in the README
authored
68 * `java.util.Date`
936fae6 @pholser update README to include enum, and mention that sample size is disreg…
authored
69 * any `enum`
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
70 * `java.util.ArrayList` of those types
71 * `java.util.HashSet` of those types
72 * `java.util.HashMap` of those types
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
73 * arrays of those types
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
74
6766ff9 @pholser README changes
authored
75 When multiple generators can satisfy a given theory parameter based on its type (for example, `java.io.Serializable`),
76 on a given generation one of the multiple generators will be chosen at random with equal probability.
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
77
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
78 #### Generating values of other types
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
79
6766ff9 @pholser README changes
authored
80 If you wish to generate random values for theory parameters of other types, or to override the default means of
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
81 generation for a supported type, mark the theory parameter already marked as `@ForAll` with `@From` and supply the
82 class(es) of the `Generator` to be used. If you give multiple classes in `@From`, one will be chosen on every
83 generation with equal probability.
84
85 If you wish to add a generator for a type without having to use `@From`, you can package your `Generator` in a
86 [ServiceLoader](http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html) JAR file and place the JAR on
87 the class path. junit-quickcheck will make those generators available for use. Any generators you supply in this manner
88 for already-supported types complement the built-in generators, they do not override them.
89
90 #### Configuring generators
91
92 Over the period of generating values for a single theory parameter, you can feed specific configurations to the
93 generator(s) for values of the parameter's type. If you mark a theory parameter already marked as `@ForAll` with an
94 annotation that is itself marked as `@GeneratorConfiguration`, then if the `Generator` for that parameter's type has a
95 public method named `configure` that accepts a single parameter of the annotation type, junit-quickcheck will call the
96 `configure` method reflectively, passing it the annotation:
97
7a1c104 @pholser more README
authored
98 @Target(PARAMETER)
99 @Retention(RUNTIME)
f89a81f @pholser some repackaging, javadoc
authored
100 @GeneratorConfiguration
101 public @interface Stuff {
102 // ...
103 }
104
105 public class FooGenerator extends Generator<Foo> {
106 // ...
107
108 public void configure(@Stuff stuff) {
109 // ...
110 }
111 }
112
113 @RunWith(Theories.class)
114 public class FooTheories {
115 @Theory
116 public void holds(@ForAll @Stuff Foo f) {
117 // ...
118 }
119 }
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
120
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
121 A `Generator` can have many such `configure` methods.
e6fbace @pholser more README work
authored
122
f89a81f @pholser some repackaging, javadoc
authored
123 #### Constraining generated values
124
125 ##### Assumptions
126
127 Theories often use _assumptions_ to declare conditions under which the theories hold:
128
129 @RunWith(Theories.class)
130 public class PrimeFactorsTheories {
131 @Theory
132 public void factorsPassPrimalityTest(@ForAll int n) {
133 assumeThat(n, greaterThan(0));
134
135 for (int each : PrimeFactors.of(n))
136 assertTrue(BigInteger.valueOf(each).isProbablePrime(1000));
137 }
138
139 @Theory
140 public void factorsMultiplyToOriginal(@ForAll int n) {
141 assumeThat(n, greaterThan(0));
142
143 int product = 1;
144 for (int each : PrimeFactors.of(n))
145 product *= each;
146
147 assertEquals(n, product);
148 }
149
150 @Theory
151 public void factorizationsAreUnique(@ForAll int m, @ForAll int n) {
152 assumeThat(m, greaterThan(0));
153 assumeThat(n, greaterThan(0));
154 assumeThat(m, not(equalTo(n)));
155
156 assertThat(PrimeFactors.of(m), not(equalTo(PrimeFactors.of(n))));
157 }
158 }
159
160 There are times when using assumptions with junit-quickcheck may yield too few values that meet the desired
161 criteria:
162
163 @RunWith(Theories.class)
164 public class SingleDigitTheories {
165 @Theory
166 public void hold(@ForAll int digit) {
167 // hope we get enough single digits
168 assumeThat(digit, greaterThanOrEqualTo(0));
169 assumeThat(digit, lessThanOrEqualTo(9));
170
171 // ...
172 }
173 }
174
175 Here, junit-quickcheck will generate 100 values, but there's not much guarantee that we'll get very many, if any,
176 values to test out the theory.
177
178 ##### Generator configuration methods
179
180 Generator configuration methods and annotations can serve to constrain the values that a generator emits. For example,
181 the `@InRange` annotation on theory parameters of `int` or narrower causes the generators for those types to emit
182 values that fall within a configured minimum/maximum:
183
184 @RunWith(Theories.class)
185 public class SingleDigitTheories {
186 @Theory
187 public void hold(@ForAll @InRange(min = "0", max = "9") int digit) {
188 // ...
189 }
190 }
191
192 Now, the generator will be configured to ensure that every value generated meets the desired criteria -- no need to
193 couch the desired range of values as an assumption.
194
195 When using assumptions with junit-quickcheck, every value fed to a `@ForAll` theory parameter counts against the
196 sample size, even if it doesn't pass any assumptions made against it in the theory. You could end up with no values
197 passing the assumption.
198
199 Using generator configurations, assumptions aren't very important, if needed at all -- every value fed to a `@ForAll`
200 theory parameter counts against the sample size, but will meet some expectations that assumptions would otherwise have
201 tested.
202
203 ##### Constraint expressions
204
205 Constraint expressions enable you to filter the values that reach a theory parameter. Mark a theory parameter already
206 marked as `@ForAll` with `@SuchThat`, supplying an [OGNL](http://commons.apache.org/ognl) expression that will be used
207 to decide whether a generated value will be given to the theory method.
208
209 @RunWith(Theories.class)
210 public class SingleDigitTheories {
211 @Theory
212 public void hold(@ForAll @SuchThat("#root > 0") int digit) {
213 // ...
214 }
215 }
216
7a1c104 @pholser more README
authored
217 `#root` refers to the theory parameter itself. Currently, constraint expressions cannot refer to other theory
218 parameters.
f89a81f @pholser some repackaging, javadoc
authored
219
7a1c104 @pholser more README
authored
220 junit-quickcheck generates values for a theory parameter with a constraint expression until `sampleSize` values pass
221 the constraint, or until the ratio of constraint passes to constraint failures is greater than the `discardRatio`
222 specified by `@ForAll`, if any. Exceeding the discard ratio is treated as an assumption violation.
f89a81f @pholser some repackaging, javadoc
authored
223
e6fbace @pholser more README work
authored
224 ### How it works
225
effea96 @pholser Updating README. @InRange-ing the short generator.
authored
226 junit-quickcheck leverages the
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
227 [`ParameterSupplier`](http://kentbeck.github.com/junit/javadoc/latest/org/junit/experimental/theories/ParameterSupplier.html)
228 feature of JUnit.
e6fbace @pholser more README work
authored
229
6766ff9 @pholser README changes
authored
230 By default, when the `Theories` runner executes a theory, it attempts to scrape _data points_ off the theory class to
231 feed to the theories. Data points come from static fields or methods annotated with `@DataPoint` (single value) or
232 `@DataPoints` (array of values). The `Theories` runner arranges for all combinations of data points of types matching a
233 theory parameter to be fed to the theory for execution.
e6fbace @pholser more README work
authored
234
8286f5f @pholser README 80 columns
authored
235 Marking a theory parameter with an annotation that is itself annotated with
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
236 [`@ParametersSuppliedBy`](http://kentbeck.github.com/junit/javadoc/latest/org/junit/experimental/theories/ParametersSuppliedBy.html)
6766ff9 @pholser README changes
authored
237 tells the `Theories` runner to ask a `ParameterSupplier` for values for the theory parameter instead. This is how
238 junit-quickcheck interacts with the `Theories` runner -- `@ForAll` tells the runner to use junit-quickcheck's
aeb15fa @pholser Removing ObjectExtractor. Adding to README.
authored
239 `ParameterSupplier` rather than the `DataPoint`-oriented one.
e6fbace @pholser more README work
authored
240
6766ff9 @pholser README changes
authored
241 In order for junit-quickcheck (or any `ParameterSupplier`, for that matter) to be able to leverage generics info on a
242 theory parameter (e.g. to distinguish between a parameter of type `List<String>` and a parameter of type
243 `List<Integer>`), [this issue](https://github.com/KentBeck/junit/issues#issue/64) will need to be resolved.
e6fbace @pholser more README work
authored
244
245 ### Similar projects
246
6766ff9 @pholser README changes
authored
247 * [JCheck](http://www.jcheck.org/). This uses its own test runner, whereas junit-quickcheck leverages the existing
248 `Theories` runner and `ParameterSupplier`s.
249 * [QuickCheck](http://java.net/projects/quickcheck/pages/Home). This appears to be test framework-agnostic, focusing
250 instead on generators of random values.
e6fbace @pholser more README work
authored
251 * [fj.test package of FunctionalJava (formerly Reductio)](http://functionaljava.org/)
6766ff9 @pholser README changes
authored
252 * [ScalaCheck](http://code.google.com/p/scalacheck/), if you wish to test Java or Scala code using Scala.
dbbc73c @pholser working on README
authored
253
254 ### About junit-quickcheck
255
6766ff9 @pholser README changes
authored
256 junit-quickcheck was written by Paul Holser, and is distributed under the MIT License.
dbbc73c @pholser working on README
authored
257
258 The MIT License
259
7db0d1a @pholser Resolve #6 -- restrict sample size to 2 for boolean theory parms
authored
260 Copyright (c) 2010-2012 Paul R. Holser, Jr.
dbbc73c @pholser working on README
authored
261
262 Permission is hereby granted, free of charge, to any person obtaining
263 a copy of this software and associated documentation files (the
264 "Software"), to deal in the Software without restriction, including
265 without limitation the rights to use, copy, modify, merge, publish,
266 distribute, sublicense, and/or sell copies of the Software, and to
267 permit persons to whom the Software is furnished to do so, subject to
268 the following conditions:
269
270 The above copyright notice and this permission notice shall be
271 included in all copies or substantial portions of the Software.
272
273 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
274 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
275 MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
276 NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
277 LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
278 OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
279 WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Something went wrong with that request. Please try again.