Skip to content

Commit

Permalink
Improve dynamic test demos in User Guide
Browse files Browse the repository at this point in the history 
Issue: #275
  • Loading branch information
@sbrannen
sbrannen committed Jun 3, 2016
1 parent e7f45c9 commit 9d1f8a3
Show file tree
Hide file tree
Showing 2 changed files with 107 additions and 71 deletions.
71 changes: 38 additions & 33 deletions documentation/src/docs/asciidoc/writing-tests.adoc
View file
Expand Up @@ -268,56 +268,61 @@ The above tests are merely meant as examples and therefore not complete.

=== Dynamic Tests

The standard `@Test` annotation of JUnit 5 is described in <<annotations>>.
It is very similar to the `@Test` annotation of JUnit 4.
The standard `@Test` annotation of JUnit 5 described in <<annotations>>
is very similar to the `@Test` annotation of JUnit 4.
Both describe methods implementing test cases.
These test cases are static in the sense that they are fully specified at compile-time
These test cases are static in the sense that they are fully specified at compile-time,
and their behavior cannot be changed by anything happening at run-time.
(Assumptions do provide a basic form of dynamic behavior but are intentionally rather limited in their expressiveness.)
_Assumptions provide a basic form of dynamic behavior but are intentionally rather limited in their expressiveness._

In addition to these standard tests a completely new kind of test programming model is introduced in JUnit 5.
This new kind of test is a dynamic test which is generated at run-time by a test factory method.
Such a factory method must be annotated with `@TestFactory`.

In contrast to `@Test` such a factory is not itself a test case but creates test cases.
Thus, dynamic tests are the products of the factory.
More technically, a `@TestFactory` method must return a `Stream` or `Iterable` of `DynamicTest` instances.
These `DynamicTest` instances will then be executed lazily
This new kind of test is a _dynamic test_ which is generated at run-time by a test factory method
which must be annotated with `@TestFactory`.

In contrast to `@Test` such a factory is not itself a test case but rather creates test cases.
Thus, dynamic tests are the products of a factory.
Technically speaking, a `@TestFactory` method must return a `Stream`,
`Collection`, `Iterable`, or `Iterator` of `DynamicTest` instances.
These `DynamicTest` instances will then be executed lazily,
enabling dynamic and even non-deterministic generation of test cases.

As with @Test methods, `@TestFactory` methods must not be `private` or `static`
As with `@Test` methods, `@TestFactory` methods must not be `private` or `static`
and may optionally declare parameters to be resolved by `ParameterResolvers`.

A `DynamicTest` is a test case generated at runtime.
It is composed of a display name and an `Executable` which is a `@FunctionalInterface`.
Note that dynamic tests are quite different from standard `@Test` cases
since callbacks such as `@BeforeEach` and `@AfterEach` methods are not executed for dynamic tests.
The execution of those callbacks would not be useful in most scenarios
as variables used in lambdas are bound to instance variables at execution time of the factory method
It is composed of a _display name_ and an `Executable` which is a `@FunctionalInterface`.
Note that the execution lifecycle of dynamic tests is quite different than it is for standard `@Test` cases.
Specifically, `@BeforeEach` and `@AfterEach` methods and their corresponding extension callbacks
are not executed for dynamic tests.
The execution of those callbacks would not be useful in most dynamic testing scenarios
since variables used in lambdas are bound to instance variables at execution time of the factory method
rather than execution time of the lambda itself.

The following demo test class shows a few examples of test factories and dynamic tests.
==== Dynamic Test Examples

[source,java]
[subs="verbatim"]
----
include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide]
----
The following `DynamicTestsDemo` class demonstrates several examples of test factories and dynamic tests.

The first method shows an illegal return type.
As such bad return types cannot be detected at compile-time
The first method returns an invalid return type.
Since invalid return types cannot be detected at compile-time
a `JUnitException` is thrown when it is detected at run-time.

The following four methods are very simple examples to demonstrate
the generation of a `List`, a `Stream`, an `Iterator` and an `Iterable` of `DynamicTest` instances, respectively.
The next four methods are very simple examples that demonstrate the generation of a
`Stream`, a `Collection`, an `Iterable`, and an `Iterator` of `DynamicTest` instances, respectively.
These examples do not really exhibit dynamic behavior
but merely demonstrate the allowed syntaxes in principle.
but merely demonstrate the supported return types in principle.

The last two methods are truely dynamic in nature.
They use generator functions to create an `Iterator` or a `Stream` of `DynamicTest` instances.
The last method demonstrates non-deterministic behavior.
While this is of course in conflict with test repeatability and should thus be used with care
it serves to demonstrate the expressiveness of dynamic tests.
The last two methods are truly dynamic in nature.
`dynamicTestsFromCustomIterator()` implements a custom `Iterator` that generates `DynamicTest` instances;
whereas, `generateRandomNumberOfTests()` implements an `Iterator` that generates random numbers, a custom
display name generator, and a test generator and provides all three to `DynamicTest.stream()`.
Although the non-deterministic behavior of `generateRandomNumberOfTests()` is of course in conflict with
test repeatability and should thus be used with care, it serves to demonstrate the expressiveness of dynamic tests.

As of M1, dynamic tests must always be created by factory methods.
This might be complemented by a registration facility in one of the upcoming releases.

[source,java]
[subs="verbatim"]
----
include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide]
----
107 changes: 69 additions & 38 deletions documentation/src/test/java/example/DynamicTestsDemo.java
View file
Expand Up @@ -10,67 +10,90 @@

package example;

// tag::user_guide[]
import static org.junit.gen5.api.Assertions.assertFalse;
import static org.junit.gen5.api.Assertions.assertTrue;
import static org.junit.gen5.api.Assertions.fail;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Stream;

import org.junit.gen5.api.DynamicTest;
import org.junit.gen5.api.Tag;
import org.junit.gen5.api.TestFactory;

// end::user_guide[]
@Tag("exclude")
// tag::user_guide[]
class DynamicTestsDemo {

// This will result in a JUnitException!
@TestFactory
List<String> dynamicTestsWithWrongReturnType() {
List<String> tests = new ArrayList<>();
tests.add("Hello");
return tests;
List<String> dynamicTestsWithInvalidReturnType() {
return Arrays.asList("Hello");
}

@TestFactory
List<DynamicTest> dynamicTestsFromList() {
List<DynamicTest> tests = new ArrayList<>();

tests.add(new DynamicTest("succeedingTest", () -> assertTrue(true, "succeeding")));
tests.add(new DynamicTest("failingTest", () -> assertTrue(false, "failing")));

return tests;
Stream<DynamicTest> dynamicTestsFromStream() {
// end::user_guide[]
// @formatter:off
// tag::user_guide[]
return Stream.of("test1", "test2", "test3")
.map(displayName -> new DynamicTest(displayName, () -> { /* ... */ }));
// end::user_guide[]
// @formatter:on
// tag::user_guide[]
}

@TestFactory
Stream<DynamicTest> dynamicTestsFromStream() {
String[] testNames = new String[] { "test1", "test2" };
return Arrays.stream(testNames).map(name -> new DynamicTest(name, () -> {
}));
Collection<DynamicTest> dynamicTestsFromCollection() {
// end::user_guide[]
// @formatter:off
// tag::user_guide[]
return Arrays.asList(
new DynamicTest("succeedingTest", () -> assertTrue(true)),
new DynamicTest("failingTest", () -> fail("failing"))
);
// end::user_guide[]
// @formatter:on
// tag::user_guide[]
}

@TestFactory
Iterator<DynamicTest> dynamicTestStreamFromIterator() {
List<DynamicTest> tests = new ArrayList<>();
tests.add(new DynamicTest("succeedingTest", () -> assertTrue(true, "succeeding")));
tests.add(new DynamicTest("failingTest", () -> assertTrue(false, "failing")));
return tests.iterator();
Iterable<DynamicTest> dynamicTestsFromIterable() {
// end::user_guide[]
// @formatter:off
// tag::user_guide[]
return Arrays.asList(
new DynamicTest("succeedingTest", () -> assertTrue(true)),
new DynamicTest("failingTest", () -> fail("failing"))
);
// end::user_guide[]
// @formatter:on
// tag::user_guide[]
}

@TestFactory
Iterable<DynamicTest> dynamicTestStreamFromIterable() {
List<DynamicTest> tests = new ArrayList<>();
tests.add(new DynamicTest("succeedingTest", () -> assertTrue(true, "succeeding")));
tests.add(new DynamicTest("failingTest", () -> assertTrue(false, "failing")));
return tests;
Iterator<DynamicTest> dynamicTestsFromIterator() {
// end::user_guide[]
// @formatter:off
// tag::user_guide[]
return Arrays.asList(
new DynamicTest("succeedingTest", () -> assertTrue(true)),
new DynamicTest("failingTest", () -> fail("failing"))
).iterator();
// end::user_guide[]
// @formatter:on
// tag::user_guide[]
}

@TestFactory
Iterator<DynamicTest> generatedTestsFromGeneratorFunction() {
Iterator<DynamicTest> generator = new Iterator<DynamicTest>() {
Iterator<DynamicTest> dynamicTestsFromCustomIterator() {
return new Iterator<DynamicTest>() {

int counter = 0;

Expand All @@ -85,30 +108,38 @@ public DynamicTest next() {
return new DynamicTest("test" + index, () -> assertTrue(index % 11 != 0));
}
};
return generator;
}

@TestFactory
Stream<DynamicTest> generatedRandomNumberOfTests() {
final int AVERAGE = 49;
Stream<DynamicTest> generateRandomNumberOfTests() {

Iterator<Integer> generator = new Iterator<Integer>() {
// Generates random positive integers between 0 and 100 until
// a number evenly divisible by 7 is encountered.
Iterator<Integer> inputGenerator = new Iterator<Integer>() {

final Random random = new Random();
int last = -1;
Random random = new Random();

@Override
public boolean hasNext() {
return last % AVERAGE != 0;
return last % 7 != 0;
}

@Override
public Integer next() {
last = random.nextInt();
last = random.nextInt(100);
return last;
}
};
return DynamicTest.stream(generator, index -> "test" + index, index -> assertFalse(index % AVERAGE == 0));

// Generates display names like: input:5, input:37, input:85, etc.
Function<? super Integer, String> displayNameGenerator = (input) -> "input:" + input;

// Generates tests based on the current input value.
Consumer<? super Integer> testGenerator = (input) -> assertTrue(input % 3 == 0);

// Creates a stream of dynamic tests.
return DynamicTest.stream(inputGenerator, displayNameGenerator, testGenerator);
}

}
Expand Down

0 comments on commit 9d1f8a3

Please sign in to comment.