value-provider

value-provider is a free library that facilitates writing realistic test data and in turn better tests for your Java application.
It works best in conjunction with reusable test data factories that encapsulate creating valid instances for your data objects.

value-provider consists of two major parts:

• the ValueProvider class which populates properties of test data objects with random values
• infrastructure for reproducing said random data in case of test failures (JUnit5 extension, JUnit4 rules)

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update the tests as appropriate.

For further information, please refer to CONTRIBUTING.md. For technical details, you may find the sequence diagrams in the doc directory helpful.

Installation

value-provider has the following prerequisites:

• Java 8 and above
• JUnit 5.5 and above for the JUnit5 infrastructure
• JUnit 4.12 and above for the JUnit4 infrastructure

Gradle

// core library
testImplementation 'com.tngtech.valueprovider:value-provider-core:1.2.2'

// infrastructure
// for JUnit 5
testImplementation 'com.tngtech.valueprovider:value-provider-junit5:1.2.2'
// alternatively, for JUnit 4
testImplementation 'com.tngtech.valueprovider:value-provider-junit4:1.2.2'

Maven

<!-- ... -->
<dependencies>
    <!-- ... -->

    <!-- core library -->
    <dependency>
        <groupId>com.tngtech.valueprovider</groupId>
      <artifactId>value-provider-core</artifactId>
      <version>1.2.2</version>
      <scope>test</scope>
    </dependency>

    <!-- infrastructure -->
    <!-- for JUnit 5 -->
    <dependency>
        <groupId>com.tngtech.valueprovider</groupId>
      <artifactId>value-provider-junit5</artifactId>
      <version>1.2.2</version>
      <scope>test</scope>
    </dependency>
    <!-- alternatively, for JUnit 4 -->
    <dependency>
        <groupId>com.tngtech.valueprovider</groupId>
      <artifactId>value-provider-junit4</artifactId>
      <version>1.2.2</version>
      <scope>test</scope>
    </dependency>

    <!-- ... -->
</dependencies>

Usage

We strongly recommend implementing reusable test data factories that encapsulate creating valid instances for your test data objects.

A Simple test data factory

Consider a simple Product¹...:

// ...
@Getter
@ToString
@EqualsAndHashCode
@RequiredArgsConstructor(staticName = "of")
public class Product {
    @NonNull
    private final ProductCategory category;
    @NonNull
    private final String name;
    @NonNull
    private final String description;
}

... the test data factory would look like... (see also ProductTestDataFactory):

import com.tngtech.valueprovider.ValueProvider;

import static com.tngtech.valueprovider.ValueProviderFactory.createRandomValueProvider;

public class ProductTestDataFactory {

    private ProductTestDataFactory() {
    }

    public static Product createProduct() {
        return createProduct(createRandomValueProvider());
    }

    public static Product createProduct(ValueProvider values) {
        return Product.of(
                values.oneOf(ProductCategory.class),
                values.fixedDecoratedString("name"),
                values.fixedDecoratedString("description"));
    }
}

The ValueProvider is used to

• select a ProductCategory at random
• populate the name and description properties with a so called decorated string

What is a decorated string? Let's have a look at the following example output when invoking ProductTestDataFactory.createProduct() multiple times:

Product(category=CAR, name=nameaPr, description=descriptionaPr)
Product(category=COMPUTER, name=nameyBp, description=descriptionyBp)
Product(category=COMPUTER, name=namejeM, description=descriptionjeM)

The decoration is simply a 3 letter suffix that is appended to the base string provided as parameter to the fixedDecoratedString() method.

Note that the suffix of all String properties of one Product instance stays the same, since the values are populated using the same ValueProvider instance. Conversely, different Product instances contain properties with different suffixes. This is achieved by invoking createRandomValueProvider(), when calling the ProductTestDataFactory.

Complex test data factories

Having seen the basics, let's move on to a more complex example. Consider an Order:

// ...
@Getter
@ToString
@EqualsAndHashCode
@Builder(toBuilder = true)
public class Order {
    @Singular
    @NonNull
    private final ImmutableList<OrderItem> orderItems;
    @NonNull
    private final Customer customer;
    @NonNull
    private final Address shippingAddress;
    @NonNull
    private final Optional<Address> billingAddress;

    public Address getBillingAddress() {
        return billingAddress.orElse(shippingAddress);
    }
    // ...
}

This time, let's start with the example output of invoking OrderTestDataFactory.createOrder():

Order(
orderItems=[
OrderItem(product=Product(category=COMPUTER, name=A-nameFhG, description=A-descriptionFhG), quantity=66),
OrderItem(product=Product(category=FOOD, name=B-nameFhG, description=B-descriptionFhG), quantity=8),
OrderItem(product=Product(category=COMPUTER, name=C-nameFhG, description=C-descriptionFhG), quantity=35),
OrderItem(product=Product(category=BOOK, name=D-nameFhG, description=D-descriptionFhG), quantity=89)],
customer=Customer(firstName=firstNameFhG, lastName=lastNameFhG, birthDate=1975-05-06),
shippingAddress=Address(zip=96874, city=S-cityFhG, street=S-streetFhG, number=315),
billingAddress=Address(zip=32924, city=B-cityFhG, street=B-streetFhG, number=120))
Order(
orderItems=[
OrderItem(product=Product(category=MAGIC_EQUIPMENT, name=A-namerwk, description=A-descriptionrwk), quantity=1),
OrderItem(product=Product(category=MAGIC_EQUIPMENT, name=B-namerwk, description=B-descriptionrwk), quantity=69),
OrderItem(product=Product(category=COMPUTER, name=C-namerwk, description=C-descriptionrwk), quantity=85)],
customer=Customer(firstName=firstNamerwk, lastName=lastNamerwk, birthDate=2002-06-08),
shippingAddress=Address(zip=08583, city=cityrwk, street=streetrwk, number=86),
billingAddress=Address(zip=08583, city=cityrwk, street=streetrwk, number=86))
Order(
orderItems=[
OrderItem(product=Product(category=COMPUTER, name=A-namekwh, description=A-descriptionkwh), quantity=73)],
customer=Customer(firstName=firstNamekwh, lastName=lastNamekwh, birthDate=1929-08-23),
shippingAddress=Address(zip=81571, city=S-citykwh, street=S-streetkwh, number=174),
billingAddress=Address(zip=71331, city=B-citykwh, street=B-streetkwh, number=169))

Note that the 3 letter suffix that we already saw in the Product example is shared for the entire hierarchy of objects that comprise an order. It therefore eases recognizing objects that belong together.

The output also demonstrates further aspects of randomization in test data factories. The 3 order objects all have a different number of order items, and e.g., the shipping and billing addresses have random zip codes or house numbers.

Last but not least, note the second aspect of string decoration. The products in the order items have an additional prefix in their string properties (e.g., 'A-', 'B-', ...). This is required to differentiate multiple objects of the same kind. The same applies to the shipping and billing addresses. They have different prefixes, if they differ ('S-' vs. 'B-'), but no prefix, if they are the same.

Let's take a look at OrderTestDataFactory - how to achieve all this:

import com.tngtech.valueprovider.example.Order.OrderBuilder;
import com.tngtech.valueprovider.ValueProvider;

import static com.tngtech.valueprovider.example.AddressTestDataFactory.createAddress;
import static com.tngtech.valueprovider.example.CustomerTestDataFactory.createCustomer;
import static com.tngtech.valueprovider.example.OrderItemTestDataFactory.createOrderItem;
import static com.tngtech.valueprovider.ValueProviderFactory.createRandomValueProvider;

public final class OrderTestDataFactory {

    private OrderTestDataFactory() {
    }

    public static Order createOrder() {
        return createOrder(createRandomValueProvider());
    }

    public static Order createOrder(ValueProvider values) {
        return createOrderBuilder(values).build();
    }

    public static OrderBuilder createOrderBuilder() {
        return createOrderBuilder(createRandomValueProvider());
    }

    public static OrderBuilder createOrderBuilder(ValueProvider values) {
        OrderBuilder builder = Order.builder()
                .customer(createCustomer(values));
        setAddress(builder, values);
        addItems(builder, values);
        return builder;
    }

    private static void setAddress(OrderBuilder builder, ValueProvider values) {
        boolean useDifferentBillingAddress = values.booleanValue();
        if (useDifferentBillingAddress) {
            builder
                    .shippingAddress(createAddress(values.copyWithChangedPrefix("S-")))
                    .billingAddress(createAddress(values.copyWithChangedPrefix("B-")));

        } else {
            builder
                    .shippingAddress(createAddress(values));
        }
    }

    private static void addItems(OrderBuilder builder, ValueProvider values) {
        int numOrderItems = values.intNumber(1, 5);
        for (int i = 0; i < numOrderItems; i++) {
            char prefix = (char) ('A' + i);
            ValueProvider prefixedProvider = values.copyWithChangedPrefix("" + prefix + "-");
            builder.orderItem(createOrderItem(prefixedProvider));
        }
    }
}

Sharing the same suffix for the entire object hierarchy is easy, just pass the ValueProvider instance to each invoked test data factory as in:

// ...
public final class OrderTestDataFactory {
    // ...
    public static OrderBuilder createOrderBuilder(ValueProvider values) {
        OrderBuilder builder = Order.builder()
                .customer(createCustomer(values));
        // ...
        return builder;
    }
    // ...
}

If you need multiple objects of the same type, add a prefix, like for the order item:

// ...
public final class OrderTestDataFactory {
    // ...
    private static void addItems(OrderBuilder builder, ValueProvider values) {
        int numOrderItems = values.intNumber(1, 5);
        for (int i = 0; i < numOrderItems; i++) {
            char prefix = (char) ('A' + i);
            ValueProvider prefixedProvider = values.copyWithChangedPrefix("" + prefix + "-");
            builder.orderItem(createOrderItem(prefixedProvider));
        }
    }
    // ...
}

The copyWithChangedPrefix() method takes the suffix of the ValueProvider for which it is called, and creates a new instance with the passed prefix. Like the suffix, the prefix remains the same for the lifetime of the ValueProvider.

A final aspect that is related to using lombok: As opposed to a Product, creating an Order is done via a builder rather than via a factory method. The OrderTestDataFactory therefore has 4 methods, a pair of createOrder() and a pair of createOrderBuilder() methods. Again, one of each pair has the ValueProvider as parameter to allow passing it on to invoked test data factories. The other one without parameter creates a new random ValueProvider.

Using factories in tests

Now comes the easy part: If you need a valid data object for your test, but don't care about its content, create one. If you need more than one data object with different but valid data, create another one:

import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrder;
// ...

class MyOrderTest {
    @Test
    void do_something_with_a_single_order() {
        Order anOrder = createOrder();
        // ...
    }

    @Test
    void do_something_with_two_different_orders() {
        Order anOrder = createOrder();
        Order anotherOrder = createOrder();
        // ...
    }
}

If you want to control specific aspects of a data object that are important for your test, restrict your test code to only these aspects:

import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrderBuilder;
// ...

class MyOrderTest {
    // ...
    @Test
    void shipping_address_is_used_as_default_for_billing_address() {
        Order useShippingAddressAsBillingAddress = createOrderBuilder()
                .billingAddress(empty())
                .build();
        // ...
    }
}

Please refer to OrderTest for more examples, and to the ValueProvider and its Javadoc to learn more about the methods it offers to populate the properties of your data objects.

Reproducing test failures caused by random data

As you have learned by now, using randomness helps minimize the code for creating test data. However, this comes at a price: If you want to reproduce test failures that might be related to random data, especially fom your CI suite of hundreds or even thousands of tests, it is vital to use the same data.

value-provider supports this use case out of the box by providing infrastructure for reproducing test failures.

Infrastructure

JUnit 5

For JUnit5, use the ValueProviderExtension:

import com.tngtech.valueprovider.ValueProviderExtension;
// ...

@ExtendWith(ValueProviderExtension.class)
class MyOrderTest {
    // ...
}

If your test class is derived from a base class, make sure to specify the ValueProviderExtension in the base class of the inheritance hierarchy.

JUnit 4

For JUnit4, use the ValueProviderRule:

import com.tngtech.valueprovider.ValueProviderRule;
// ...

public class MyOrderTest {
    @Rule
    public ValueProviderRule valueProviderRule = new ValueProviderRule();
    // ...
}

If your test uses static test data created by using a ValueProvider, use the ValueProviderClassRule in addition:

import com.tngtech.valueprovider.ValueProviderClassRule;
import com.tngtech.valueprovider.ValueProviderRule;
// ...
import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrder;
// ...

public class MyOrderTest {
    @ClassRule
    public static final ValueProviderClassRule staticProviders = new ValueProviderClassRule();
    @Rule
    public ValueProviderRule instanceProviders = new ValueProviderRule();

    // static test data
    private static final Order DEFAULT_ORDER = createOrder();
    // ...
}

If your test class is derived from a base class, make sure to specify the rule(s) in the base class of the inheritance hierarchy. Otherwise, your test (or other tests in a CI suite) may fail.

Reproducing test failures

If a test using the infrastructure fails, it provides information about the seed values used for generating the random data as shown in the following example:

org.junit.ComparisonFailure:
Expected :"testUserName1"
Actual   :"testUserName"
<Click to see difference>
...
    Suppressed: com.tngtech.valueprovider.ValueProviderException: If the failure is related to random ValueProviders, specify the following system properties for the JVM to reproduce:
-Dvalue.provider.factory.test.class.seed=0
-Dvalue.provider.factory.test.method.seed=-1608847119246027406
-Dvalue.provider.factory.reference.date.time=2021-06-04T15:28:34.004
at com.tngtech.valueprovider.ValueProviderRule.handleFailure(ValueProviderRule.java:57)
at com.tngtech.valueprovider.ValueProviderRule.access$100(ValueProviderRule.java:17)
at com.tngtech.valueprovider.ValueProviderRule$1.evaluate(ValueProviderRule.java:38)
...

If the failure is related to random data, you can easily reproduce it. Just specify the above shown JVM system properties in the command line when you re-run the test, e.g., in your IDE:

-Dvalue.provider.factory.test.class.seed=0
-Dvalue.provider.factory.test.method.seed=-1608847119246027406
-Dvalue.provider.factory.reference.date.time=2021-06-04T15:28:34.004

Note that seed values relate to individual test classes and methods, even if they have been run in a CI build together with other tests. Thus, it is sufficient to rerun the individual test in order to reproduce the failure.

Reproducible ValueProviders

The above example code always used ValueProviderFactory.createRandomValueProvider() to create a ValueProvider that, in turn, generates random data. To be more precise, the ValueProvider is initialized with a random seed and will generate exactly the same data, if it is initialized with the same seed, and the same sequence of method invocations is executed. As you may have guessed already, reproducing test failures is based on this functionality.

So, if you need reproducible test data, e.g., to test a transformation of Java data to XML by verifying against a previously stored XML file, use ValueProviderFactory.createReproducibleValueProvider() and provide a seed value of your choice to create the ValueProvider. Your test data factories will accept this ValueProvider just as any other one.

Extending ValueProvider functionality

The ValueProvider offers a considerable amount of common methods to fill properties of test data objects. Sooner or later however, the need will arise to add project specific functionality.

We advise the following approach:

• Create your own ValueProvider class. Let it extend the AbstractValueProvider provided by this library, and add the methods you need
• Create your own ValueProviderFactory class and implement 2 static methods that create instances of your derived AbstractValueProvider class
  • createRandomValueProvider()
  • createReproducibleValueProvider()
• Use your own classes instead of the ones provided by this library in your test data factories and other test code.

Refer to CustomValueProvider , CustomValueProviderFactory , and CustomValueProviderFactoryTest for a fully functional example.

Limitations wrt. multithreaded tests

The infrastructure uses thread-local data to store the seed values and can therefore be used in parallel CI builds without any problems.

However, reproducing test failures is not possible for multithreaded test code. Likewise, a ValueProvider initialized with a fixed seed value will not necessarily generate the same sequence of data if it is used by multiple threads, as the sequence of method invocations from different threads is not reproducible, and neither the ValueProvider nor the infrastructure provide any synchronisation.

License

value-provider is published under the Apache License 2.0, see license file for details.

¹Please note that we use lombok and immutable data objects in our examples for convenience, but this is not a requirement for using value-provider.