A benchmark for Bean Validation implementations
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Bean Validation benchmark


This work is derived from the Apache BVal benchmark described in this article from 2010.

Note that the new benchmarks have a slightly different behavior than what is described in the article.

It is licensed under the Apache 2 license.

Implementations available

Implementation Version Profile name
Apache BVal 1.1.2 bval
Hibernate Validator 5.4.2.Final hv-5.4
Hibernate Validator 6.0.10.Final hv-6.0
Hibernate Validator 6.1.0-SNAPSHOT hv-current

Generating the beans

Creating the scenario

To execute a benchmark, a scenario is required.

A scenario is a property file located at the root directory of the bean-generator-bv-1.1 module and called scenario.properties.

A default scenario file can be found at bean-generator-bv-1.1/src/main/resources/generator.default.properties.

If you don't define a specific scenario, the default one is used.

Generating the beans

Once your scenario is in place in the bean-generator-bv-1.1 module, you need to generate the beans.

From the root directory, run:

pushd bean-generator-bv-1.1
mvn clean install

The install goal must be used here so that the jars get installed to the local .m2 repository and can be reused in further builds. This is required to be able to run multiple benchmarks with the same set of generated beans.

Running the benchmark

Now that you have created your generated beans, corresponding JMH benchmarks jars can be created for the available Bean Validation implementations.

Let's say you want to compare the results of the full benchmark for the current stable and latest snapshot versions of Hibernate Validator.

First, jars including the JMH benchmarks and the corresponding implementation should be prepared.

Assuming root is the current location:

pushd jmh-benchmarks
mvn clean package -Phv-current
mvn package -Phv-6.0

Finally, you can run the benchmarks as follows:

java -jar target/bv-benchmarks-hv-current.jar
java -jar target/bv-benchmarks-hv-6.0.jar

The following benchmarks are available:

Name Main goal
RawValidationSpeedBenchmark Evaluate validation only
ParsingBeansSpeedBenchmark Evaluate the metadata building phase

It is also possible to run a single benchmark if needed. To do so pass a benchmark test name from the above table as a parameter:

java -jar target/bv-benchmarks-${impl}.jar RawValidationSpeedBenchmark


For now, our benchmarks only tests the Bean Validation 1.1 features.

It would be also good to have another bean-generator with beans using Bean Validation 2.0 features and some benchmarks for it.