Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit

from 3.1.10 to 3.1.11.
Increased version to 3.1.11.</li>
Implemented the Java 15 feature &quot;text blocks&quot;.</li>
CheckStyle clean-up.</li>
CheckStyle clean-up.</li>
Fixed issue <a
&quot;Use Java's ServiceLoader instead of</li>
Fixed issue <a
&quot;Use Java's ServiceLoader instead of</li>
Bug fixed: Unlabeled BREAK did break a labeled statement, which is wrong
(it ...</li>
Bug fixed: Unlabeled BREAK did break a labeled statement, which is wrong
(it ...</li>
Bug fixed: Unlabeled BREAK did break a labeled statement, which is wrong
(it ...</li>
<li>Additional commits viewable in <a
<br />

[![Dependabot compatibility

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)


<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


Signed-off-by: dependabot[bot] <>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]>

Git stats


Failed to load latest commit information.
Latest commit message
Commit time
December 7, 2023 13:17
December 7, 2023 13:17
December 7, 2023 13:17
December 7, 2023 13:17


BenchBase (Java with Maven)

BenchBase (formerly OLTPBench) is a Multi-DBMS SQL Benchmarking Framework via JDBC.

Table of Contents


To clone and build BenchBase using the postgres profile,

git clone --depth 1
cd benchbase
./mvnw clean package -P postgres

This produces artifacts in the target folder, which can be extracted,

cd target
tar xvzf benchbase-postgres.tgz
cd benchbase-postgres

Inside this folder, you can run BenchBase. For example, to execute the tpcc benchmark,

java -jar benchbase.jar -b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true

A full list of options can be displayed,

java -jar benchbase.jar -h


Benchmarking is incredibly useful, yet endlessly painful. This benchmark suite is the result of a group of PhDs/post-docs/professors getting together and combining their workloads/frameworks/experiences/efforts. We hope this will save other people's time, and will provide an extensible platform, that can be grown in an open-source fashion.

BenchBase is a multi-threaded load generator. The framework is designed to be able to produce variable rate, variable mixture load against any JDBC-enabled relational database. The framework also provides data collection features, e.g., per-transaction-type latency and throughput logs.

The BenchBase framework has the following benchmarks:

This framework is design to allow for easy extension. We provide stub code that a contributor can use to include a new benchmark, leveraging all the system features (logging, controlled speed, controlled mixture, etc.)

Usage Guide

How to Build

Run the following command to build the distribution for a given database specified as the profile name (-P). The following profiles are currently supported: postgres, mysql, mariadb, sqlite, cockroachdb, phoenix, and spanner.

./mvnw clean package -P <profile name>

The following files will be placed in the ./target folder:

  • benchbase-<profile name>.tgz
  • benchbase-<profile name>.zip

How to Run

Once you build and unpack the distribution, you can run benchbase just like any other executable jar. The following examples assume you are running from the root of the expanded .zip or .tgz distribution. If you attempt to run benchbase outside of the distribution structure you may encounter a variety of errors including java.lang.NoClassDefFoundError.

To bring up help contents:

java -jar benchbase.jar -h

To execute the tpcc benchmark:

java -jar benchbase.jar -b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true

For composite benchmarks like chbenchmark, which require multiple schemas to be created and loaded, you can provide a comma separated list:

java -jar benchbase.jar -b tpcc,chbenchmark -c config/postgres/sample_chbenchmark_config.xml --create=true --load=true --execute=true

The following options are provided:

usage: benchbase
 -b,--bench <arg>               [required] Benchmark class. Currently
                                supported: [tpcc, tpch, tatp, wikipedia,
                                resourcestresser, twitter, epinions, ycsb,
                                seats, auctionmark, chbenchmark, voter,
                                sibench, noop, smallbank, hyadapt, otmetrics]
 -c,--config <arg>              [required] Workload configuration file
    --clear <arg>               Clear all records in the database for this
    --create <arg>              Initialize the database for this benchmark
 -d,--directory <arg>           Base directory for the result files,
                                default is current directory
    --dialects-export <arg>     Export benchmark SQL to a dialects file
    --execute <arg>             Execute the benchmark workload
 -h,--help                      Print this help
 -im,--interval-monitor <arg>   Throughput Monitoring Interval in
    --load <arg>                Load data using the benchmark's data
 -s,--sample <arg>              Sampling window

How to Run with Maven

Instead of first building, packaging and extracting before running benchbase, it is possible to execute benchmarks directly against the source code using Maven. Once you have the project cloned you can run any benchmark from the root project directory using the Maven exec:java goal. For example, the following command executes the tpcc benchmark against postgres:

mvn clean compile exec:java -P postgres -Dexec.args="-b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true"

this is equivalent to the steps above but eliminates the need to first package and then extract the distribution.

How to Enable Logging

To enable logging, e.g., for the PostgreSQL JDBC driver, add the following JVM property when starting...


To modify the logging level you can update and/or

How to Release

./mvnw -B release:prepare
./mvnw -B release:perform

How use with Docker

  • Build or pull a dev image to help building from source:



    docker run -it --rm --pull \
      -v /path/to/benchbase-source:/benchbase \
      -v $HOME/.m2:/home/containeruser/.m2 \
  • Build the full image:

    # build an image with all profiles
    # or if you only want to build some of them
    BENCHBASE_PROFILES='postgres mysql' ./docker/benchbase/
  • Run the image for a given profile:

    BENCHBASE_PROFILE='postgres' ./docker/benchbase/ --help # or other benchbase args as before


    docker run -it --rm --env BENCHBASE_PROFILE='postgres' \
      -v results:/benchbase/results --help # or other benchbase args as before

See the docker/benchbase/ for further details.

Github Codespaces and VSCode devcontainer support is also available.

How to Add Support for a New Database

Please see the existing MySQL and PostgreSQL code for an example.


We welcome all contributions! Please open a pull request. Common contributions may include:

  • Adding support for a new DBMS.
  • Adding more tests of existing benchmarks.
  • Fixing any bugs or known issues.

Known Issues

Please use GitHub's issue tracker for all issues.


BenchBase is the official modernized version of the original OLTPBench.

The original OLTPBench code was largely written by the authors of the original paper, OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases, D. E. Difallah, A. Pavlo, C. Curino, and P. Cudré-Mauroux. In VLDB 2014. Please see the citation guide below.

A significant portion of the modernization was contributed by Tim Veil @ Cockroach Labs, including but not limited to:

  • Built with and for Java 11 17.
  • Migration from Ant to Maven.
    • Reorganized project to fit Maven structure.
    • Removed static lib directory and dependencies.
    • Updated required dependencies and removed unused or unwanted dependencies.
    • Moved all non .java files to standard Maven resources directory.
    • Shipped with Maven Wrapper.
  • Improved packaging and versioning.
    • Moved to Calendar Versioning (
    • Project is now distributed as a .tgz or .zip with an executable .jar.
    • All code updated to read resources from inside .jar instead of directory.
  • Moved from direct dependence on Log4J to SLF4J.
  • Reorganized and renamed many files for clarity and consistency.
  • Applied countless fixes based on "Static Analysis".
    • JDK migrations (boxing, un-boxing, etc.).
    • Implemented try-with-resources for all java.lang.AutoCloseable instances.
    • Removed calls to printStackTrace() or System.out.println in favor of proper logging.
  • Reformatted code and cleaned up imports.
  • Removed all calls to assert.
  • Removed various forms of dead code and stale configurations.
  • Removed calls to commit() during Loader operations.
  • Refactored Worker and Loader usage of Connection objects and cleaned up transaction handling.
  • Introduced Dependabot to keep Maven dependencies up to date.
  • Simplified output flags by removing most of them, generally leaving the reporting functionality enabled by default.
  • Provided an alternate Catalog that can be populated directly from the configured Benchmark database. The old catalog was proxied through HSQLDB -- this remains an option for DBMSes that may have incomplete catalog support.

Citing This Repository

If you use this repository in an academic paper, please cite this repository:

D. E. Difallah, A. Pavlo, C. Curino, and P. Cudré-Mauroux, "OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases," PVLDB, vol. 7, iss. 4, pp. 277-288, 2013.

The BibTeX is provided below for convenience.

  author = {Djellel Eddine Difallah and Andrew Pavlo and Carlo Curino and Philippe Cudr{\'e}-Mauroux},
  title = {OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases},
  journal = {PVLDB},
  volume = {7},
  number = {4},
  year = {2013},
  pages = {277--288},
  url = {},