OLTP-Bench II

UPDATE 2021-09-28: Project moved to BenchBase.

The team behind the original OLTP-Bench project was kind enough to incorporate a majority of my changes from this fork back into their original codebase. Given the significance of those changes the team has decided to create a brand new repository and deprecate the original. As a result I will focus any new development effort on the newly created BenchBase project and limit my changes here. Thanks again to Andy Pavlo and team for their help and collaboration.

Forked from https://github.com/oltpbenchmark/oltpbench with a focus on cleanup and modernization. Given the volume and scope of these changes, I have elected not to submit pull requests to the original project as it is unlikely they would or could be accepted. Please see Modifications to Original for changes in this fork.

See also: OLTP-Bench: An extensible testbed for benchmarking relational databases D. E. Difallah, A. Pavlo, C. Curino, and P. Cudre-Mauroux. In VLDB 2014.

Benchmarks

From Original Paper

Added Later

TPC-H
TPC-DS - no configuration
hyadapt - no configuration
NoOp

Removed

JPAB - this project appears abandoned and hasn't seen an update since 2012. I don't have a great deal of faith in a Hibernate benchmark that hasn't kept pace with Hibernate.
LinkBench - no implementation

How to Build

Run the following command to build the distribution:

./mvnw clean package

The following files will be placed in the ./target folder, oltpbench2-x.y.z.tgz and oltpbench2-x.y.z.zip. Pick your poison.

The resulting .zip or .tgz file will have the following contents:

├── CONTRIBUTORS.md
├── LICENSE
├── README.md
├── config
│   ├── cockroachdb
│   │   ├── sample_auctionmark_config.xml
│   │   ├── sample_chbenchmark_config.xml
│   │   ├── sample_epinions_config.xml
│   │   ├── sample_noop_config.xml
│   │   ├── sample_resourcestresser_config.xml
│   │   ├── sample_seats_config.xml
│   │   ├── sample_sibench_config.xml
│   │   ├── sample_smallbank_config.xml
│   │   ├── sample_tatp_config.xml
│   │   ├── sample_tpcc_config.xml
│   │   ├── sample_tpcds_config.xml
│   │   ├── sample_tpch_config.xml
│   │   ├── sample_twitter_config.xml
│   │   ├── sample_voter_config.xml
│   │   ├── sample_wikipedia_config.xml
│   │   └── sample_ycsb_config.xml
│   ├── plugin.xml
│   └── postgres
│       └── ...
├── data
│   ├── tpch
│   │   ├── customer.tbl
│   │   ├── lineitem.tbl
│   │   ├── nation.tbl
│   │   ├── orders.tbl
│   │   ├── part.tbl
│   │   ├── partsupp.tbl
│   │   ├── region.tbl
│   │   └── supplier.tbl
│   └── twitter
│       ├── twitter_tweetids.txt
│       └── twitter_user_ids.txt
├── lib
│   └── ...
└── oltpbench2.jar

How to Run

Once you build and unpack the distribution, you can run oltpbench2 just like any other executable jar. The following examples assume you are running from the root of the expanded .zip or .tgz distribution. If you attempt to run oltpbench2 outside of the distribution structure you may encounter a variety of errors including java.lang.NoClassDefFoundError.

To bring up help contents:

java -jar oltpbench2.jar -h

To execute the tpcc benchmark:

java -jar oltpbench2.jar -b tpcc -c config/cockroachdb/sample_tpcc_config.xml --create=true --load=true --execute=true -s 5

For composite benchmarks like chbenchmark, which require multiple schemas to be created and loaded, you can provide a comma separated list: `

java -jar oltpbench2.jar -b tpcc,chbenchmark -c config/cockroachdb/sample_chbenchmark_config.xml --create=true --load=true --execute=true -s 5

The following options are provided:

usage: oltpbenchmark
 -b,--bench <arg>               [required] Benchmark class. Currently
                                supported: [tpcc, tpch, tatp, wikipedia,
                                resourcestresser, twitter, epinions, ycsb,
                                seats, auctionmark, chbenchmark, voter,
                                sibench, noop, smallbank, hyadapt]
 -c,--config <arg>              [required] Workload configuration file
    --clear <arg>               Clear all records in the database for this
                                benchmark
    --create <arg>              Initialize the database for this benchmark
 -d,--directory <arg>           Base directory for the result files,
                                default is current directory
    --dialects-export <arg>     Export benchmark SQL to a dialects file
    --execute <arg>             Execute the benchmark workload
 -h,--help                      Print this help
 -im,--interval-monitor <arg>   Throughput Monitoring Interval in
                                milliseconds
    --load <arg>                Load data using the benchmark's data
                                loader
 -s,--sample <arg>              Sampling window

How to see Postgres Driver logging

To enable logging for the PostgreSQL JDBC driver, add the following JVM property when starting...

-Djava.util.logging.config.file=src/main/resources/logging.properties

To modify the logging level you can update logging.properties

How to Release

./mvnw -B release:prepare
./mvnw -B release:perform

How to Add Support for a New Database

coming soon

Known Issues

Cockroach DB

My first priority is simply getting this code working against CockroachDB. No work has been put in to optimizing either the Database or the configurations for performance.

Benchmark	Config	Load	Run	Notes
`auctionmark`	✔️	✔️	✔️	~~issue #4~~, ~~issue #40~~
`chbenchmark`	✔️	✔️	✔️	~~issue #5~~, ~~issue #6~~
`epinions`	✔️	✔️	✔️	~~issue #7~~
`hyadapt`	❌			no config - issue #8
`noop`	✔️	✔️	✔️
`resourcestresser`	✔️	✔️	〰️	issue #41
`seats`	✔️	✔️	✔️	~~issue #10~~
`sibench`	✔️	✔️	✔️
`smallbank`	✔️	✔️	✔️
`tatp`	✔️	✔️	✔️
`tpcc`	✔️	✔️	✔️
`tpcds`	❌			no config - issue #11
`tpch`	✔️	✔️	✔️	~~issue #12~~
`twitter`	✔️	✔️	✔️	~~issue #13~~
`voter`	✔️	✔️	✔️
`wikipedia`	✔️	✔️	✔️
`ycsb`	✔️	✔️	✔️

Modifications to Original

This fork contains a number of significant structural modifications to the original project. This was done in an effort to clean up and modernize the code base, not to alter the spirit or function of the project. To this end, I did my best to leave the actual benchmark code functionally unchanged while improving where possible. My modifications are summarized below:

License

timveil-cockroach/oltpbench

ErrorLooks like something went wrong!

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Languages

Error
Looks like something went wrong!

Packages