Downloading the datasets
In order to run the benchmarks locally, you need the required datasets. Within the base of this directory, you can run the
./bin/download-dataset.sh script to download them into the data folder.
Running the benchmark locally
With the datasets downloaded, you can run the benchmark using
lein run -m crux.main.bench. This will run a Watdiv benchmark, and a benchmark using the DevOps devices and Weather datasets. The Watdiv benchmark against Crux will attempt to load results from other database benchmarks from AWS S3, so credentials are required if you wish to have comparison data in the outputs.
You can run individual benchmarks at the REPL by passing them a node (
(user/crux-node) works well for this, if you follow the instructions in the main README) - e.g.
Setting up AWS Credentials
If you wish to save to / load from S3 on your local benchmarks (used within the various Watdiv benchmarks) or to trigger AWS tasks locally, you will first need to set up AWS credentials. See further information on the AWS documentation.
Every push on
master triggers a deployment job on CircleCI, building a new docker image and uploading it to the Elastic Container Registry on AWS. The latest of these
master images is ran at midnight every single weekday.
Manually running a benchmark on AWS
Circle is responsible for building and pushing the bench docker images - it does this for every commit on the origin repo for branches with an active PR.
To deploy Docker images to ECR from your own branches, set up CircleCI to build your fork, and ensure you've added the four AWS env vars (
Assuming you have
jq installed, and you are authenticated with AWS and ECR (see here), you can run the bench for a CI-built commit using
Running Non-Crux Benchmarks
There are a number of other benchmarks for Watdiv on different databases, using our benchmark harness. We generally wish to run these on AWS, and use their results for comparison against Crux. They are ran based off of the
latest docker image pushed to AWS, passing a command to docker to override the default main namespace (which runs the Crux benchmark) with whichever namespace you wish to run. There are scripts to run these task overrides under