Skip to content

/ ndbench

Netflix Data Store Benchmark
HTML Java Other
  1. HTML 60.5%
  2. Java 39.3%
  3. Other 0.2%
Branch: master
Find file
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Open in Desktop Download ZIP

Latest commit

@vinaykumarchella
vinaykumarchella Removing retries on backfill operation failures, since it could lead … 
…to long running backfill operations.
Latest commit 5a114d1 Apr 11, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
codequality Replaces tabs with spaces (#130) Apr 9, 2018
gradle Updating gradle and nebula oss to latest versions Jul 12, 2018
img update tutorial overview diagram Oct 17, 2017
ndbench-api Separated AWS specific implementations, DefaultDataGenerator and Cass… Sep 7, 2018
ndbench-aws Use lambda and functional notation, remove unused imports Oct 17, 2018
ndbench-cass-plugins Adding column name validation in tandem with checksum validation Feb 11, 2020
ndbench-cli Commenting out dyno-plugin due to build issues Sep 6, 2019
ndbench-cockroachdb-plugins Closing connection to return connection back to the connection pool i… Jan 28, 2019
ndbench-core Removing retries on backfill operation failures, since it could lead … Apr 10, 2020
ndbench-dax-plugins Remove the aws module dependency for config initialization in dax and… Sep 8, 2018
ndbench-dynamodb-plugins Bubbling up exceptions so they count towards write failures as approp… Jan 15, 2019
ndbench-dyno-plugins Creating Host instance using HostBuilder Sep 19, 2019
ndbench-es-plugins Adding checksum validation capability Jul 12, 2019
ndbench-geode-plugins Use lambda and functional notation, remove unused imports Oct 17, 2018
ndbench-janusgraph-plugins Fix all PMD warnings throughout all modules (#186) Jul 10, 2018
ndbench-sample-plugins
ndbench-web Enable ndbench-dyno-plugins Sep 19, 2019
.gitignore Adding cockroach db plugin Oct 15, 2018
.travis.yml Migrate to openjdk8 as oraclejdk8 is deprecated on travis Aug 23, 2019
CONTRIBUTING.md Update CONTRIBUTING.md Nov 17, 2016
CONTRIBUTORS.md contributors Apr 9, 2018
LICENSE adding Apache License Oct 20, 2016
OSSMETADATA Initial commit for OSS Sep 1, 2016
README.md Removing EVCache plugins as the upcoming version of evcache client ha… Sep 6, 2018
build.gradle Delete unused gradle plugin from build file Jan 9, 2019
buildViaTravis.sh Gradle updates Mar 31, 2017
dynomite.yml make it work with local dynomite - change gitignore to igonore eclips… Sep 2, 2016
gradlew DynamoDB plugin (#76) Feb 13, 2018
gradlew.bat updates for gradle4 Sep 1, 2017
manifest.yml Add support for Geode and Cloud Foundry (#32) Aug 12, 2017
ndbench-proxy.js PR changes Oct 18, 2016
settings.gradle Enable ndbench-dyno-plugins Sep 19, 2019

README.md

NdBench

Build Status Dev chat at https://gitter.im/Netflix/ndbench Apache V2 License

Netflix Data Benchmark (NDBench) is a pluggable cloud-enabled benchmarking tool that can be used across any data store system. NDBench provides plugin support for several data stores and can be extended to other client APIs.

NDBench logo

Client APIs

  • Apache Cassandra (Thrift and CQL)
  • Redis
  • Elasticsearch
  • Elassandra
  • Amazon DynamoDB
  • Apache Geode
  • JanusGraph
  • Netflix Dynomite (Redis API)

Features

  • Dynamically change the benchmark configurations while the test is running, hence perform tests along with our production microservices.
  • Be able to integrate with platform cloud services such as dynamic configurations, discovery, metrics, etc.
  • Run for an infinite duration in order to introduce failure scenarios and test long running maintenances such as database repairs.
  • Provide pluggable patterns and loads.
  • Support different client APIs.
  • Deploy, manage and monitor multiple instances from a single entry point.
  • Support multiple cloud platform due to integration with Cloud Foundry

Details about the features can be found in the Wiki

Workflow

The stable version of NdBench is the master branch.

For questions or contributions, please consider reading CONTRIBUTING.md.

Build

NdBench comes with a Gradle wrapper

./gradlew build

The gradlew script will pull down all necessary gradle components/infrastructure automatically, then run the build.

NdBench provides several default implementations ( NdBenchConfiguration, LocalClusterDiscovery etc). You can use these or choose to create your own. NdBench currently works on AWS or your local environment. We are open to contributions to support other platforms as well.

How to

The first step before building ndbench is to configure the interfaces related to your environment in the InjectedWebListener. Checkout the Wiki for further explanation on what interfaces to bind based on your environment.

  1. Build ndbench
  2. Set up Auto-Scale Group (ASG) and spin up instances
  3. Deploy ndbench-web.war in your container

Deploy to Cloud Platforms

DISCOVERY_ENV environment variable is responsible to bind the proper Cluster Discovery. Currently AWS, CF and Local are supported. Hence set the environment variable DISCOVERY_ENV based on the environment you are deploying the app. For Amazon Web Services use AWS, for Cloud Foundry use CF, and for local deployments any other name. For Cloud Foundry, build and upload the ndbench war to CF using cf push.

Run

./gradlew appRun

Configuration

You can provide properties by using ndbench{version}.jar in your web container and then implementing IConfiguration Interface. More details on the how the configuration can be found in the Wiki.

Help

Need some help with either getting up and going or some problems with the code?

  • Submit a Github issue
  • Chat with us on Dev chat at https://gitter.im/Netflix/ndbench

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

You can’t perform that action at this time.