Skip to content
Permalink
Browse files

Merge pull request #179 from sneceesay77/GORA-532

GORA-532: Apache Gora Benchmark Module
  • Loading branch information...
lewismc committed Sep 8, 2019
2 parents c837c5a + 45a393c commit a354b35c5d77c0e67d4c2d01ca0306dd5f91e3d8
Showing with 5,473 additions and 1 deletion.
  1. +12 −0 gora-benchmark/.gitignore
  2. +60 −0 gora-benchmark/README.md
  3. +506 −0 gora-benchmark/gora-insert-csv
  4. +251 −0 gora-benchmark/gora-workloada.csv
  5. +251 −0 gora-benchmark/gora-workloadb.csv
  6. +196 −0 gora-benchmark/pom.xml
  7. +112 −0 gora-benchmark/src/main/avro/user.json
  8. +117 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/Constants.java
  9. +257 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java
  10. +391 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkUtils.java
  11. +1,963 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/generated/User.java
  12. +20 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/generated/package-info.java
  13. +20 −0 gora-benchmark/src/main/java/org/apache/gora/benchmark/package-info.java
  14. +43 −0 gora-benchmark/src/main/resources/gora-couchdb-mapping.xml
  15. +46 −0 gora-benchmark/src/main/resources/gora-hbase-mapping.xml
  16. +43 −0 gora-benchmark/src/main/resources/gora-mongodb-mapping.xml
  17. +40 −0 gora-benchmark/src/main/resources/gora.properties
  18. +309 −0 gora-benchmark/src/test/java/org/apache/gora/benchmark/GoraClientTest.java
  19. +20 −0 gora-benchmark/src/test/java/org/apache/gora/benchmark/package-info.java
  20. +283 −0 gora-benchmark/workloads/tsworkload_template
  21. +46 −0 gora-benchmark/workloads/tsworkloada
  22. +207 −0 gora-benchmark/workloads/workload_template
  23. +41 −0 gora-benchmark/workloads/workloada
  24. +36 −0 gora-benchmark/workloads/workloadb
  25. +38 −0 gora-benchmark/workloads/workloadc
  26. +41 −0 gora-benchmark/workloads/workloadd
  27. +46 −0 gora-benchmark/workloads/workloade
  28. +37 −0 gora-benchmark/workloads/workloadf
  29. +41 −1 pom.xml
@@ -0,0 +1,12 @@
bin
target
lib
bmstuff
workloads/*.swp
*.swp
pom.xml
runWorkloads.sh
setupgoramongo.js
setupycsbmongo.js
*.csv

@@ -0,0 +1,60 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# Background
This guide will take you through compiling and running the gora-benchmark module on a target database.

# Prerequisite
1. Install and setup the required database to test. Currently, the benchmark is tested on

1. MongoDB
2. HBase
3. CouchDB.

2. Install maven

# Setup the datasource to benchmark

1. Edit the gora.properties file located at src/main/resources/gora.properties and set configurations accordingly.

a. To test MongoDB set gora.datastore.default=org.apache.gora.mongodb.store.MongoStore

b. To test HBase set gora.datastore.default=org.apache.gora.hbase.store.HBaseStore

c. To test CouchDB set gora.datastore.default=org.apache.gora.couchdb.store.CouchDBStore

2. As of writing this guide, the YCSB project does not publish its jars in maven central. The simplest way to solve this problem is to do a local maven install. First download YCSB source and execute mvn clean install in the root directory. This may take some time to complete. You can add the -DskipTests switch to skip all test.

3. From the module directory i.e. gora-benchmark, run mvn clean install

4. Now run the benchmark using the following command

5. Load the database

./bin/gora-bench.sh load -threads 15 -s -p fieldcount=20 -p recordcount=1000 -p operationcount=1000 -P workloads/workloada

6. Run the workload

./bin/gora-bench.sh run -threads 15 -s -p readallfields=true -p measurementtype=timeseries -p timeseries.granularity=2000 -p operationcount=1000 -P workloads/workloadb


More details about the parameters and their usage can be found in the links below.

https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload

https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties

0 comments on commit a354b35

Please sign in to comment.
You can’t perform that action at this time.