Benchmark Suite for Apache Spark
- Current version: 2.0
Release date: 2015-5-10
- Getting Started
- Advanced Configuration
- Possible Issues
What's Benchmark Suite for Apache Spark ?
Spark-Bench is a benchmarking suite specific for Apache Spark. It comprises a representative and comprehensive set of workloads belonging to four different application types that currently supported by Apache Spark, including machine learning, graph processing, streaming and SQL queries.
The chosen workloads exhibit different workload characteristics and exercise different system bottlenecks; currently we cover CPU, memory, and shuffle and IO intensive workloads.
It also includes a data generator that allows users to generate arbitrary size of input data.
Why the Benchmark Suite for Apache Spark ?
While Apache Spark has been evolving rapidly, the community lacks a comprehensive benchmarking suite specifically tailored for Apache Spark. The purpose of such a suite is to help users to understand the trade-off between different system designs, guide the configuration optimization and cluster provisioning for Apache Spark deployments. In particular, there are four main use cases of Spark-Bench.
Usecase 1. It enables quantitative comparison for Apache Spark system optimizations such as caching policy and memory management optimization, scheduling policy optimization. Researchers and developers can use Spark-Bench to comprehensively evaluate and compare the performance of their optimization and the vanilla Apache Spark.
Usecase 2. It provides quantitative comparison for different platforms and hardware cluster setups such as Google cloud and Amazon cloud.
Usecase 3. It offers insights and guidance for cluster sizing and provision. It also helps to identify the bottleneck resources and minimize the impact of resource contention.
Usecase 4. It allows in-depth study of performance implication of Apache Spark system in various aspects including workload characterization, the study of parameter impact, scalability and fault tolerance behavior of Apache Spark system.
Machine Learning Workloads:
- Logistic Regression
- Support Vector Machine
- Matrix Factorization
Graph Computation Workloads:
- Triangle Count
- RDD Relation
- Twitter Tag
- Page View
- KMeans, LinearRegression, DecisionTree, ShortestPaths, LabelPropagation, ConnectedComponent, StronglyConnectedComponent, PregelOperation
Supported Apache Spark releases:
- Spark 1.2, 1.3, 1.4, 1.5 and 1.6
System setup and compilation.
Setup JDK, Apache Hadoop-YARN, Apache Spark runtime environment properly.
Download wikixmlj package: cd to a directory for download and type the next commands
git clone https://github.com/synhershko/wikixmlj.git cd wikixmlj mvn package install
Download/checkout Spark-Bench benchmark suite
<SPARK_BENCH_HOME>/bin/build-all.shto build Spark-Bench.
<SparkBench_Root>/conf/env.sh, and set it according to your cluster.
Make sure below variables has been set:
SPARK_HOME The Spark installation location
HADOOP_HOME The HADOOP installation location
SPARK_MASTER Spark master
HDFS_MASTER HDFS master
Note for SQL applications
For SQL applications, by default it runs the RDDRelation workload. To run Hive workload, execute
Note for streaming applications For Streaming applications such as TwitterTag,StreamingLogisticRegression First, execute
<SPARK_BENCH_HOME>/Streaming/bin/gen_data.shin one terminal; Second, execute
<SPARK_BENCH_HOME>/Streaming/bin/run.shin another terminal;
In order run a particular streaming app (default: PageViewStream): You need to pass a subApp parameter to the gen_data.sh or run.sh like this: <SPARK_BENCH_HOME>/Streaming/bin/run.sh TwitterPopularTags *Note: some subApps do not need the data_gen step. In those you will get a "no need" string in the output. You can make a certain subApp default by changing Streaming/conf/env.sh and changing the subApp= line with your choice of the streaming application.
In addition, StreamingLogisticRegression requires the
run.shscripts which launches Apache Spark applications can run simultaneously.
View the result.
<SPARK_BENCH_HOME>/reportto check for the final report.
Configuration for running workloads.
<SPARK_BENCH_HOME>/bin/applications.lstfile defines the workloads to run when you execute the bin/run-all.sh script under the package folder. Each line in the list file specifies one workload. You can use # at the beginning of each line to skip the corresponding bench if necessary.
You can also run each workload separately. In general, there are 3 different files under one workload folder.
<Workload>/bin/config.shchange the workload specific configurations
Apache Spark configuration.
spark.executors.memory Executor memory, standalone or YARN mode spark.driver.memory Driver memory, standalone or YARN mode spark.rdd.cache