Virtual-Infrastructure and Cloud Performance Benchmark
Java Perl JavaScript Perl 6 Python Smarty Other
Clone or download
Permalink
Failed to load latest commit information.
auctionApp If an item does not get any bids, put it up for auction again Jun 15, 2017
auctionConfigManager Fix for #33: CM uses external port May 15, 2017
auctionWeb Added initial READMEs for the sub-projects Mar 31, 2017
configFiles Change configuration of PostgreSQL container to use env variables Sep 20, 2017
dbLoader Merge branch 'master' of github.com:vmware/weathervane Mar 31, 2017
doc Add Config files for upcoming paper using Weathervane with Docker on … Sep 11, 2017
dockerImages Move MongoDB Storage Sanity-Check to on shutdown Jan 29, 2018
gradle/wrapper Adding code. Starting version is 1.0.9 Mar 31, 2017
images Adding code. Starting version is 1.0.9 Mar 31, 2017
runHarness If fetch of /proc/cpuinfo doesn't work, default numCpus to 1 May 21, 2018
tests Fixed formatting of tests/README.md Apr 5, 2017
workloadConfiguration Adding code. Starting version is 1.0.9 Mar 31, 2017
workloadDriver Remove extra postgres vacuum on startup: Nov 10, 2017
.gitignore Updated gitignore Mar 23, 2017
BRANCHING.md Initial BRANCHING and Branches documents Mar 21, 2017
Branches.md Fix formatting of Branches.md Apr 3, 2017
CONTRIBUTING.md Update contribution terms from CLA to DCO Apr 3, 2017
ISSUES.md Add notice that we will need output on bug reports Mar 23, 2017
LICENSE.txt Added updated License files Apr 5, 2017
Notice.txt Added updated License files Apr 5, 2017
README.md Switch to small logo on README page Apr 27, 2017
autoSetup.pl Fix to make sure autoSetup still works as root Apr 17, 2018
build.gradle Update software versions Oct 19, 2017
buildDockerImages.pl Fix discrepancy between configuration manager image name and name exp… Dec 13, 2017
gradle.properties Adding code. Starting version is 1.0.9 Mar 31, 2017
gradlew Adding code. Starting version is 1.0.9 Mar 31, 2017
gradlew.bat Adding code. Starting version is 1.0.9 Mar 31, 2017
runmany.sh Adding code. Starting version is 1.0.9 Mar 31, 2017
settings.gradle Adding code. Starting version is 1.0.9 Mar 31, 2017
version.txt Increment version numbers for a new release Apr 17, 2018
weathervane.config Increase default number of users to 1000 to improve sucess with firstRun Nov 1, 2017
weathervane.config.hierarchical Increase default number of users to 1000 to improve sucess with firstRun Nov 1, 2017
weathervane.config.simple Increase default number of users to 1000 to improve sucess with firstRun Nov 1, 2017
weathervane.pl Check for JAVA_HOME at the start of run Nov 10, 2017
weathervane_users_guide.docx Doc updates for 1.0.17 Nov 10, 2017
weathervane_users_guide.pdf Updated pdf of user's guide for 1.0.17 Nov 10, 2017

README.md

Weathervane

Overview

Weathervane is an application-level performance benchmark designed to allow the investigation of performance tradeoffs in modern virtualized and cloud infrastructures. It consists of an application, a workload driver that can drive a realistic and repeatable load against the application, and a run-harness that automates the process of executing runs and collecting results and relevant performance data. It can be used to investigate the performance characteristics of cloud and virtual infrastructures by deploying the application on the environment of interest, driving a load against the application, and examining the resulting performance metrics. A common use-case would involve varying some component or characteristic of the infrastructure in order to compare the effect of the alternatives on application-level performance.

A Weathervane Deployment

The figure above shows the logical layout of a Weathervane deployment. The Weathervane application is a web-application for hosting real-time auctions. This Auction application uses a scalable architecture that allows deployments to be easily sized for a large range of user loads. A deployment of the application involves a wide variety of support services, such as caching, messaging, NoSQL data-store, and relational database tiers. These services can be deployed in VMs or in Docker containers. The number of instances of some of the services can be scaled elastically at run time in response to a preset schedule or to monitored performance metrics. In addition, Weathervane supports deploying and driving loads against multiple instances of the Auction application in a single run, thus enabling performance investigations of resource constraints or other aspects of multi-tenancy. The Weathervane run-harness manages the complexity of configuring, deploying, and tuning the application, automating most of the tasks involved in configuring the application services and running the benchmark.

Additional background information about Weathervane can be found at https://blogs.vmware.com/performance/2015/03/introducing-weathervane-benchmark.html and http://blogs.vmware.com/performance/2017/04/weathervane-performance-benchmarking-now-open-source.html.

Warning

You should exercise caution and good judgement about where you deploy Weathervane. Weathervane can place a significant load on system resources that can impact the performance of unrelated workloads. As a result Weathervane is not intended to be used in production environments.

In addition, the setup procedure for Weathervane may open ports or present other security issues that are not acceptable in many secure environments. A Weathervane deployment is intended only for a performance test environment and may not be hardened sufficiently for all security requirements. As a result, Weathervane should never be installed on virtual machines or OS instances that have been or may ever be deployed in a production environment, or that contain sensitive or personal information.

Getting Started

Weathervane runs on one or more hosts or virtual machines (VMs) running the Centos 7 operating system. It includes a script to automate the initial configuration of the VM, including the installation of all necessary services. Simple runs of the benchmark can be performed on a single VM, but the application and workload driver can be scaled out to perform benchmark runs involving dozens of VMs.

The process of setting up and running Weathervane is covered in detail in the Weathervane User's Guide.

Documentation

The Weathervane User's Guide covers all aspects of installing, deploying, and running the Weathervane benchmark.

Communicating about Weathervane

The Weathervane project team can be contacted in the following ways:

  • Slack: This is the primary community channel. If you don't have an @vmware.com or @emc.com email, please sign up at https://code.vmware.com/web/code/join to get a Slack invite.

  • Gitter: Gitter is monitored but go to slack if you need a response quickly.

Feel free to contact us with questions or comments. Please use these method to contact us for questions specific to your use of Weathervane.

We also encourage the use of Issues in the GitHub repository for any questions or suggestions that may be of general interest. We may move discussion of questions or issues from Slack to a GitHub Issue if we feel that the question or answer may be of more general interest. Before creating an Issue, please read ISSUES.md.

Issues

The Weathervane project encourages liberal use of Issues for bugs, feature requests, documentation shortcoming, or general questions. Please read the issue guideline in ISSUES.md before filing an Issue. Be sure to follow the guidelines regarding issue labels to ensure a prompt response.

Releases & Major Branches

The branching and release strategy for Weathervane is discussed in BRANCHING.md.

Contributing

The Weathervane project team welcomes contributions from the community. Before you start working with Weathervane, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.

License

Weathervane is available under the the BSD 2-clause "Simplified" License.

Weathervane uses additional components which are downloaded when the project is built and which have additional licensing terms. The licenses for these components can be found in the license file.