What is this?
A load testing tool for Meteor applications.
** Important! Please don't load test sites running on meteor.com! Use your own servers! **
You can watch a video presentation about this from Meteor DevShop here: https://www.youtube.com/watch?v=tCPBGVI3PdA
This tool utilizes the Grinder to manage agents which execute a customized test script capable of speaking DDP with the target Meteor server. Each agent simulates 1 or more client connections and records performance metrics.
Users can configure options such as:
- Meteor methods to call with parameters
- Meteor subscriptions to initiate with parameters
- User login via email/pass or resumeTokens (for OAUTH)
- Number of clients to simulate
- Wait time between thread start
- Client ramp-up (process increment)
- Note: make sure you use leiningen version 2+, which you can get by using the installation
script from the web page. Some package managers, such as Debian/Ubuntu, will give you an older v1 version
which will result in
Could not find or load main classerrors. You can check which version of leiningen you have by running
- Note: make sure you use leiningen version 2+, which you can get by using the installation script from the web page. Some package managers, such as Debian/Ubuntu, will give you an older v1 version which will result in
Setup the project
git clone git://github.com/alanning/meteor-load-test.git cd meteor-load-test lein deps # downloads dependencies
There are three parts to running your load tests:
- The system you are testing
- The agents which execute the test script
- A console which manages all the agents
The System Under Test (SUT)
To start an example server
cd sut ./run
Open app in browser
The Grinder is organized into two tiers:
- the console, which controls agents and distributes the test scripts, and
- the agents, which execute the tests and report back to the console.
The console sends the configuration and test scripts to the agents, on demand, so modifying your tests and re-executing is straight-forward. Each agent can start multiple processes and each process can run your test script multiple times so generating a large amount of load is quite efficient.
Number of simulated clients = agents * processes * threads
Each thread (client) will:
- Request initial payload
- Request css & scripts found in initial response
- Initiate DDP connection
- Login with randomized user credentials (if supplied)
- Subscribe to subscriptions specified
- Perform method calls 'runs' number of times (see working.properties)
To start an agent
# in meteor-load-test directory bin/grinder agent start [optional host url of console - defaults to localhost]
Monitor agent log file
# in separate console window, from meteor-load-test directory cd log tail -f agent_1.log
To start the console
# in separate console window, from meteor-load-test directory bin/grinder console start
To run tests
In the Script tab, set the root directory to $PROJECT_HOME/grinder
working.properties and set it as the properties file to use (the star button)
working.properties and adjust as appropriate. Default setup will load test http://localhost:3000/
Click the play button in the top left (tooltip says, "Start the worker processes")
In the results tab, you should see test results
How to use
working.properties as appropriate. See the Grinder documentation for more options
Grinder agents should be started on separate boxes (not your webserver).
Note: Currently collection updates are received via Meteor subscriptions but there are no metrics gathered for how long it takes an update to be delivered under load. The closest we have right now is peak and mean TPS for DDP calls.
Given that, probably the most realistic way to test responsiveness of your app under load is to spin up your agents, kick off the tests, wait for them to saturate your server, and then visit your site via your own browser.
Controlling a remote agent
You will probably want to spin up agents on cloud machines so here's a couple useful things related to that:
- Install java, leiningen, and
meteor-load-teston all cloud instances and your local machine
- Start console on local machine:
$ bin/grinder console start
- SSH into cloud instances and remote forward port 6372:
$ ssh -i ~/.ssh/creds.pem -R 6372:localhost:6372 email@example.com
- Start agents on each cloud instance:
$ ssh <cloud instance> $ bin/grinder agent start
- Use grinder console to distribute properties script and start tests. Probably want to delete
Source: Goranka Bjedov, Google Tech Talk
- given load X, how quickly will the system return a result
- when will the system fail and how will it fail
- under what load will the system fail
- 80% of max load, what happens if run load for long period of time
- if I increase a certain resource, how will my through-put change
- used for very specific drill-down
- run load for 72+ hours
- five 'nines'
- if a system fails, how quickly will the system come back online
Why test Performance?
- Can our system return results as fast as customers want?
- To prove stats for agreements
- Peace of mind
Why NOT to test Performance?
- functional testing
Create Chef or Pallet scripts to automate creation of Grinder agent instances in the cloud
Explore recording timing information for length of time to finish receiving all collection updates
Based on load-testing-with-clojure by Andy Kriger which load-tests stateless websites.
Uses the java-ddp-client by Ken Yee to communicate with the target Meteor server.
Evolved from discussion on the meteor-talk google group: Load Testing Meteor. In particular, major thanks to Andrew Wilcox, Sam Hatoum, Matt DeBergalis, and Tom Coleman for their guidance and suggestions.