|
|
@@ -23,7 +23,7 @@ or chat with us on [Freenode](https://freenode.net/faq.shtml#whatwhy) at `#teche |
|
|
remote server dedicated to this project or by using a virtual machine.
|
|
|
Here's a short example:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
# Clone the project, then move into the right directory
|
|
|
$ cd ~/FrameworkBenchmarks/deployment/vagrant-development
|
|
|
# Turn on the VM. Takes at least 20 minutes
|
|
|
@@ -104,42 +104,39 @@ See [here](deployment) for additional details. |
|
|
|
|
|
**Setting up the `load server`**
|
|
|
|
|
|
```
|
|
|
toolset/run-tests.py --install client --verbose --install-only
|
|
|
```bash
|
|
|
$ toolset/run-tests.py --install client --install-only
|
|
|
```
|
|
|
|
|
|
**Setting up the `database server`**
|
|
|
|
|
|
```
|
|
|
toolset/run-tests.py --install database --verbose --install-only
|
|
|
# We are still working to automate MongoDO. Until this, please run
|
|
|
# this as well (replacing database-ip with your own value)
|
|
|
mongo --host database-ip < config/create.js
|
|
|
```bash
|
|
|
$ toolset/run-tests.py --install database --install-only
|
|
|
```
|
|
|
|
|
|
**Setting up the `app server`**
|
|
|
|
|
|
You can choose to selectively install components by using the
|
|
|
`--test` and `--exclude` flags.
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
# Install just the software for beego (as an example)
|
|
|
toolset/run-tests.py --install server --test beego --verbose --install-only
|
|
|
$ toolset/run-tests.py --install server --test beego --verbose --install-only
|
|
|
|
|
|
# Install all php software but php-fuel (as another example)
|
|
|
toolset/run-tests.py --install server --test php* --exclude php-fuel --verbose --install-only
|
|
|
$ toolset/run-tests.py --install server --test php* --exclude php-fuel --verbose --install-only
|
|
|
|
|
|
# Install *all* framework software. Expect this to take hours!
|
|
|
# If running on a remote server, use `screen` or `tmux` or `nohup` to
|
|
|
# prevent the installation from being terminated if you are disconnected
|
|
|
toolset/run-tests.py --install server --verbose --install-only
|
|
|
$ toolset/run-tests.py --install server --verbose --install-only
|
|
|
```
|
|
|
|
|
|
## Listing Tests
|
|
|
|
|
|
You can easily list all available tests
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
$ toolset/run-tests.py --list-tests
|
|
|
activeweb
|
|
|
activeweb-raw
|
|
|
@@ -156,23 +153,23 @@ aspnet-mono-mongodb-raw |
|
|
|
|
|
There are a number of options that can be specified:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
# Run a verification for test beego
|
|
|
toolset/run-tests.py --test beego --mode verify
|
|
|
$ toolset/run-tests.py --test beego --mode verify
|
|
|
|
|
|
# Run the default benchmark for the beego test
|
|
|
toolset/run-tests.py --test beego
|
|
|
$ toolset/run-tests.py --test beego
|
|
|
|
|
|
# Specify which test types are run during benchmark
|
|
|
toolset/run-tests.py --test beego --type json
|
|
|
toolset/run-tests.py --test beego --type db
|
|
|
toolset/run-tests.py --test beego --type fortune
|
|
|
$ toolset/run-tests.py --test beego --type json
|
|
|
$ toolset/run-tests.py --test beego --type db
|
|
|
$ toolset/run-tests.py --test beego --type fortune
|
|
|
|
|
|
# Specify a number of options for how the load is generated
|
|
|
toolset/run-tests.py --test beego --max-concurrency 24 --max-threads 24 --duration 20 --max-queries 200
|
|
|
$ toolset/run-tests.py --test beego --max-concurrency 24 --max-threads 24 --duration 20 --max-queries 200
|
|
|
|
|
|
# Run a tiny benchmark
|
|
|
toolset/run-tests.py --test beego --max-threads 2 --max-concurrency 2
|
|
|
$ toolset/run-tests.py --test beego --max-threads 2 --max-concurrency 2
|
|
|
```
|
|
|
|
|
|
## Finding output logs
|
|
|
@@ -199,7 +196,9 @@ and all changes. These guidelines prevent us from having to give repeated feedba |
|
|
the same topics:
|
|
|
|
|
|
* **Use specific versions**: If you're updating any software or dependency, please be
|
|
|
specific with the version number. Also, update the appropriate `README` to reflect that change
|
|
|
specific with the version number. Also, update the appropriate `README` to reflect
|
|
|
that change. Don't rely on the package manager to deliver a specific version, apt
|
|
|
consistently returns different versions on Ubuntu 12.04 vs 14.04.
|
|
|
* **Rope in experts**: If you're making a performance tweak, our team may not be
|
|
|
able to verify your code--we are not experts in every language. It's always helpful
|
|
|
to ping expert users and provide a basic introduction on their credentials. If you
|
|
|
@@ -217,6 +216,10 @@ request will be verifed as normal. |
|
|
different concepts and frameworks, and it can really help to read the README's, such
|
|
|
as this one, the one inside the `toolset/` directory, and the ones inside specific
|
|
|
framework directories
|
|
|
* **Use the Development Virtual Machine**: Our Vagrant scripts can setup a VM for you
|
|
|
that looks nearly identical to our test environment. This is even better than relying
|
|
|
on the Travis-CI verification, and you are strongly encouraged to use this. See
|
|
|
the [deployment directory](deployment) for specifics
|
|
|
|
|
|
---
|
|
|
|
|
|
|
0 comments on commit
25f8944