Add the ability to run tests concurrently

[thedrow]

Since end to end tests are usually time consuming, the ability to run the tests concurrently will help shortening each test cycle.

[crdoconnor]

It actually might not. The service startup code is substantially parallelized and in the example test, for instance, starting postgres, celery, django, redis and firefox all at the same time chews up all of my CPU cores (not to mention I/O).

I tried running two tests side by side in their own virtual machine and it ran slower than running them sequentially in the VM.

If you have a substantial number of CPU cores (e.g. 32 or 64), it might be worth figuring out a way to parallelize tests on one machine, but at the moment most people seem to have 4 or 8 at most.

I am working on a way to parallelize test runs by running each test in its own short-lived virtual machine on a platform like EC2. This will let you run, say, 500 x 1 minute tests in parallel in just one minute rather than 8 hours. Stay tuned - that is coming.