This repository has been archived by the owner. It is now read-only.
A plugin to allow nose to run tests in a random order
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
randomize
tests
.gitignore
README
README.md
requirements.txt
setup.cfg
setup.py

README.md

nose-randomize

A plugin to allow nose to run tests in a random order

Note: Project archived!

Hello, I have decided to archive this project. Please do not expect any further updates to it.

About

When nose runs the tests in a given project, the order that the tests are loaded and excuted is always the same.

For unit tests, an ideal situation is one of isolation: Each test should be able to run independently of the other tests in the project. If there is a dependency (such as test_A must run first in order for test_B to pass) then the test authors might want to look into how their tests are setup.

This plugin allows for randomization of the tests in a test class when they are run. In theory this should prove (or disprove) the isolation of the tests because they can be run in a random order every time and this should expose any pre-condition dependencies that might exist.

Installation

To install from source:

  • Checkout or download the source
  • cd into the directory and execute:
python setup.py install

To install from pip:

pip install randomize

Usage

Basic usage

The following will execute the tests and print out the seed that was used to start the random number generator. This is useful for tracking down test failures: if there is a depedency that causes a failure, re-running the test suite with the same seed should allow the failure to be repeatable.

nosetests --randomize

To re-run the tests with a given seed number, use this command line:

nosetests --randomize --seed=<whatever number you wish to use>

To only randomize the methods within a speicific class:

nosetests --randomize --class-specific

Note: To use the --class-specific feature you need to add a decorator to the class that will be randomized.

Limitations

Currently this plugin is only able to randomize the tests within a Class or Module. It does not support running the Classes in a random order.

For example if you have 3 test classes (TestClass1, TestClass2, TestClass3), they will be called in that order. The tests within each class will be executed in a random order, but the classes them selves will be called in the same order every time.


Important notes

  • The ability to randomize across all tests, not just within the TestCase class
  • High (as close to 100% as possible) test coverage. Currently 82%.