Skip to content
Nose plugin to facilitate randomized unit testing
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
nose_random
.gitignore
LICENSE
README.md
setup.py

README.md

nose-random

nose-random is designed to facilitate Monte-Carlo style unit testing. The idea is to improve testing by running your code against a large number of randomly generated input scenarios.

Even with random testing it's important that test success/failure is reproducible, otherwise it's hard to

  • know if you've fixed a failing test
  • know if an test fails only on some machines or configurations and not others
  • debug a failing test

nose-random avoids this pitfall because it

  • uses a fixed seed so that each test run is identical
  • tells you which scenario caused a test to fail
  • lets you to run the test only on a specific scenario to facilitate debugging

Installation

pip install git+https://github.com/ZoomerAnalytics/nose-random.git

or clone the repo and run python setup.py install.

Usage

The following example shows how to set up a randomized test.

# tests.py
import unittest

from nose_random import randomize

class RandomTestCase(unittest.TestCase):
    
    def generate_scenario(self, rng):
        return rng.random(), 10 * rng.random()
        
    @randomize(1000, generate_scenario)
    def failing_test(self, scenario):
        x, y = scenario
        self.assertLess(x, y)

    @randomize(1000, generate_scenario)
    def passing_test(self, scenario):
        x, y = scenario
        self.assertLess(x, y + 1)

Running nosetests from the same folder will produce the following output:

F.
======================================================================
FAIL: failing_test (tests.RandomTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\dev\nose-random\nose_random\__init__.py", line 62, in randomized_test
    test(self, scenario)
  File "C:\Dev\nose-random\examples\tests.py", line 13, in failing_test
    self.assertLess(x, y)
AssertionError: 0.9903329105632304 not less than 0.0015605977653532221 with scenario 2FDO1B08J20A (27 of 1000)

----------------------------------------------------------------------
Ran 2 tests in 0.067s

This means that the failing_test function succeeded with the first 26 scenarios but then failed with the 27th. To debug this scenario you can run the same command specifying the scenario id.

nosetests --scenario=2FDO1B08J20A

This will cause the tests to only be run against the failing scenario (note '1 of 1' as opposed to '27 of 1000').

AssertionError: 0.9903329105632304 not less than 0.0015605977653532221 with scenario 2FDO1B08J20A (1 of 1)

PyCharm

Note that the module will also pick up the --scenario= string if passed as a parameter to a PyCharm unittest run/debug configuration.

You can’t perform that action at this time.