What is it?
spec is a Python (2.6+ and 3.3+) testing tool that turns this:
- Colorized, specification style output
- Colorized tracebacks and summary
- Optional timing display for slow tests
- Test-running CLI tool which enables useful non-default options and implements
relaxed test discovery for less
spec is a BDD-esque
nose plugin designed to provide "specification"
style test output (similar to Java's
TestDox or Ruby's
RSpec). Spec-style output provides a more
structured view of what your tests assert, compared to
default "flat" mode of operation.
For example, this
nose-style test module:
class TestShape(object): def test_has_sides(self): pass def test_can_calculate_its_perimeter(self): pass class TestSquare(object): def test_is_a_shape(self): pass def test_has_four_sides(self): pass def test_has_sides_of_equal_length(self): pass
normally tests like so, in a single flat list:
TestShape.test_has_sides ... ok TestShape.test_can_calculate_its_perimeter ... ok TestSquare.test_has_four_sides ... ok TestSquare.test_has_sides_of_equal_length ... ok TestSquare.test_is_a_shape ... ok
spec enabled (
--with-spec), the tests are visually grouped by class,
and the member names are tweaked to read more like regular English:
Shape - has sides - can calculate its perimeter Square - has four sides - has sides of equal length - is a shape
In other words:
- Class-based tests are arranged with the class name as the subject, and the methods as the specifications;
- Any module-level tests are arranged with the module name as the subject;
- All objects' docstrings are used as their descriptions, if found. Otherwise:
CamelCaseNames(typically classes) have any leading/trailing
Teststripped, as well as any trailing underscore;
CamelCaseNamesalso get turned into sentences if necessary, so e.g.
Camel case names;
underscored_nameshave any leading/trailing
test(with its attached underscore) stripped;
underscored_nameshave underscores turned into spaces;
spec ships with a same-name command-line tool which may be used as a more
nosetests. In addition to toggling a number of useful default options
spec-the-program will honor
any and all public objects defined within your project's
meaning any file, function or class whose name does not begin with an
'_') and which is defined locally.
For example, given the following code inside
from external_module import a_function, AClass def _helper_function(args): return a_function(args) class _Parent(object): def this_will_not_get_tested(): pass class Feature(_Parent): def should_have_some_attribute(self): _helper_function(AClass) def does_something_awesome(self): self._helper_method() def _helper_method(self): pass def something_tested_by_itself_outside_a_class(): pass
only the following items will be picked up as test cases:
The imported function and class, the underscored functions/methods, and the methods inherited from a parent class, are all ignored.
Enhanced output via the Spec class
As with some other spec-style tools,
spec provides a means for nesting your
test "contexts" so they display nicely during test runs. Just use the
class as your primary superclass and inner classes will get parsed
from spec import Spec class ClassUnderTest(Spec): def it_behaves_like_this(self): # ... class init: "__init__" def takes_arg1(self): # ... def takes_arg2(self): # ...
The above results in output like so:
Class under test - it behaves like this __init__ - takes arg1 - takes arg2
This indentation makes output even easier to follow & helps keep things organized.
Accessing outer classes from inner ones
Frequently, you may have a useful
setup method in your outer class, and wish
to access objects attached to
self from inner classes. As of Spec 0.11.0 this
is now possible and is quite transparent; failed attribute lookups will check
an instantiated + setup'd copy of the outer class:
class MainClass(Spec): def setup(self): self.x = 'y' def outer_test(self): assert self.x == 'y' class some_inner_class: def inner_test(self): # Here, because some_inner_class has no real 'x' attribute, we end # up seeing the outer class' value. assert self.x == 'y'
Right now this support is pretty basic and assumes your setup methods are
setUp or whatnot. This will likely improve in
spec-the-tool's discovery algorithm, and
name transformation, we suggest the following for both readable code and
readable test output:
- Store tests in
tests/, with whatever file-by-file organization you like best;
- Within files, import the classes under test normally, e.g.
from mymodule import MyClass;
- Name the test classes identically, but with a trailing underscore to avoid
name collisions, and inheriting from the
class MyClass_(Spec): [...]
- Name their methods like English sentences, e.g.
def has_attribute_X(self): [...].
- Tests not yet filled out should call the
skipfunction, which raises a "skip this test" exception Nose handles correctly.
from spec import Spec, skip from mypackage import MyClass, MyOtherClass class MyClass_(Spec): def has_attribute_A(self): skip() class MyOtherClass_(Spec): def also_has_attribute_A(self): skip() def has_attribute_B(Spec): skip()
MyClass - has attribute A MyOtherClass - also has attribute A - has attribute B
Activation / command-line use
After installation via
pip or what have you,
expose these new additional options/flags:
--with-spec: enables the plugin and prints out your tests in specification format. Also automatically sets
--verbose(i.e. the spec output is a verbose format.)
--no-spec-color: disables color output. Normally, successes are green, failures/errors are red, and skipped tests are yellow.
--spec-doctests: enables (experimental) support for doctests.
--with-timing: enables timing display for tests that take >=
TIMING_THRESHOLD(see below) to run.
--timing-threshold: configuration of timing threshold (in seconds) for
--with-timing. Defaults to 0.1.
Why would I want to use it?
Specification-style output can make large test suites easier to read, and like any other BDD tool, it's more about framing the way we think about (and view) our tests, and less about introducing new technical methods for writing them.
Where did it come from?
spec is heavily based on the
spec plugin for Titus Brown's
set of Nose extensions. Said plugin was originally written by Michal
pinocchio and its
spec plugin are copyright © 2007
in the above two gentlemen's names, respectively.
This version of the plugin was created and distributed by Jeff Forcier, ©
2011. It tweaks the original source to be Python 2.7 compatible, based on
It also fixes a handful of bugs such as broken
compatibility under Nose 1.x, and then adds some additional functionality on
top (most notably the
spec command-line tool.)
What's the license?
Because this is heavily derivative of
spec is licensed the same
way -- under the MIT