What is it?
Testly is a language-agnostic tool for testing command line programs that read from
stdin and write to
stderr. It only tests the external behaviour of the program, not that of any internal functions, meaning that it is suited to testing simple programs, to quickly verify that they behave as intended. Use it to test your next fun, hacky project.
How do I get it?
[sudo] pip install testly
git clone git://github.com/lavelle/testly.git cd testly python setup.py install
How do I use it?
testly init to make a spec file in your project's directory, edit it to define your program's behaviour and then run
testly to run the tests.
Test cases are defined in a YAML file called
tests.yaml, in your project's root directory. You must run Testly from this directory, but you can store it anywhere (although somewhere in your
PATH is recommended for convenience).
The YAML file contains the command to execute to run your program and an object containing a heirarchy of test cases, defining arguments, input, expected output and errors and more. An example spec file is included in this repository. You can also read the full specification of the file format in the wiki.
Remember that JSON is a subset of YAML so if you are not familiar with YAML you can use JSON. YAML is advised, however, because it has support for multiline strings, allowing you to define your tests with a less verbose syntax.
For every test case, if the output matches the spec, the test passes. If not, a diff is shown, so you can work out what's going wrong.
You can pass
-w to the program and it will watch the source files of your program and re-run the tests every time they change. The files and directories to be watched are defined by an array of strings under the
watch property in the test spec file.
What does it need?
- Python 2.6
- Argparse if using Python version < 2.7
- Watchdog if you want the file watch functionality
- Pystache if you want templating support
- Added init feature
- Added support for testing error messages
- Added support for command line arguments, both global and per-case
- Added summary message showing total number of failed tests
- Tests are now defined in YAML instead of JSON
- Added support for templates
- Inital Release