Examples and units tests

The .help command

Willie has a built in command .help that can be used to get information about a particular command. You can add information to .help about your command in two basic ways. The first is to add a docstring and the second is to add an example usage with the @example decorator. Take the following module for example.

@willie.module.commands('hello')
def helloworld(bot, trigger):
    bot.say('Hello, %s!' % (trigger.group(2),))

It has no docstring or example so saying ".help hello" will do nothing. If the program is modified to include either of them like so:

@willie.module.commands('hello')
@willie.module.example('.hello world', 'Hello, world!')
def helloworld(bot, trigger):
    """Replies with what ever the first parameter is."""
    bot.say('Hello, %s!' % (trigger.group(2),))

Then we might get the following output:

User> .help hello
Willie> User: Replies with what ever the first parameter is.
Willie> e.g. .hello world

Unit tests with the @example decorator

The second parameter of @example is optional and provides a way of documenting what the expected output is. If you do add the expected output however, Willie will automatically make unit-tests to check that the example output is what really results from calling the command with the example input.

To run the unit tests and see the results you must have pytest installed and run pytest_run.py in Willies directory. You should see something like the following:

C:\Users\Venti\workspace2\willie>python pytest_run.py
============================= test session starts =============================
platform win32 -- Python 2.7.5 -- pytest-2.3.5
collected 27 items / 12 errors

willie/modules/calc.py .FF.
willie/modules/dice.py .......
willie/modules/exampletest.py .
willie/modules/rand.py .....
willie/modules/units.py .........
willie/modules/url.py .
=================================== ERRORS ====================================
(stuff removed for brevity)
================ 2 failed, 25 passed, 12 error in 4.63 seconds ================

You can also run the unit tests of your module directly by appending the following to the end of your module.

if __name__ == "__main__":
    from willie.test_tools import run_example_tests
    run_example_tests(__file__)

Now that you run the the module directly in your IDE or in command line, you should get something like the following.

C:\Users\Venti\workspace2\willie>set PYTHONPATH=C:\Users\Venti\workspace2\willie
C:\Users\Venti\workspace2\willie>python willie\modules\exampletest.py
============================= test session starts ==============================
platform win32 -- Python 2.7.5 -- pytest-2.3.5
collected 1 items

exampletest.py .

=========================== 1 passed in 0.01 seconds ===========================

Although only the first example will be added to the .help command, as many @example decorators as needed can be added for unit tests. This can be quite useful when making changes or adding new features.

We could for example add a test for the edge condition of calling .hello with no arguments.

@willie.module.example('.hello', 'Hello, !')

And then we can rerun the tests without restarting Willie or even reloading the module. Instantly seeing that the result was not what was expected and what the difference between expectation and reality was.

============================= test session starts ==============================
platform win32 -- Python 2.7.5 -- pytest-2.3.5
collected 2 items

exampletest.py F.

=================================== FAILURES ===================================
__________________________ test_example_helloworld_0 ___________________________
Traceback (most recent call last):
  File "C:\Users\Venti\workspace2\willie\willie\test_tools.py", line 112, in test
    assert result == output
AssertionError: assert 'Hello, !' == 'Hello, None!'
  - Hello, !
  + Hello, None!
  ?        ++++
====================== 1 failed, 1 passed in 0.03 seconds ======================

Using regular expressions with the @example decorator

TODO