Running Tests

RoryH edited this page Oct 24, 2012 · 4 revisions

This chapter tells you how to playback tests that you’ve written/recorded and played back.

  • Important Note: Testing Safari when it’s running your test application on localhost will give you a Python traceback, this is currently not supported.

Running and Loading

Windmill distinguishes between running and loading tests. This distinction helps when you’re debugging and editing tests.

When running a test file or set of test files you are telling the service to run those tests in the IDE as soon as it sends them. Loading tests means that you want all the test to load in to the server and in the IDE as quickly as possible and not run them. This is very useful for when you want to load up a test for editing.

All the test authoring libraries written and maintained by the windmill project support loading and running of tests but windmill’s open architecture allows and encourages third parties to develop authoring libraries and it is up to them to implement loading and running.

Running Python Tests from the Command Line

When windmill starts you can give it the test option. This option allows you to define a single test entry point to be run. This can be a file or directory ( provided the directory is a valid python module, more on this later ).

windmill firefox

Or load your test with

windmill firefox

Again, these options can also take test directories if they are valid python modules.

Running or Loading Python Tests from the Shell

The following methods can take either a single file or a directory and will run each as expected. If you are passing a directory, it can either be absolute or relative.



Command Line

Run Tests: ex. windmill firefox test=/path/to/tests/

Load Tests: ex. windmill firefox loadtest=/path/to/tests/

Organizing Tests

When you’re using multiple directories/modules to run tests those directories/modules need to begin with “test”. Those directories also need to include a file named This file can be empty.


Creating setup/teardown and Managing Test Dependencies for Python Tests

The setup/teardown process is handled in python by functest (or nose if you run your tests from nose). In this section we’ll try to provide you with a crash course in setting up simple setup/teardown and dependencies.

Imagine the following scenario. You have about 30 tests across 3 pages (page1.html, page2.html, page3.html) on your site. Pages 2 and 3 require you to login, page 1 requires you are not logged in. The directory structure of your tests should look something like this;

           / ( Defines any overall setup/teardown for the entire test run)
           /test_page1/ ....
           /test_logged_in/ (Defines a setup_module() which logs in and a teardown_module() which logs out.
           /test_logged_in/test_page2/ ...
           /test_logged_in/test_page3/ ...

This illustrates a fairly simple the way you can use the module hierarchy as your dependency structure.

If you need to work on and tell windmill to run/load it, the collector will work it’s way up the module chain and find the root module, it will then run any setup_module’s that exist in parent directories before it runs

Detailing the

This file is the core of the setup and tear down functionality, and it is important for you to have an understanding of how it works.


 def setup_module(module):
    client = WindmillTestClient(__name__)

    client.waits.forElement(id=u'email', timeout=u'30000')'email')
    client.type(text=functest.registry['email'], id=u'email')
    client.waits.forElement(id=u'pass', timeout=u'30000')  
    client.type(text=functest.registry['password'], id=u'pass')"Login")

def teardown_module(module):
    client = WindmillTestClient(__name__)
    client.waits.forElement(link=u'Home', timeout=u'100000')'Home')
    client.waits.forElement(timeout=u'100000', id=u'q')
    client.waits.forElement(link=u'Logout', timeout=u'100000')'Logout')

Running Tests from Nose

As of windmill version 1.1 a nose plugin is included with windmill. If you run:
nosetests --help

You’ll see a bunch of additional windmill commands.

Running your tests from nose is a lot like running them in functest, you do need to add one additional import in your base test module.
from windmill.authoring import setup_module, teardown_module

Now you can run your tests right from the command line with nose from inside your test directory.
nosetests --wmbrowser firefox --wmtesturl

Running JavaScript Tests

To run your JavaScript tests you can use one of the following methods.

From the shell: run_js_tests('dir/to/js/tests')

From the command line: windmill firefox jsdir=./js/test/dir

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.