Mozmill is a test tool and UI automation framework for writing tests and other automation scripts for Gecko based applications like Firefox and Thunderbird. It's built as a python command-line tool. The python package provides a mechanism for running the tests from the command line as well as providing a way to test restarting the application. Mozmill has an extensive API to help you write functional tests that simulate user interactions.
The Mozmill test automation project was started in January 2009 and covers the automation work for Firefox. Checkout the project page or have a look at the Mozmill Tests documentation to get an impression of how to contribute in writing and running Mozmill tests. Existing tests get run in the release testing cycle for new major or security releases of Firefox.
Also the Mozilla Messaging team has an active project which handles Thunderbird Testing with Mozmill.
Mozmill is available as a python package. See the installation page for instructions for how to get Mozmill set up on your system.
The Mozmill python package invokes and runs a Gecko application, performs automatic test scripting, and accumulates and reports results.
Running the command line client
the python package you can run Mozmill with the
mozmill command is run with one or more test (
-t mytest.js) or
test manifest (
mozmill -m functional_tests.ini mozmill -t mytest.js -t myothertest.js
mozmill --help displays the available command-line options and more
in-depth information about the command line utility. For the format
and usage of test manifests, see
The Mozmill python package bundles the Mozmill and jsbridge extensions into a profile on invocation.
Mozmill is run like:
mozmill -app firefox -b path/to/binary -t path/to/test.js [options]
This will do the following:
the application, in this case
firefox, will be looked for by mozrunner
a profile object will be created of the type appropriate to the application under test
test.jsfile will be sent over the jsbridge where it is loaded and executed (see: resource://mozmill/modules/frame.js )
upon test run conclusion, the results will be reported by pluggable event handlers
Example API Usage
Since Mozmill 2.0, the MozMill class is usable as a robust API. An example API usage is available at https://github.com/mozilla/mozmill/tree/master/mozmill .
Python Mozmill is a test harness and an event dispatcher.
The Mozmill python package is built of a number of different package dependencies:
- mozrunner : Reliable start/stop/configuration of Mozilla Applications (Firefox, Thunderbird, etc.)
- mozinfo : unified Mozilla interface to system information
- manifestparser : parses test and addon manifests
See Architecture for additional information on program design.
Getting Data to and From the Tests
persistedobject: a JSObject that is persisted between tests even if the application under test is shutdown or restarted. Each MozMill instance carries a
persistedobject. The amount of persisted data must be kept small, however, or jsbridge will fail.
- create your own JSObject for finer-tuned control getting data to and from the tests.
event handler included with Mozmill. The
firePythonCallback() function, which takes the
the name of the
method in the file, a list of ordered
args, and a
kwargs object. This function will dispatch a
mozmill.firePythonCallback event to the
which will import and fire the appropriate callback. The
return value from the python callback will not be sent to the
It is important for successful runs that the python callback is fired successfully. Otherwise a jsbridge error will occur via the python error and the harness will fail.
Restart and Shutdown
- user shutdown : the test indicates a shutdown or restart. This does
not stop the browser but indicates that a further action will cause
a restart or shutdown (such as triggering
- runner shutdown : the test tells the runner to shutdown or restart, potentially giving a next test to run in the same file.
Both cases fire an event,
that lets the python harness anticipate the type of shutdown or
restart. The following parameters are sent with the event:
user: true or false; whether the shutdown was signalled by a "user" event
restart: true or false; whether the shutdown is a restart or not
next: name of the next test function to run, in the current test file, if any; otherwise the next test file (if any) will be run
resetProfile: true or false; whether to reset the profile to the beginning state. Note that this is not available to user restart events as there is a race condition that does not permit the profile to be reliably reset before application restart
See the methods
stopApplication on the
MozMillController for specifics.
mozmill --restart signals a harness restart between
every test file. This is good for isolating test behaviour, but
negative in that the browser restart causes the run to take longer.
Learning Mozmill Testing
- Introduction to Mozmill : detailed tutorial that walks through introducing each Mozmill API object as it is needed
- Mozmill tests : how to setup and run the QA mozmill tests
Mozmill Test API
- controller object reference
- element object reference (Mozmill 2.0+)
- finding mozmill elements (Mozmill 2.0+)
- mozmill object reference
- extending the element hierarchy
- elementslib object reference (deprecated in Mozmill 2.0 - see finding mozmill elements)
- assertions API reference
Finding and Reporting Bugs
Mozmill is under active development. Check out the Auto-tools Mozmill project page for information on development. If you think you've found a bug in Mozmill, please check the list of existing bugs. If your found bug is not listed there, please file a new bug in bugzilla under the "Testing" Product and "Mozmill" Component. Please provide as much as possible details and attach the Mozmill test if available, which shows the problem. Thanks for helping us make Mozmill better!
Updating the Documentation
Several online resources exist for Mozmill:
- MDN Mozmill page
- github repository
- Python Package Index page
- Auto-tools Mozmill project page for the development of Mozmill
In addition a
#mozmill channel exists on irc://irc.mozilla.org/
. Please stop by and say hi!