Running functional tests

Please refer to the functional test architecture page to get an overview of the functional test suite architecture. The rest of this page assumes an understanding of the architecture.

The Cadasta Functional Test Suite can currently be run locally in the development VM against the local Django web server, or as an automated test build on Travis CI. The rest of this page explains how to set up the development VM so you can run all or some of the functional tests based on the selected Git branches of the cadasta-platform and cadasta-test repositories.

Setting up

Before running the functional tests, the following are required:

• Xvfb (X virtual frame buffer), a web browser, and the browser's corresponding Selenium WebDriver are installed
• The VM is pointing to the desired branches/commits/tags of the cadasta-platform and cadasta-test repositories
• The Django database is loaded with the functional test fixtures
• The Django web server is running

Setting up Xvfb, browser, and WebDriver

This should already be set-up when the development VM is provisioned. The relevant Ansible playbook should already take care of this for you.

However, there may be times that the Ubuntu repositories still do not have the latest packages or you may need to install a specific version of a browser or WebDriver. Consult Google for information on how to do this. You can find the GeckoDriver releases here and the ChromeDriver releases here.

Setting up the Git repository branches

In the VM, you can switch to the desired cadasta-platform branch by using git checkout on your host machine.

For the cadasta-test repository, you need to temporarily update the repository's line in the requirements/dev.txt file. After the @ symbol, you can specify either the branch name, commit hash, or tag. Examples:

• Point to the master branch of the repository:

  git+https://github.com/Cadasta/cadasta-test.git@master
  

• Point to a specific commit of the repository:

  git+https://github.com/Cadasta/cadasta-test.git@319e7bd1497bbde920678106dd92d22fedba8743
  

• Point to the v0.2.7 tag of the repository:

  git+https://github.com/Cadasta/cadasta-test.git@v0.2.7
  

After modifying the requirements file to point to the desired branch/commit/tag of the cadasta-test repository, please update the packages as follows:

$ cd /opt/cadasta/cadasta-platform
$ pip install -r requirements/dev.txt

Note: If the Python package version indicated in the desired branch/commit/tag is the same as before, the step above may not work correctly as pip will think that the package is already installed. In that case, uninstall the package first (shown below) before running the commands above:

$ pip uninstall cadasta-test

Loading the functional test fixtures

There is no need to wipe the database first as the fixtures are designed to avoid collisions with any existing data. (But it won't hurt to reset the database, if you don't need any existing data.)

(Optional) To reset the database:

$ cd /opt/cadasta/cadasta-platform/cadasta
$ ./manage.py resetdatabase

To load the fixtures:

$ cd /opt/cadasta/cadasta-platform/cadasta
$ ./manage.py loadfunctestfixtures

Starting the Django web server

This is a pretty standard step:

$ cd /opt/cadasta/cadasta-platform/cadasta
$ ./runserver &

The & character means that the server will be run in the background and any messages will be shown on the current shell session. If you prefer to have this on a different shell session, just SSH into the VM again to create a second session and then start the server in the foreground (without the & character):

$ cd /opt/cadasta/cadasta-platform/cadasta
$ ./runserver

Running functional tests

From the /opt/cadasta/cadasta-platform directory, you can now run functional tests.

To run all the tests:

$ ./runtests.py --functional

To run tests with a method name that matches a string pattern (this is based on standard Pytest functionality):

$ ./runtests.py --functional -k <pattern>

To run tests with a particular Pytest marker:

$ ./runtests.py --functional -m <marker>

To run tests without a particular Pytest marker:

$ ./runtests.py --functional -m 'not <marker>'

You can actually use many (not all) Pytest command arguments after the --functional argument as all subsequent arguments are passed directly to Pytest. See the Pytest usage documentation for more information.

Running tests with a particular browser

Running tests with a particular browser is controlled by the CADASTA_TEST_WEBDRIVER environment variable.

To use Chromium with the ChromeDriver, which is the default configuration, either leave the CADASTA_TEST_WEBDRIVER unset, or set it to Chrome (case sensitive, also not Chromium).

To use Firefox with the GeckoDriver, set CADASTA_TEST_WEBDRIVER to Firefox (case sensitive).

Running functional tests (alternative method)

The cadasta-test repository provides a local runtests.py script that provides greater control over which tests can be run. This script is located at /opt/cadasta/env/lib/python3.5/site-packages/cadasta/test/.

The problem with the runtests.py script in the cadasta-platform repository is that you cannot make full use of Pytest's mechanism of selecting tests by drilling down into directories, modules, and test methods because the script is not in the directory tree of the functional tests; you can only use the -k or the -m mechanisms.

The usage notes of the local runtests.py script is as follows:

$ ./runtests.py -h
usage: runtests.py [-h] [--host HOST]
                   [-w {Chrome,Firefox,BrowserStack-Chrome}]
                   [pyargs [pyargs ...]]

Runs the Cadasta platform functional test suite.

positional arguments:
  pyargs                optional arguments to be passed to pytest

optional arguments:
  -h, --help            show this help message and exit
  --host HOST           HTTP host and optional port pointing to the Cadasta
                        server to be tested (default: "http://localhost:8000")
  -w {Chrome,Firefox,BrowserStack-Chrome}, --webdriver {Chrome,Firefox,BrowserStack-Chrome}
                        Selenium WebDriver to use (default: Chrome)

For the pyargs arguments mentioned in the usage notes, in order to avoid triggering errors due to Python's parsing of command-line arguments, Pytest arguments that start with a hyphen (-) should instead be changed to start with a plus sign (+). For example, to enable Pytest's verbose mode, you can invoke the script as follows:

$ ./runtests.py +v

or:

$ ./runtests.py +-verbose

To run all functional tests in the account_tests directory:

$ ./runtests.py account_tests/

To run all login functional tests:

$ ./runtests.py account_tests/test_login.py

To run just a single specific login test:

$ ./runtests.py account_tests/test_login.py::TestLogin::test_user_can_login_by_username