tox-ansible

Introduction

tox-ansible is a utility designed to simplify the testing of Ansible content collections.

Implemented as a tox plugin, tox-ansible provides a simple way to test Ansible content collections across multiple Python interpreters and Ansible versions.

tox-ansible uses familiar python testing tools to perform the actual testing. It uses tox to create and manage the testing environments, ansible-test sanity to run the sanity tests, and pytest to run the unit and integration tests. This eliminated the black box nature of other approaches and allowed for more control over the testing process.

When used on a local development system, each of the environments are left intact after a test run. This allows for easy debugging of failed tests for a given test type, python interpreter and Ansible version.

By using tox to create and manage the testing environments, Test outcomes should always be the same on a local development system as they are in a CI/CD pipeline.

tox virtual environments are created in the .tox directory. These are easily deleted and recreated if needed.

Talk to us

Need help or want to discuss the project? See our Contributor guide join the conversation!

Installation

Install from pypi:

pip install tox-ansible

Usage

From the root of your collection, add a [tool.tox-ansible] section to your pyproject.toml:

# pyproject.toml
[tool.tox]
requires = ["tox>=4.2"]

[tool.tox-ansible]

Then list the available environments:

tox list --ansible

A list of dynamically generated Ansible environments will be displayed:

default environments:
...
integration-py3.11-2.14      -> Integration tests for ansible.scm using ansible-core 2.14 and python 3.11
integration-py3.12-devel     -> Integration tests for ansible.scm using ansible-core devel and python 3.11
...
sanity-py3.11-2.14           -> Sanity tests for ansible.scm using ansible-core 2.14 and python 3.11
sanity-py3.12-devel          -> Sanity tests for ansible.scm using ansible-core devel and python 3.11
...
unit-py3.11-2.14             -> Unit tests for ansible.scm using ansible-core 2.14 and python 3.11
unit-py3.12-devel            -> Unit tests for ansible.scm using ansible-core devel and python 3.11

These represent the available testing environments. Each denotes the type of tests that will be run, the Python interpreter used to run the tests, and the Ansible version used to run the tests.

To run tests with a single environment, simply run the following command:

tox -e sanity-py3.11-2.14 --ansible

To run tests with multiple environments, simply add the environment names to the command:

tox -e sanity-py3.11-2.14,unit-py3.11-2.14 --ansible

To run all tests of a specific type in all available environments, use the factor -f flag:

tox -f unit --ansible -p auto

To run all tests across all available environments:

tox --ansible -p auto

Note: The -p auto flag will run multiple tests in parallel. Note: The specific Python interpreter will need to be pre-installed on your system, e.g.:

sudo dnf install python3.10

To review the specific commands and configuration for each of the integration, sanity, and unit factors:

tox config --ansible

Generate specific GitHub action matrix as per scope mentioned with --matrix-scope:

tox --ansible --gh-matrix --matrix-scope unit

Note: If your project uses tox-ansible.ini instead of pyproject.toml, add --conf tox-ansible.ini to each command above.

Configuration

tox-ansible is configured via a [tool.tox-ansible] section in pyproject.toml. The skip keyword filters environments by substring match:

# pyproject.toml
[tool.tox-ansible]
skip = [
    "2.16",
    "devel",
]

This will skip tests in any environment that uses Ansible 2.16 or the devel branch. The list of strings is used for a simple string in string comparison of environment names.

Alternatively, if using a tox-ansible.ini file, configure the [ansible] section:

# tox-ansible.ini
[ansible]
skip =
    2.16
    devel

See the configuration guide for details on overriding environment settings.

Release process

tox-ansible is released with CalVer scheme version numbers. The particular scheme we are using is YY.MM.MICRO, meaning that a release in March 2025 will be named 25.3.0, and if a patch (ie, non-feature) release is required for that release, it will be named 25.3.1, even if it is released in April. The month will not increment until a new version with features or other significant changes is released. More details about calver release process can be seen here.

Note to version 1.x users

Users of tox-ansible v1 should use the stable/1.x branch because the default branch is a rewrite of the plugin for tox 4.0+ which is not backward compatible with the old plugin.

Version 1 of the plugin had native support for molecule. Please see the "Running molecule scenarios" above for an alternative approach.

License

MIT