Testing: Running the test suite

This page is about running SCT's test suite. If you would like to write new tests for SCT, please refer to the Writing New Tests Wiki page.

Pytest

Running unit/functional tests

Unit tests and functional tests are run using the pytest Python package. They are run as part of SCT's Continuous Integration (CI) builds. If you would like to test locally, however, you can simply activate the sct venv and run pytest:

# Setup environment
cd $SCT_DIR
source python/etc/profile.d/conda.sh
conda activate venv_sct

# Download a fresh copy of sct_testing_data
sct_download_data -d sct_testing_data

# Run all of the tests
pytest .

# Run the same tests as .ci.sh does
pytest testing/api/ testing/cli/

# Run only one test module
pytest testing/cli/test_sct_deepseg.py

Due to issue #2911, it's necessary to download a fresh copy of sct_testing_data before each run, otherwise pytest fails its integrity check.

Some of the tests may be skipped, because they depend on larger/private data files, run for a long time, or require a GPU. If you're a member of NeuroPoly and want to run these tests, check out the sct-testing-release dataset.

Global pytest configuration

Entries within the setup.cfg file define which pytest settings are applied by default.

You can override these settings locally by adding a separate pytest.ini file. This takes precedence over setup.cfg.

A note about parallelization

SCT does not currently use parallelization for tests. pytest-xdist was used for several months, however its downsides outweighed its benefits, so it was removed. (See PR #3212 for more context.)

batch_processing.sh

In addition to the test suites above, SCT also provides a shell script called batch_processing.sh to demonstrate typical usage of SCT's CLI scripts within a pipeline (i.e. chaining multiple CLI scripts together). This script is also run and checked for each pull request using SCT's continuous integration workflows. You can execute this script locally as follows:

cd $SCT_DIR
./batch_processing.sh

Changing your shell

To change the default shell for testing purpose, use the following command (example below to change for tcsh):

chsh -s /bin/tcsh