Setting up test run configurations with PyCharm
As described in the setup and installation guide it is possible to set up run configurations to allow easily running the TLO tests locally in PyCharm. Below we describe how to set up multiple configurations to run different subsets of the tests, optionally run tests in parallel across multiple cores and run the tests with multiple different random seeds for the simulation random number generator.
Although in all the examples below, the runs are configured to run all tests in the tests subdirectory of the TLOModel repository root directory, by changing the text-entry field below Target: in the run configurations dialog, this can be changed to for example run only the tests in a specific test module.
A PyTest plugin pytest-xdist can be used to distribute test runs across multiple CPU (cores) to speed up the overall run time. If you have multiple CPU cores available on your local machine you may therefore wish to install this plugin to allow quicker local test runs. This step is optional however and can be skipped (in this case be careful to not add the -n argument specifying the number of CPU cores to use to the Additional Arguments field in the run configurations below).
To install the pytest-xdist package from the Python Packaging Index (PyPI), first open the Python Packages tab in PyCharm. This is usually by default located in the tab strip at the bottom of the screen, if you cannot find it here however it can be accessed by going to the View menu and under the Tool Windows submenu selecting Python Packages.
Once you have opened the Python Packages tool, you should be presented with an interface like below
In the search dialog labelled Search for more packages type pytest-xdist to search for the package. After you have finished typing the pytest-xdist a list of matching packages in the available package indicies will be listed. Under the PyPI repository tree selector there should be a pytest-xdist entry as below
To the top-right there is a Install button next to the version selector (which by default should have latest selected). Click this to install the package in your local TLOModel Python environment. If the package successfully installs you should see a dialog similar to the below appear
The starting point to add a new run configuration (or edit an existing one) is to open the Run/Debug Configurations dialog. This can be accessed from the Run menu on the main menu bar by going to Edit Configurations...
Clicking this menu entry will bring up a dialog something like the following
To add a new PyTest run configuration, select the ➕ (+) icon on the top left toolbar and scroll down throug the list to find the pytest entry under Python tests as illustrated below
On the right panel a form to specify the new run configuration will then be displayed as shown below
The Name: field should be set to a descriptive shorthand to allow easily identifying the particular run configuration from the list of all currently defined run configurations. Information you may wish to include is an indication of the test target directory or file (here tests to indicate all tests in the TLOModel/tests directory will be run) and any additional run options being used (in this case blank).
The Target: radio-button field should be set to select Script path and the immediately following text-entry field to the path to the directory or file containing the PyTest tests to run. For example, on Windows if you used the default location for you local working copy of the TLOModel repository the path to the top-level tests directory should be C:/Users/<username>/PyCharmProjects/TLOmodel/tests where <username> is your Windows username.
The Additional Arguments: text entry field can be used to pass additional arguments to pytest to alter the run configuration - see subsections below for example of how to use this field.
The Python interpreter: dropdown menu should be set to Project Default (Python 3.x (TLOmodel)) where 3.x is the specific Python version you are using in your local TLOmodel Python environment.
The Working directory: text entry field can be left blank or set to the path to the top-level tests directory in your local working copy of the TLOmodel repository.
To set up a configuration equivalent to that set up in the installation guide which runs all tests but (assuming pytest-xdist has been installed as described above) running across multiple processes in parallel, the optional -n argument added by pytest-xdist can be specifed in Additional Arguments field in the run configuration. The general pattern for setting the Additional Arguments field is -n <number of processes to use or 'auto'>. Specifying -n auto will use a number of processes equal to the number of CPU cores on your system, while specifying a specific positive integer value, for example -n 4 (as in the image below) will distribute the tests across the number of processes specified - that is four here.
Our PyTest configuration allows passing an optional --skip-slow argument to skip running any tests marked as slow with a @pytest.mark.slow decorator (the aim is to have any tests that take approximately more than 10s to complete to be marked like this). By adding --skip-slow to the Additional Arguments: field we can therefore create a run configuration that runs only the subset of faster running tests. Combined with using the -n argument to distribute the tests across multiple processes this can allow running a suite of tests with reasonable coverage in a relatively short time period that makes it more amenable to regularly running the tests locally as part of your development workflow.
To run all selected tests which take a seed argument specifying the seed to use for the simulation random number generator with multiple different seeds we can use the option --seed argument in our PyTest configuration.
The general pattern for setting the Additional Arguments: field is -n <number of processes to use or 'auto'> --seed <space delimited list of integer seeds> for example -n 4 --seed 822569775 2137449171 2671936806 3512589365 to run across four processes with seeds 822569775, 2137449171, 2671936806 and 3512589365. Unsuprisingly running tests with multiple different seeds will lead to a much larger overall run time so this sort of run configuration would be better suited to for example a run you do as a final check before pushing changes to GitHub rather than something to be run regularly while developing.
