Skip to content

Scenario workflow

Jump to bottom
Asif Tamuri edited this page Oct 20, 2020 · 1 revision

Running simulations in the cloud

Breakdown of simulations:

  • A scenario is essentially an analysis script, where all of the boilerplate is handled by the parent BaseScenario class
  • Draw: Set of parameter values that are overridden in the simulation, from a python-defined distribution
    • 100s sets of draws for override parameters
    • draw index: enumeration of the draw set
  • Sample: an individual run of a simulation, each simulation being set with a different seed
    • 1000s of seeds per draw

Suggested flow of work

Local:

  • Create a Scenario class
    • inheriting from BaseScenario, which handles all of the setup and running of the simulation
    • Set up configuration (start date, pop size etc.)
    • Define registered modules and their configuration
    • Define overriding parameters for modules which are drawn from a distribution (a set of these is a draw)
  • Can run this script directly using pycharm at this point
    • if __name__ == '__main__': block sets up and runs the scenario class
    • can adjust population and duration for local running in this block
  • Ensure that parameter draws look reasonable
    • Using TLO command line interface, create draw configuration files
    • Optional: run a single sample from a draw configuration file.
  • Commit final changes and push branch

Optional on azure:

  • Log into dedicate compute node via ssh (similar to myriad or any other cluster)
  • Pull branch with work on it
  • Using tlo command line too, create draw configuration files
  • Run a single sample from a draw configuration file

Local: submit to azure

  • Use TLO command line interface to submit job to azure
    • python script that contains the scenario class
    • branch to use (defaults to master)
    • commit id to use (defaults to latest commit)
    • scenario seed 5
    • number of draws 100
    • samples to run 1-1000
  • Job sucessfully submitted message

Using BatchExplorer application

  • login and view status of nodes
  • Check status of nodes by looking at the stdout.txt file on a node
  • Job failure: check status or automated email?
  • All samples complete:
    • All data pooled together - zip of dataframes for each sample or combining of dataframes?
    • Check status on azure batch? email?
  • Download pooled data using BatchExplorer to local machine
    • Carry out analysis by reading in pickle files

Worked example

Example scenario script:

Setup and running of simulation is all handled within the BaseScenario.

from numbers import Real
from typing import Dict, List, Type, Union

from tlo import BaseScenario, Date, Module, logging
from tlo.methods import (
    contraception,
    demography,
    enhanced_lifestyle,
    healthseekingbehaviour,
    healthsystem,
    labour,
    pregnancy_supervisor,
    symptommanager,
)


class EnhancedLifestyleScenario(BaseScenario):
    def __init__(self, scenario_seed: int = 1):
        """
        Example scenario setting up all expected custom configuration for its simulations.
        :param scenario_seed: seed for the scenario as a whole
        """
        # initialise base scenario
        super().__init__(scenario_seed)

        # Set custom data
        self.start_date = Date(2010, 1, 1)
        self.end_date = Date(2050, 1, 1)
        self.pop_size = 1_000_000
        self.log_config['custom_levels'] = {
            # Customise the output of specific loggers. They are applied in order:
            "*": logging.CRITICAL,
            "tlo.methods.demography": logging.INFO,
            "tlo.methods.enhanced_lifestyle": logging.INFO
        }

    def draw_parameters(self) -> Dict[Type[Module], Dict[str, Union[Real, List[Real]]]]:
        """
        Creates dictionary that defines overriding parameters for specific modules in the form:
            {tlo.methods.DiseaseModule: {'parameter_1': parameter_value_1, 'parameter_2': parameter_value_2}}.

        Parameters which are not manually set to one value should use self.rng to draw from a distribution.

        :return: dictionary that instructs how to override parameters
        """
        return {
            demography.Demography: {
                'fraction_of_births_male': self.rng.randint(480, 500) / 1000
            },
            contraception.Contraception: {
                'r_init_year': 0.125,
                'r_discont_year': self.rng.exponential(0.1),
            },
        }

    def get_modules(self) -> List[Module]:
        """
        Creates list of modules to be registered in the simulation.
        For the resources at 'TLOmodel/resources' use self.resources

        :return: list of modules to be registered in the simulation.
        """
        # Used to configure health system behaviour
        service_availability = ["*"]

        # list of all modules
        modules = [
            demography.Demography(resourcefilepath=self.resources),
            enhanced_lifestyle.Lifestyle(resourcefilepath=self.resources),
            healthsystem.HealthSystem(resourcefilepath=self.resources,
                                      disable=True,
                                      service_availability=service_availability),
            symptommanager.SymptomManager(resourcefilepath=self.resources),
            healthseekingbehaviour.HealthSeekingBehaviour(resourcefilepath=self.resources),
            contraception.Contraception(resourcefilepath=self.resources),
            labour.Labour(resourcefilepath=self.resources),
            pregnancy_supervisor.PregnancySupervisor(resourcefilepath=self.resources),
        ]
        return modules


if __name__ == '__main__':
    # for running simulation locally only
    scenario = EnhancedLifestyleScenario()
    # For local testing, use shorter time and smaller population
    scenario.end_date = Date(2015, 1, 1)
    sceanrio.pop_size = 1_000
    # run the scenario
    output = scenario.run()
    print(output)

Interface for interacting with scenario scripts

We create sample metadata

tlo create-samples enhanced_lifestyle_scenario.py --seed 5 --draws 100

Draw configuration files follow pattern outputs/{scenario file name}/draw_{draw_index}/config.json

example sample metadata file: enhanced_lifestyle_scenario/draw_3/config.json

{
    "scenario_seed": 5,
    "path": "scripts/enhanced_lifestyle_analyses/enhanced_lifestyle_scenario.py",
    "draw": 3,
    "override_parameters": {
        "tlo.methods.demography.Demography": {
            "fraction_of_births_male": 0.509
        },
        "tlo.methods.contraception.Contraception": {
            "r_init_year": 0.125,
            "r_discont_year": 0.0670234079717601
        }
    }
}

You can then run a specific draw and sample, reading in the metadata json and then running the simulation with this data.

tlo run-samples enhanced_lifestyle_scenario/draw_1/config.json --samples 1-1000

Running on Azure

After you are happy with a local run, you commit the scenario python file and push this to your branch.

You can log in to a dedicated node in azure, pull your branch, generate samples and run one to make sure that this is working correctly.

When you are ready to run an entire scenario use the tlo CLI:

tlo run-azure-scenario contraception_example.py --seed 70 --draws 100 --samples 1-1000 --branch stef/testing --commit-id 8b71b5be5f293387eb270cffe5a0925b0d97830f (if no branch, master is used, if no commit id, latest commit is used)

This uses the configuration data in your repository to:

  • create a job for the scenario
  • on startup
    • checkout the correct branch and commit
    • run tlo create-samples with the seed and draws
  • each node is assigned a task, or series of tasks (if we want a maximum number of nodes) to run an individual sample by the path to the json file
  • After all tasks are complete, postprocessing task pools/zips the sample json files and output data frames

TLO Model Wiki

Clone this wiki locally