How to create a Parameter Handler

The Parameter Handler has the responsibility to apply the Scenario Parameters values to the Datasets which will be used by the engine.

It is run during the ApplyingScenarioParameters step of the ScenarioRun Life Cycle.

Please read first this general how-to about Solution Run Template Handlers.

Python project

A Parameter Handler is a Python 3 project stored by default in code/run_templates/%CSM_RUNTEMPLATE_ID%/parameters_handler/. The storage place can be set in the Solution.yaml definition.

The Parameter Handler must contains at least a main.py file which is the entry point called by the Cosmo Tech Platform.

It can define to a requirements.txt file to define specific dependancies.

Apart from this considerations the Python project can be organize the way you want.

The main.py Python script will be executed in an environment definining the followinf Env Vars:

AZURE_TENANT_ID="12345678"
AZURE_CLIENT_ID="12345678"
AZURE_CLIENT_SECRET=azertyuiop
CSM_API_URL=https://api.comostech.com
CSM_API_TOKEN=azertyuiopqsdfghjklm
CSM_PARAMETERS_HANDLER_PROVIDER=local
CSM_PARAMETERS_HANDLER_PATH=run_templates/hundred/parameters_handler/
CSM_DATASET_ABSOLUTE_PATH=/mnt/scenariorun-data
CSM_PARAMETERS_ABSOLUTE_PATH=/mnt/scenariorun-parameters

The goal of a Parameter Handler project is to modify inplace the Dataset files in $CSM_DATASET_ABSOLUTE_PATH with the information of the input parameters in $CSM_PARAMETERS_ABSOLUTE_PATH.

Dataset files

Dataset files in $CSM_DATASET_ABSOLUTE_PATH are CSV files compatible with the CosML loaders. Each Dataset stores its files in $CSM_DATASET_ABSOLUTE_PATH.

Input parameters parameters.json file

Input parameters values are stored in a json file in $CSM_PARAMETERS_ABSOLUTE_PATH/parameters.json has the following structure:

parametersValues:
  type: array
  description: the list of Solution Run Template parameters values
  items:
    type: object
    description: the value of Analysis parameter for a Scenario for this ScenarioRun
    properties:
      parameterId:
        type: string
        description: the parameter Id
      varType:
        type: string
        readOnly: true
        description: the parameter value type
      value:
        type: string
        description: the parameter value

Input parameters values example file:

[{"parameterId": "prefix", "varType": "string", "value": "new"}]

Parameters referencing Datasets

Some input parameters have the special varType %DATASETID%. In this case the Dataset has been previously fetch in a directory in $CSM_PARAMETERS_ABSOLUTE_PATH/{%PARAMETERID%}.

Packaging

If the Parameter Handler project is stored in local default path code/run_templates/%CSM_RUNTEMPLATE_ID%/parameters_handler/ within the Project Solution, it will be automatically package in the Solution container image when running:

csm twin-engine docker release [options]

If the Parameter Handler project is stored in a remote Storage you must upload it with the appropriate storage tools and set its URI in the Solution.yaml file.

Don't forget to put a requirements.txt file for every dependency your Project needs.

Example script

import os
from pathlib import Path
import json
from types import SimpleNamespace
import logging
import csv
from tempfile import NamedTemporaryFile
import shutil

def getValue(values, searchId):
    for v in values:
        if searchId == v.parameterId:
            return v
    logging.warning("Parameter not found")

dataPath = Path(os.environ["CSM_DATASET_ABSOLUTE_PATH"])
customerPath = dataPath / "BE_Customer.csv"
roomPath = dataPath / "CustomerRoom.csv"
parametersPath = Path(os.environ["CSM_PARAMETERS_ABSOLUTE_PATH"])
parametersFile = parametersPath / "parameters.json"

with open(parametersFile, 'r') as f:
    parameters = json.load(f, object_hook=lambda d: SimpleNamespace(**d))

pValues = parameters.parametersValues

prefixParam = getValue(pValues, "prefix")
prefix = prefixParam.value
tempfile = NamedTemporaryFile('w+t', newline='', delete=False)

with open(customerPath, newline='') as csvfile:
    customerReader = csv.reader(csvfile)
    customerWriter = csv.writer(tempfile)
    line = 0
    for row in customerReader:
        if line > 0:
            name = row[1]
            newname = prefix + name
            print(name + "=>" + newname)
            row[1] = newname
        customerWriter.writerow(row)
        line = line + 1
shutil.move(tempfile.name, customerPath)

tempfile = NamedTemporaryFile('w+t', newline='', delete=False)
with open(roomPath, newline='') as csvfile:
    roomReader = csv.reader(csvfile)
    roomWriter = csv.writer(tempfile)
    line = 0
    for row in roomReader:
        if line > 0:
            left = row[0]
            newleft = prefix + left
            row[0] = newleft
            right = row[1]
            newright = prefix + right
            row[1] = newright
        roomWriter.writerow(row)
        line = line + 1
shutil.move(tempfile.name, roomPath)