Importing External Transformations

MINT-Transformation is capable of quickly importing third-party Python transformation libraries into our system. In order to be imported, the provided libraries should follow the requirements below:

Requirements:

• A Python file that contains:
  • Main function
  • Function description using docstring (descriptions of input and output should be included)
  • Input variables with data types (using Python 3 typing)
  • Returned variables should be output files
• A requirements.txt file that contains all of the package dependencies of the transformation function

Example:

An example main function can be written as follows:

def main(in_file: Path, out_file: Path, out_bounds: string, out_xres_sec: float, out_yres_sec: float):
    """
    Example main function for Topoflow transformation
    :param in_file: Input file
    :param out_file: Output file
    :param out_bounds: bounding box
    :param out_xres_sec: x resolution
    :param out_yres_sec: y resolution
    """
    regrid.regrid_geotiff(
        in_file=in_file,
        out_file=out_file,
        out_bounds=out_bounds,
        out_xres_sec=out_xres_sec,
        out_yres_sec=out_yres_sec,
        RESAMPLE_ALGO="bilinear",
        REPORT=True,
    )
    ...
    return out_file

Notebook Instruction:

If transformations are written in Jupyter notebooks, run the following commands to convert the notebook to Python script:

jupyter nbconvert [notebook_file] --to python 

Then write a main (wrapper) function and follow the requirements above.

Pipeline Construction (Advanced):

Using the transformation functions defined externally, users can a wrapper transformation adapters as follows (We use the previous function as our walkthrough example):

• Create a Python class that inherits dtran.ifunc.IFunc
• Specify the class variables id and description

id = "test_func"
description = "An playtest adapter that does nothing"

• Specify the class variable inputs using the main function parameters and types

inputs = {
  "infile": ArgType.FilePath,
  "outfile": ArgType.FilePath,
  ...
  "out_yres_sec": ArgType.Number
}

• Specify the class variable outputs using the main returning variable

outputs = {
  "outfile": ArgType.FilePath
}

• Define the class __init__() method with the same signature as the transformation function and assign the instance properties

def __init__(
        self,
        in_file,
        out_file,
        out_bounds,
        out_xres_sec,
        out_yres_sec,
    )
    self.in_file = in_file
    self.out_file = out_file
    ...
    self.out_yres_sec = out_yres_sec

• Define the class exec() method that calls the original function

def exec(self):
   return main(in_file=self.in_file, out_file=self.out_file, self.out_bounds, self.out_xres_sec, self.out_yres_sec)