Adding custom models

What are pipelines?

In FastPathology, all integrated image analysis methods are implemented using so-called FAST text pipelines (FPLs). These files are simple descriptions of how the input data should be handled to produce and output, and how it should be presented to the reader, and stored on disk.

An example of such a pipeline can be seen below (all built-in pipelines lie in this directory):

PipelineName "Nuclei segmentation"
PipelineDescription "Patch-wise segmentation of cell nuclei at 20x magnification using U-Net"
PipelineInputData WSI "Whole-slide image"
PipelineOutputData Segmentation stitcher 0
Attribute classes "Background;Cell nuclei"

### Processing chain

ProcessObject tissueSeg TissueSegmentation
Input 0 WSI

ProcessObject patch PatchGenerator
Attribute patch-size 256 256
Attribute patch-magnification 20
Attribute patch-overlap 0.1
Input 0 WSI
Input 1 tissueSeg 0

ProcessObject network SegmentationNetwork
Attribute scale-factor 0.003921568627451
Attribute model "$CURRENT_PATH$/../models/high_res_nuclei_unet.onnx"
Input 0 patch 0

ProcessObject stitcher PatchStitcher
Input 0 network 0

### Renderers
Renderer imgRenderer ImagePyramidRenderer
Input 0 WSI

Renderer segRenderer SegmentationRenderer
Attribute border-opacity 0.5
Input 0 stitcher 0

This pipeline is used to perform semantic segmentation of cell nuceli in whole slide images. It applies a pretrained deep learning model across independent patches, and stitches them to form a segmentation image. The resulting segmentation is drawn by the SegmentationRenderer, as each patch is being processed. The result is also stored on disk by stating the following statement, at the top of the pipeline PipelineOutputData Segmentation stitcher 0.

Most computational pathology methods are so-called multi-input/output methods, which take some inputs (e.g., WSI) and produces some outputs (e.g., semantic segmentation or patch-wise classifications). An example of a patch-wise classification pipeline can be seen here. The pipeline is quite similar to the one above, with the exception of a HeatmapRenderer being used to display the result.

Note that a pipeline does not need to include a pretrained deep learning-based model. It could in-theory be anything, just as long as FAST supports it. Below is a simple pipeline which performs tissue segmentation based on a simple thresholding algorithm:

PipelineName "Tissue segmentation"
PipelineDescription "Tissue segmentation using simple color thresholding and morphology"
PipelineInputData WSI "Whole-slide image"
PipelineOutputData segmentation segmentation 0
Attribute classes "Background;Tissue"

ProcessObject segmentation TissueSegmentation
Input 0 WSI

### Renderers

Renderer imgRenderer ImagePyramidRenderer
Input 0 WSI

Renderer segRenderer SegmentationRenderer
Attribute opacity 0.2
Attribute border-opacity 0.5
Attribute label-colors "1" "red"
Input 0 segmentation 0

How can I add my own pipelines?

Similarly, you can also create your own custom pipelines, which can easily be integrated into the software. Simply go the the bottom of the Process section, and click the button "Add pipelines from disk". This prompts a file explorer, which you can use to find and select the pipeline(s) you wish to integrate. Note that if the pipeline uses a pretrained model, you will need to import that as well. Simply click the button below "Add models from disk", and choose which model(s) to import.

All pipelines and models will be added to a common location, such that FastPathology can find them during runtime, and knows where a specific pipeline and its corresponding (models) lie.

How can I customize a pipeline?

You can easily change the integrated pipelines as you please using the built-in Script Editor (see here) for more information.