Support applying pre-trained Segmentation models (Ilastik)

[ambrosejcarr]

Objective

As a user relying on a 3rd party automated segmentation tools, I want to assign spots to cells using a pre-trained model so that I can generate a spatially annotated gene expression matrix.

I want starfish to orchestrate this so that I can take advantage of starfish’s image processing before segmentation and track the provenance of my segmentation results.

Acceptance Criteria

• User can apply a pre-trained Ilastik “Pixel Classifier” to a slice of an image stack (e.g. DAPI) and get a Pixel Prediction Map
• User can apply a simple “maximum probability” threshold to a Pixel Prediction Map and get a binary segmentation mask
• User can apply a pre-trained Ilastik “Object Classifier” to a Pixel Prediction Map and get a labelled image
• User can specify the path to Ilastik in an environment variable or a configuration file
• A rendered tutorial exists in documentation demonstrating how to apply a pre-trained Pixel Classifier and Object Classifier, visualize the segmented cells, assign spots to cells, and generate a gene expression matrix
• Any intermediate files are cleaned up by starfish when segmentation is complete
• Starfish manages resources properly when calling Ilastik

Notes & reference material

• Prototype implementation, using ilastik to perform pixel classification and scikit-image to do labeling
• Ilastik’s Pixel Classification docs
• Ilastik’s Object Classification docs
• Ilastik docs on running headless
• Example of a similar implementation in CellProfiler

From @ambrosejcarr:

• Prototype API: starfish.image.Segment.IlasticPretrained(imagestack, path_to_ilastic_model, threshold, cell_label=0).
• An Ilastik Pixel Classification model will generate a probability map that is of shape (n, y, x), where n is the number of labels. One of the labels will be "cell", and we can default to picking the first one.
• To convert the probabilities from this model, one must threshold the probability field which is of (shape=(1, y, x)). This produces a binary image where cells are labeled as 1 and background 0.
• The binary image can be submitted to skimage.feature.label to extract the nuclei
• The resulting label image can be transformed into a set of masks with SegmentationMaskCollection.from_label_image

Validation

• User feedback on tutorial demonstrating loading and applying ilastik models
• User feedback on API docs for ilastik model

[kevinyamauchi]

This would be really useful for us. I think it would be nice if the code to convert a probability image to a cell label image wasSegmentationMaskCollection constructor we can input probability images from other segmentation pipelines.

[ambrosejcarr]

Cell Profiler's Ilastik tie-in is as follows: https://github.com/CellProfiler/CellProfiler/wiki/How-to-use-Pixel-Classification-in-CellProfiler

I interpret this as shelling out, not calling Ilastik directly from their python code, which may induce portability problems.