transformify-plugins · sofroniewn · Sep 5, 2019 · Sep 5, 2019 · Sep 5, 2019 · Sep 5, 2019
diff --git a/README.md b/README.md
@@ -1,5 +1,139 @@
-# segmentify
+# Segmentify
 
-Python image segmentation plugin
+Segmentify is an interactive and general purpose cell segmentation plugin for the multi-dimentional image viewer [Napari](https://github.com/napari/napari).
+
+ <p align=center"><img src="figs/intro.gif" /></p>
+
+In the example above, the user is using segmentify to segment out the nucleus, cytoplasm and the background for all the cells in the image. The user uses the paint brush tool to label some training examples on the **trian layer**. The entire image is featurized using a pretrained featurizer, and the selected examples are used to train a Random Forest Classifier. Lastly, the trained classifier is used to predict the label for all the remaining unlabeled pixels, and the segmentation output is displayed at the **output layer**. 
+
+## Installation
+
+Segmentify can be installed using **pip**
+
+```
+pip install segmentify
+```
+
+## Launching Segmentify Viewer
+
+Segmentify's viewer can be launched either using the command line interface or a python script.
+
+
+### Segmentify Command Line Interface
+
+The Segmentify Viewer can be launched from the command line by entering:
+
+```
+segmentify <path to your images>
+```
+
+### Segmentify Script
+
+To open Segmentify's Viewer from a python script, enter the following scripts:
+
+```python
+from segmentify import Viewer, gui_qt, util
+
+img = util.parse_img(<path to your image>)
+
+with gui_qt():
+    viewer = Viewer(img)
+```
+
+An example can be found [here](./examples/viewer_example.py)
+
+## Featurization
+
+Segmentify works by featurizing input images using either pretrained UNet models or some classical image filters. Users can then use the paint brush to label some of the pixels to train a Random Forest Classifier. The trained classifier is used to predict the label for all remaining unlabeled pixels. The training labels should be provided in the **train layer** and the segmentation will be displayed on the **output layer**. Segmentify includes the following featurization strategies:
+- HPA\_4: Trained by decomposing images from the [Human Protein Atlas](https://www.kaggle.com/c/human-protein-atlas-image-classification) into Nucleus, ER, Protein and Cytoplasm
+- HPA\_3: Trained by decomposing images from the [Human Protein Atlas](https://www.kaggle.com/c/human-protein-atlas-image-classification) into Nucleus, ER, Cytoplasm
+- HPA: Trained by segmenting out the nucleus in images from the [Human Protein Atlas](https://www.kaggle.com/c/human-protein-atlas-image-classification)
+
+- Nuclei: Trained by segmenting out the nucleus in images from the [Kaggle Data Science Bowl 2018](https://www.kaggle.com/c/data-science-bowl-2018/overview)
+- Filter: Combining several classical image filter (Gaussian, Sobel, Laplace, Gabor, Canny)
+
+For more information about the training strategies for each featurizer, please refer to the notebooks [here](https://github.com/marshuang80/CellSegmentation/tree/master/notebooks)
+
+### Training your own featurizer
+
+Users can also train their own image featurizers using [Cell Segmentation](https://github.com/marshuang80/CellSegmentation). The trained model should be placed in the *./segmentify/model/saved_model* directory. 
+
+## Key Bindings
+
+The following is a list of Segmentify supported functions along with their key bindings:
+- Segmentation (Shift-S)
+- Uncertainty Heatmap (Shift-H)
+- Next Featurizer (Shif-N)
+- Dilation (Shift-D)
+- Erosion (Shift-E)
+- Close (Shift-C)
+- Open (Shift-O)
+- Fill holes (Shift-F)
+- Next featurizer (Shift-N)
+
+> For all image morphology steps (dilation, erosion, close, open, fill holes), **the operations will only be applied to the selected label for the selected layer**. For example, if you want the blue labels in the segmented images to dilate, you will have to click on the *output layer* and increment to the desired label. 
+
+| Morphology on labels | Morphology on segmentation |                            
+| --- | --- |
+| ![](figs/label_morph.gif) | ![](figs/seg_morph.gif) |
+
+
+### Segmentation (Shift-S)
+
+Some labeled training examples must be provided before segmentify can segment the input images. The training labels can be provided by using the paint brush on the left-hand side of the Viewer to paint on the image. Please make that you are incrementing the labels in the **train** layer. 
+
+![](figs/seg_example.gif)
+
+The segmented output can be found on the **output** layer.
+
+### Uncertainty Heatmap (Shift-H)
+
+The Segmentify Viewer can also output a heatmap of areas where the model is having low confidence in it's segmentation. Users can relabel these areas to help improve the model's prediction. The uncertainty heatmap is generated by calculating the normalized entropy for the prediction probabilities. Similar prediction probability for all classes indicates that the model does not have confidence in one particular class, which results in low entropy; whereas high probability for one particular class results in high entropy. Using normalized entropy allow us to generate the uncertainty heatmap even when there are many segmentation classes. The lower the entropy for a pixel (high uncertainty), the brighter it will appear in red on the heatmap. 
+
+To show the uncertainty heatmap, set the heatmap parameter to true when defining the Viewer:
+
+```python
+Viewer(imgs, heatmap=True)
+```
+
+If you are using the Command Line Interface, set the heatmap flag to True:
+
+```
+segmentify <path to your image> --heatmap True
+```
+
+![](figs/heatmap.gif)
+
+
+### Next Featurizer (Shift-N)
+
+This feature cycles through all the featurizers in *segmentify/model/saved_model* as well as the filter featurizer. The name of the selected featurizer is shonw in the bottom left corner of the viewer. Note that after switching to the next featurizer, the user still needs to press Shift-S to re-segment the image. 
+
+![](figs/next.gif)
+
+### Dilation (Shift-D)
+
+The [dilation operation](https://homepages.inf.ed.ac.uk/rbf/HIPR2/dilate.htm) expands all connected components with the selected label. 
+
+![](figs/dilation.gif)
+
+### Erosion (Shift-E)
+
+The [erosion operation](https://homepages.inf.ed.ac.uk/rbf/HIPR2/erode.htm) shrinks all connected components with the selected label. 
+
+![](figs/erosion.gif)
+
+### Fill Holes (Shift-F)
+
+This operation fills holes within a connected component for the selected label.
+
+![](figs/fill_holes.gif)
+
+### Close (Shift-C)
+
+The [close operation](https://homepages.inf.ed.ac.uk/rbf/HIPR2/close.htm)is done by applying dilation on the connected components with the selected label, following by an erosion. 
+
+### Open (Shift-O)
+
+The [open operation](https://homepages.inf.ed.ac.uk/rbf/HIPR2/open.htm) is done by applying erosion on the connected components with the selected label, following by an dilation. 
 
-#### **Development Status :: Pre-Alpha**
diff --git a/examples/image_stack.tif b/examples/image_stack.tif
diff --git a/examples/instance.py b/examples/instance.py
diff --git a/examples/nuclei.png b/examples/nuclei.png
diff --git a/examples/semantic.py b/examples/semantic.py
diff --git a/examples/semantic_3d.py b/examples/semantic_3d.py
diff --git a/examples/viewer_example.py b/examples/viewer_example.py
@@ -0,0 +1,10 @@
+import os
+from segmentify import Viewer, gui_qt, util
+
+# parse input file
+example_file = os.path.join(os.path.abspath(os.path.dirname(__file__)), "hpa.png")
+img = util.parse_img(example_file)
+
+with gui_qt():
+    viewer = Viewer(img)
+
diff --git a/examples/watershed.py b/examples/watershed.py
diff --git a/figs/dilation.gif b/figs/dilation.gif
diff --git a/figs/erosion.gif b/figs/erosion.gif
diff --git a/figs/fill_holes.gif b/figs/fill_holes.gif
diff --git a/figs/heatmap.gif b/figs/heatmap.gif
diff --git a/figs/intro.gif b/figs/intro.gif
diff --git a/figs/label_morph.gif b/figs/label_morph.gif
diff --git a/figs/next.gif b/figs/next.gif
diff --git a/figs/seg_example.gif b/figs/seg_example.gif
diff --git a/figs/seg_morph.gif b/figs/seg_morph.gif
diff --git a/requirements.txt b/requirements.txt
@@ -1,5 +1,11 @@
-napari >= 0.0.8
+imageio>=2.5.0
+napari>=0.1.4
+numba>=0.45.1
 numpy>=1.10.0
 scipy>=1.2.0
 scikit-image>=0.15.0
 scikit-learn>=0.21.2
+vispy>=0.6.1
+torch>=1.2.0
+torchvision>=1.2.0
+
diff --git a/segmentify/__init__.py b/segmentify/__init__.py
@@ -1 +1,5 @@
 __version__ = '0.0.0'
+
+from .viewer import Viewer
+from . import util
+from napari import gui_qt
diff --git a/segmentify/main.py b/segmentify/main.py
@@ -0,0 +1,32 @@
+"""segmentify command line viewer.
+"""
+
+import argparse
+import numpy  as np
+import imageio
+from napari import gui_qt
+from segmentify import Viewer, util
+
+
+def main(args):
+    """The main Command Line Interface for Segmentify"""
+
+    # parse in images
+    imgs = [util.parse_img(img) for img in args.images]
+
+    if len(imgs) > 1:
+        imgs = np.stack(imgs, axis=0)
+    else:
+        imgs = np.array(imgs)
+
+    with gui_qt():
+        viewer = Viewer(imgs, heatmap=args.heatmap)
+
+
+if __name__ == "__main__":
+    # parser
+    parser = argparse.ArgumentParser()
+    parser.add_argument("images", nargs="*", type=str, help="Image to view and segment.")
+    args = parser.parse_args()
+
+    main(args)
diff --git a/...entify/model/saved_model/UNet_hpa_max.pth → segmentify/model/saved_model/HPA.pth b/...entify/model/saved_model/UNet_hpa_max.pth → segmentify/model/saved_model/HPA.pth
diff --git a/segmentify/model/saved_model/HPA_3.pth b/segmentify/model/saved_model/HPA_3.pth
diff --git a/segmentify/model/saved_model/HPA_4.pth b/segmentify/model/saved_model/HPA_4.pth
diff --git a/segmentify/model/saved_model/UNet_nuclei.pth → segmentify/model/saved_model/Nuclei.pth b/segmentify/model/saved_model/UNet_nuclei.pth → segmentify/model/saved_model/Nuclei.pth