napari-aiSEGcell

A napari plugin to segment cell nuclei and whole cells in bright field microscopy images. napari-aisegcell uses aisegcell for segmentation. This plugin can only be used to use already trained models for segmentation. If you want to train a new model use aisegcell. Please cite this paper if you are using this plugin in your research.

---

This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

Contents

• Installation
  • Command line
  • One-click
• Data
• Documentation
  • First launch
  • Launch napari
  • Layer mode
  • Batch mode
  • Trained models
• Image annotation tools
• Citation
• Contributing
• License
• Issues

Installation

There are two ways to install napari-aisegcell: First, you can install napari-aisegcell from command line. Second, you have napari already installed as a graphical user interface (GUI) and install napari-aisegcell from the GUI menu.

We recommend the command line installation as it provides fine-grained control of the installation to prevent conflicts with existing napari plugins. Use the one-click installation if you do not want to concern yourself with virtual environments or the command line. Just be aware that using the one-click installation may introduce conflicts with already installed plugins or new plugin installations may disrupt this plugin.

Installing napari-aisegcell requires ~5 min and was tested with

OS = macOS 12.6.3/ubuntu 22.10/windows 10
python = 3.8.6
torch = 1.10.2
torchvision = 0.11.3
pytorch-lightning = 1.5.9

Command line

napari must be installed from command line to install napari aisegcell from command line. Installation requires a command line application (e.g. Terminal) with git and python installed. If you do not have python installed already, we recommend installing it using the Anaconda distribution. If you operate on Windows we recommend using Anaconda Powershell Prompt as command line application. An introductory tutorial on how to use git and GitHub can be found here.

1. (Optional) If you already installed napari in a virtual environment you can skip this step. However, you may want to install napari-aisegcell in a fresh environment to avoid conflicts with existing plugins. Create a new virtual environment for napari. Here is a list of different python virtual environment tools. Open your command line application and create a (e.g. conda) virtual environment

   conda create -n napari python=3.8

2. Activate your virtual environment that has napari installed or you want to install napari to

   conda activate napari

3. (Optional) Install napari. Skip this step if you have napari already installed.

   pip install "napari[all]"

4. (Optional) If you use Anaconda Powershell Prompt, install git through conda

   conda install -c anaconda git

5. Install napari-aisegcell

   1. from PyPI

   pip install napari-aisegcell

   2. from GitHub (= latest development version)

   pip install git+https://github.com/CSDGroup/napari-aisegcell.git

With step 4) completed you have successfully installed napari-aisegcell. You can proceed with the documentation on how to use napari-aisegcell. NOTE, that when opening the plugin for the first time, the remaining dependencies (torch, torchvision, pytorch-lightning) will be automatically installed via light-the-torch. If you prefer to manually install the remaining dependencies (i.e. prevent potential interference with your virtual environment), proceed with step 5).

5. (Optional) GPUs greatly speed up training and inference of aisegcell and are available for torch (v1.10.2) for Windows and Linux. Check if your GPU(s) are CUDA compatible (Windows, Linux) and update their drivers if necessary.

6. (Optional) Install torch/torchvision compatible with your system. aisegcell was tested with torch version 1.10.2, torchvision version 0.11.3, and cuda version 11.3.1. Depending on your OS, your CPU or GPU (and CUDA version) the installation may change

# Windows/Linux CPU
pip install torch==1.10.2+cpu torchvision==0.11.3+cpu -f https://download.pytorch.org/whl/cpu/torch_stable.html

# Windows/Linux GPU (CUDA 11.3.X)
pip install torch==1.10.2+cu113 torchvision==0.11.3+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

# macOS CPU
pip install torch==1.10.2 torchvision==0.11.3

7. (Optional) Install pytorch-lightning. aisegcell was tested with version 1.5.9.

# note the installation of v1.5.9 does not use pip install lightning
pip install pytorch-lightning==1.5.9

One-click

(NOT YET AVAILABLE) Using the one-click installation of napari-aisegcell is as easy as opening napari, selecting Plugins>Install/Uninstall Plugins... and searching for napari-aisegcell in the search bar. Select install and restart napari for napari-aisegcell to appear in the list of installed plugins in the Plugins menu. Please recall that one-click installing napari-aisegcell may interfere with existing plugin installations or new plugin installations may interfere with the napari-aisegcell plugin.

Data

napari-aisegcell is currently intended for single-class semantic and instance segmentation. Input images are expected to be 8-bit or 16-bit greyscale images. Segmentation masks are expected to decode background as 0 intensity and all intensities >0 are converted to a single intensity value (255). Have a look at this notebook for a data example.

Documentation

napari-aisegcell has two modes: The layer mode and the batch mode. Layer mode is intended to explore if existing trained models in napari-aisegcell are suitable for your data. Batch mode is intended for high-throughput image segmentation once you have confirmed that existing models are well suited for your data.

First launch

The first time you launch napari-aisegcell, torch, torchvision, pytorch-lightning will be automatically installed if you have skipped steps 5)-7) of the installation. The napari window will freeze during download and installation. Depending on your setup this may take several minutes (~GBs of download). Similarly, the first time you are running a pre-trained model the model weights (.CKPT file) will be downloaded and will delay the prediction (~MBs of download).

Launch napari

The napari bundled app is launched by opening the desktop shortcut. To launch napari as a command line application

1. Open the terminal and activate the environment you installed napari-aisegcell into

   conda activate napari

2. Run napari in terminal

   napari

Layer mode

Open the layer mode in the menu Plugins>napari-aisegcell>Layer mode. Select the parameters you want to use to obtain your segmentation and select the Run button.

Data section

In the Data section (magenta) you can select the images you want to segment from a drop-down menu. Only images that are loaded in the napari layer tab are available for selection. Image formats must be readable by skimage.io.imread.

Neural network section

In the neural network section (cyan) you can select the neural net you want to use for segmentation. The section has three parameters

• Model type
  • NucSeg: Select this option to use a pre-trained aisegcell model to segment nuclei (see trained models).
  • CellSeg: Select this option to use a pre-trained aisegcell model to segment whole cells (see trained models).
  • Custom: Select this option if you want to load a aisegcell model that does not ship with napari-aisegcell. Custom models can be obtained by training your own aisegcell model or obtaining aisegcell models from 3rd parties. You must select the checkpoint (.ckpt) file in the emerging Custom Model parameter.
• Pre-trained Model: Drop-down menu to select the available pre-trained models for NucSeg or CellSeg.
  • Bright Field: a model to segment nuclei/whole cells in bright field. Currently, no other image modalities are available.
• Computing Device: Drop-down menu that lists computing devices (CPU/GPUs) available with your current torch installation.

Post-processing section

In the post-processing section (green) a selection of common post-processing steps are available

• Instance Segmentation: Check this box to return instance segmentations instead of semantic segmentations.
• Remove Holes <: Removes holes in objects (e.g. nuclei) <X pixels.
• Minimum object size: Removes objects of size <X pixels before the dilation step.
• Maximum object size: Removes objects of size >X pixels before the dilation step.
• Dilation: Dilate (>0) or erode (<0) objects by X pixels (see here).

Batch mode

Open the batch mode in the menu Plugins>napari-aisegcell>Batch mode. Select the parameters you want to use to obtain your segmentations and select the Run button.

The neural network section and post-processing-section are the same as in the layer mode.

Data section

In the data section (magenta) you can select the images you want to submit for batch processing.

• Input
  • Select file: Select an existing input CSV file that follows the aisegcell input format
  • Create file: Create a CSV file that follows the aisegcell input format
    • Input directory: Select parent directory containing all images to segment. Images can be in subdirectories.
    • File pattern: All files matching this pattern in your selected Input directory will be stored in the CSV file. Use wildcard characters like * to capture all images you want to segment in one run.
      • Example 1 */*.png: will select all PNG files in in all sub-directories of Input directory.
      • Example 2 position*z1.png: will select all files in Input directory that start with "position" and end with "z1.png"
    • Save as: Name (and path) of the file that will be the input to your selected Neural Network. The file can be used as input to Select file.
    • Mask suffix: Mask suffix that will be appended to each mask file name.
      • Example suffix = "_mask": my_image.png -> my_image_mask.png
    • Output:
      • Directory: Directory to which all segmentation masks will be saved. Be aware that input images with identical file names will be appended with Mask suffix AND an ID.
      • CSD format: Store segmentation masks following the storage system of the Cell Systems Dynamics group. CSD format finds the deepest common directory of all input images, creates folders 'Analysis/Segmentation_YYMMDD', and reconstructs the unique parts of all input paths in "Segmentation_YYMMDD".

Trained models

We provide trained models:

modality	image format	model	example image	description	availability
bright field nucleus segmentation	2D grayscale	U-Net		Trained on a data set (link to data set) of 9849 images (~620k nuclei).	ETH Research Collection
bright field whole cell segmentation	2D grayscale	U-Net		Trained on a data set (link to data set) of 224 images (~12k cells).	ETH Research Collection

Image annotation tools

Available tools to annotate segmentations include:

• napari
• Labkit for Fiji
• QuPath
• ilastik

Citation

t.b.d.

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the BSD-3 license, "napari-aisegcell" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.