Cell Detection

⭐ Showcase

NeurIPS 22 Cell Segmentation Competition

https://openreview.net/forum?id=YtgRjBw-7GJ

Nuclei of U2OS cells in a chemical screen

https://bbbc.broadinstitute.org/BBBC039 (CC0)

P. vivax (malaria) infected human blood

https://bbbc.broadinstitute.org/BBBC041 (CC BY-NC-SA 3.0)

🛠 Install

Make sure you have PyTorch installed.

PyPI

pip install -U celldetection

GitHub

pip install git+https://github.com/FZJ-INM1-BDA/celldetection.git

💾 Trained models

model = cd.fetch_model(model_name, check_hash=True)

model name	training data
ginoro_CpnResNeXt101UNet-fbe875f1a3e5ce2c	BBBC039, BBBC038, Omnipose, Cellpose, Sartorius - Cell Instance Segmentation, Livecell, NeurIPS 22 CellSeg Challenge

Run a demo with a pretrained model

import torch, cv2, celldetection as cd
from skimage.data import coins
from matplotlib import pyplot as plt

# Load pretrained model
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = cd.fetch_model('ginoro_CpnResNeXt101UNet-fbe875f1a3e5ce2c', check_hash=True).to(device)
model.eval()

# Load input
img = coins()
img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
print(img.dtype, img.shape, (img.min(), img.max()))

# Run model
with torch.no_grad():
    x = cd.to_tensor(img, transpose=True, device=device, dtype=torch.float32)
    x = x / 255  # ensure 0..1 range
    x = x[None]  # add batch dimension: Tensor[3, h, w] -> Tensor[1, 3, h, w]
    y = model(x)

# Show results for each batch item
contours = y['contours']
for n in range(len(x)):
    cd.imshow_row(x[n], x[n], figsize=(16, 9), titles=('input', 'contours'))
    cd.plot_contours(contours[n])
    plt.show()

🔬 Architectures

import celldetection as cd

Contour Proposal Networks

• cd.models.CPN
• cd.models.CpnU22
• cd.models.CPNCore
• cd.models.CpnResUNet
• cd.models.CpnSlimU22
• cd.models.CpnWideU22
• cd.models.CpnResNet18FPN
• cd.models.CpnResNet34FPN
• cd.models.CpnResNet50FPN
• cd.models.CpnResNeXt50FPN
• cd.models.CpnResNet101FPN
• cd.models.CpnResNet152FPN
• cd.models.CpnResNet18UNet
• cd.models.CpnResNet34UNet
• cd.models.CpnResNet50UNet
• cd.models.CpnResNeXt101FPN
• cd.models.CpnResNeXt152FPN
• cd.models.CpnResNeXt50UNet
• cd.models.CpnResNet101UNet
• cd.models.CpnResNet152UNet
• cd.models.CpnResNeXt101UNet
• cd.models.CpnResNeXt152UNet
• cd.models.CpnWideResNet50FPN
• cd.models.CpnWideResNet101FPN
• cd.models.CpnMobileNetV3LargeFPN
• cd.models.CpnMobileNetV3SmallFPN

PyTorch Image Models (timm)

Also have a look at Timm Documentation.

import timm

timm.list_models(filter='*')  # explore available models

• cd.models.CpnTimmMaNet
• cd.models.CpnTimmUNet
• cd.models.TimmEncoder
• cd.models.TimmFPN
• cd.models.TimmMaNet
• cd.models.TimmUNet

Segmentation Models PyTorch (smp)

import segmentation_models_pytorch as smp

smp.encoders.get_encoder_names()  # explore available models

encoder = cd.models.SmpEncoder(encoder_name='mit_b5', pretrained='imagenet')

Find a list of Smp Encoders in the smp documentation.

• cd.models.CpnSmpMaNet
• cd.models.CpnSmpUNet
• cd.models.SmpEncoder
• cd.models.SmpFPN
• cd.models.SmpMaNet
• cd.models.SmpUNet

U-Nets

# U-Nets are available in 2D and 3D
import celldetection as cd

model = cd.models.ResNeXt50UNet(in_channels=3, out_channels=1, nd=3)

• cd.models.U22
• cd.models.U17
• cd.models.U12
• cd.models.UNet
• cd.models.WideU22
• cd.models.SlimU22
• cd.models.ResUNet
• cd.models.UNetEncoder
• cd.models.ResNet50UNet
• cd.models.ResNet18UNet
• cd.models.ResNet34UNet
• cd.models.ResNet152UNet
• cd.models.ResNet101UNet
• cd.models.ResNeXt50UNet
• cd.models.ResNeXt152UNet
• cd.models.ResNeXt101UNet
• cd.models.WideResNet50UNet
• cd.models.WideResNet101UNet
• cd.models.MobileNetV3SmallUNet
• cd.models.MobileNetV3LargeUNet

MA-Nets

# Many MA-Nets are available in 2D and 3D
import celldetection as cd

encoder = cd.models.ConvNeXtSmall(in_channels=3, nd=3)
model = cd.models.MaNet(encoder, out_channels=1, nd=3)

• cd.models.MaNet
• cd.models.SmpMaNet
• cd.models.TimmMaNet

Feature Pyramid Networks

• cd.models.FPN
• cd.models.ResNet18FPN
• cd.models.ResNet34FPN
• cd.models.ResNet50FPN
• cd.models.ResNeXt50FPN
• cd.models.ResNet101FPN
• cd.models.ResNet152FPN
• cd.models.ResNeXt101FPN
• cd.models.ResNeXt152FPN
• cd.models.WideResNet50FPN
• cd.models.WideResNet101FPN
• cd.models.MobileNetV3LargeFPN
• cd.models.MobileNetV3SmallFPN

ConvNeXt Networks

# ConvNeXt Networks are available in 2D and 3D
import celldetection as cd

model = cd.models.ConvNeXtSmall(in_channels=3, nd=3)

• cd.models.ConvNeXt
• cd.models.ConvNeXtTiny
• cd.models.ConvNeXtSmall
• cd.models.ConvNeXtBase
• cd.models.ConvNeXtLarge

Residual Networks

# Residual Networks are available in 2D and 3D
import celldetection as cd

model = cd.models.ResNet50(in_channels=3, nd=3)

• cd.models.ResNet18
• cd.models.ResNet34
• cd.models.ResNet50
• cd.models.ResNet101
• cd.models.ResNet152
• cd.models.WideResNet50_2
• cd.models.ResNeXt50_32x4d
• cd.models.WideResNet101_2
• cd.models.ResNeXt101_32x8d
• cd.models.ResNeXt152_32x8d

Mobile Networks

• cd.models.MobileNetV3Large
• cd.models.MobileNetV3Small

🤗 Hugging Face Spaces

Find us on Hugging Face and upload your own images for segmentation: https://huggingface.co/spaces/ericup/celldetection

Hugging Face API

Python

import requests

response = requests.post("https://ericup-celldetection.hf.space/run/predict", json={
    "data": [
        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACklEQVR4nGMAAQAABQABDQottAAAAABJRU5ErkJggg==",
        "ginoro_CpnResNeXt101UNet-fbe875f1a3e5ce2c",
    ]
}).json()

data = response["data"]

Javascript

const response = await fetch("https://ericup-celldetection.hf.space/run/predict", {
    method: "POST",
    headers: {"Content-Type": "application/json"},
    body: JSON.stringify({
        data: [
            "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACklEQVR4nGMAAQAABQABDQottAAAAABJRU5ErkJggg==",
            "ginoro_CpnResNeXt101UNet-fbe875f1a3e5ce2c",
        ]
    })
});

const data = await response.json();

🧑‍💻 Napari Plugin

Find our Napari Plugin here: https://github.com/FZJ-INM1-BDA/celldetection-napari
Find out more about Napari here: https://napari.org You can install it via pip:

pip install git+https://github.com/FZJ-INM1-BDA/celldetection-napari.git

🏆 Awards

• NeurIPS 2022 Cell Segmentation Challenge: Winner Finalist Award

📝 Citing

If you find this work useful, please consider giving a star ⭐️ and citation:

@article{UPSCHULTE2022102371,
    title = {Contour proposal networks for biomedical instance segmentation},
    journal = {Medical Image Analysis},
    volume = {77},
    pages = {102371},
    year = {2022},
    issn = {1361-8415},
    doi = {https://doi.org/10.1016/j.media.2022.102371},
    url = {https://www.sciencedirect.com/science/article/pii/S136184152200024X},
    author = {Eric Upschulte and Stefan Harmeling and Katrin Amunts and Timo Dickscheid},
    keywords = {Cell detection, Cell segmentation, Object detection, CPN},
}

🔗 Links

• Article (sciencedirect)
• PDF (sciencedirect)
• PyPI
• Documentation

🧑‍🔬 Thanks!