Skip to content
guijacquemet edited this page Sep 10, 2024 · 245 revisions

ZeroCostDL4Mic - What is it?

⚠️WARNING⚠️, 🔴IMPORTANT❗🔴 There seems to be an issue with GPU allocation for all notebooks using Tensorflow below the 2.5 version. We are investigating the issue.

Overview

ZeroCostDL4Mic is a toolbox for the training and implementation of common Deep Learning approaches to microscopy imaging. It exploits the ease of use and access to GPU provided by Google Colab.

Training data can be uploaded to Google Drive, which can be used to train models using the provided Colab notebooks in a web browser. Inference (predictions) on unseen data can then also be performed within the same notebook, therefore not requiring any local hardware or software set-up.

Want to see a short video demonstration?

Running a ZeroCostDL4Mic notebook Example data in ZeroCostDL4Mic Romain's talk @ Aurox conference Talk @ SPAOM NEUBIAS webminar

Implemented networks

ZeroCostDL4Mic provides fully annotated Google Colab optimized Jupyter Notebooks for popular pre-existing networks. These cover a range of important image analysis tasks (e.g., segmentation, denoising, restoration, label-free prediction). There are 3 types of implemented networks:

We welcome network contributions from the research community. If you wish to contribute, please read our guidelines first.

How to get the notebooks and test datasets?

Both fully supported and beta-testing versions of the individual notebooks can be directly opened from GitHub into Colab by clicking one of the respective links in the table below. You will need to create a local copy to your Google Drive in order to save and modify the notebooks. Once opened in Colab, follow the instructions described in the specific notebook that you selected to install the relevant packages, load the training dataset, train, check on test datasets and perform inference and predictions on unseen data.

With the exception of the U-net training data, we provide training and test datasets that were generated by our labs. These can be downloaded from Zenodo using the various links below. The U-net data was obtained from the ISBI segmentation contest.

⚠️WARNING⚠️, 🔴IMPORTANT❗🔴 There seems to be an issue with GPU allocation for all notebooks using Tensorflow below the 2.5 version. We are investigating the issue.

Segmentation networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
U-Net (2D) here and here Binary segmentation Fully supported 27/07/23 ✅ working (IH) here Open In Colab
U-Net (3D) here Binary segmentation Fully supported 27/07/23 ✅ working (IH) EPFL dataset Open In Colab
U-Net (2D) multilabel here and here Semantic segmentation Under beta-testing 16/07/23 ✅ working (IH) here Open In Colab
DenoiSeg here Joint denoising and binary segmentation Fully supported ⚠️ broken (no GPU) (GJ) Available soon Open In Colab
StarDist (2D) here and here Instance segmentation Fully supported 10/09/24 ✅ working (GJ) here Open In Colab
StarDist (3D) here and here Instance segmentation Fully supported 07/10/22 ✅ working (GJ) from Stardist github Open In Colab
Cellpose (2D and 3D) here Instance segmentation (Cells or Nuclei) Fully supported 05/09/23 ✅ working (IH) here Open In Colab
SplineDist (2D) here Instance segmentation Fully supported 07/10/22 ✅ working (GJ) here Open In Colab
EmbedSeg (2D) here Instance segmentation Under beta-testing 01/01/23 ✅ working (AR) here and here Open In Colab
MaskRCNN (2D) here Panoptic segmentation Under beta-testing Coming soon! Open In Colab
Interactive Segmentation - Kaibu (2D) here Interactive instance segmentation Under beta-testing Coming soon! Open In Colab

Denoising and image restoration networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
Noise2Void (2D) here Self-supervised denoising Fully supported 14/07/23 ✅ working (IH) here or here Open In Colab
Noise2Void (3D) here Self-supervised denoising Fully supported 14/07/23 ✅ working (IH) here Open In Colab
CARE (2D) here Supervised denoising Fully supported 31/07/23 ✅ working (IH) here or here Open In Colab
CARE (3D) here Supervised denoising Fully supported 31/07/23 ✅ working (IH) here Open In Colab
3D-RCAN here Supervised denoising Under beta-testing ⚠️ broken (no GPU) here Open In Colab
DecoNoising (2D) here Self-supervised denoising Under beta-testing 07/10/22 ✅ working (GJ) here or here Open In Colab

Super-resolution microscopy networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
Deep-STORM here Single Molecule Localization Microscopy (SMLM) image reconstruction from high-density emitter data Fully supported 27/07/23 ✅ working (IH) Training data simulated in the notebook or available from here Open In Colab
DFCAN here image upsampling Under beta-testing 08/10/22 ✅ working (GJ) here Open In Colab
WGAN here image upsampling Under beta-testing 22/09/22 ✅ working (IvanHidalgo & EGM) here Open In Colab

Object detection networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
YOLOv2 here Object detection (bounding boxes) Fully supported 🔴 Broken, not compatible with TF 2 here Open In Colab
Detectron2 here Object detection (bounding boxes) Under beta-testing here Open In Colab
RetinaNet here Object detection (bounding boxes) Under beta-testing here Open In Colab

Image-to-image translation networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
Label-free prediction (fnet) 2D here Artificial labelling Under beta-testing 🔴 Broken, not compatible with TF 2 Coming soon Open In Colab
Label-free prediction (fnet) 3D here Artificial labelling Fully supported here Open In Colab
CycleGAN here Unpaired Image-to-Image Translation Fully supported 14/07/23 ✅ working (IH) here Open In Colab
pix2pix here Paired Image-to-Image Translation Fully supported 12/12/23 ✅ working (EGM) here Open In Colab

Registration networks

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
DRMIME here Affine or perspective image registration Under beta-testing 12/08/22 ✅ working (GJ) Coming soon! Open In Colab

Image generation network

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
Diffusion Model for SMLM here SMLM image generation Fully supported 12/02/24 ✅ working (EGM) here and here Open In Colab

BioImage.io notebooks

Networks that are compatible with BioImage.IO and can be used in ImageJ via deepImageJ or Ilastik. The trained models in these notebooks are also exported in the BioImage.IO format and can be uploaded to the BioImage Model Zoo. Check our user guide to learn how to use the resources in the BioImage Model Zoo with ZeroCostDL4Mic.

Network Paper(s) Tasks Last test Link to example training and test dataset Direct link to the notebook in Colab
StarDist (2D) StarDist: here and here, and DeepImageJ and BioImage Model Zoo Nuclei segmentation 09/08/22 ✅ working (EGM) here Open In Colab
Deep-STORM with DeepImageJ export Deep-STORM and DeepImageJ Single Molecule Localization Microscopy (SMLM) image reconstruction from high-density emitter data 10/08/22 ✅ working (EGM) Training data simulated in the notebook or available from here Open In Colab
U-Net (2D) U-Net and DeepImageJ and BioImage Model Zoo Segmentation 16/07/23 ✅ working (IH) ISBI challenge or here Open In Colab
U-Net (2D) multilabel here and here and DeepImageJ and BioImage Model Zoo Semantic segmentation 16/07/23 ✅ working (IH) here Open In Colab
U-Net (3D) 3D U-Net and DeepImageJ and BioImage Model Zoo Segmentation 27/07/23 ✅ working (IH) EPFL dataset Open In Colab
pix2pix here Paired Image-to-Image Translation 12/12/23 ✅ working (EGM) here Open In Colab

Tools

Network Paper(s) Tasks Status Last test Link to example training and test dataset Direct link to the notebook in Colab
Augmentor here Image augmentation Fully supported 12/08/22 ✅ working (GJ) None Open In Colab
Quality Control here Error mapping and quality metrics estimation Fully supported 12/08/22 ✅ working (GJ) None Open In Colab
Mounting DropBox or MEGA in Google Colab None Tutorial explaining how to mount a Dropbox or MEGA account in Google Colab using Rsync Under beta-testing None Open In Colab

Related projects hosted elsewhere

Network Paper(s) Aim Link to the project Direct link to the notebook in Colab
DeepBacs here Toolbox to use Deep Learning to analyse microscopy images of bacteria here None
CAFI - DAIN here Content-aware frame interpolation using DAIN here Open In Colab
CAFI - ZoomingSlowMo here Content-aware frame interpolation using ZoomingSlowMo here Open In Colab
DECODE here Single Molecule Localization Microscopy (SMLM) image reconstruction here Open In Colab
EM-stellar here Electron microscopy image segmentation here Open In Colab

Contributors

Developers and testers

Founding

Newcomers

Researchers providing guidance and recommendations

Clone this wiki locally