Counting Perineuronal Nets

PyTorch code for training and evaluating cell counting and localization methodologies. We provide pretrained models for counting perineuronal nets (PNN) and parvalbumin cells (PV) in fluorescence microscopy images.

Papers

• Ciampi, Luca, et al. "Learning to count biological structures with raters’ uncertainty." Medical Image Analysis 80 (2022): 102500.
• Lupori, Leonardo, et al. "A Comprehensive Atlas of Perineuronal Net Distribution and Colocalization with Parvalbumin in the Adult Mouse Brain." bioRxiv (2023): 2023-01.

Getting Started

You'll need:

• Python 3.8
• torch 1.11.0 (torchvision 0.12.0)
• other packages in requirements.txt

Installation via conda

This is the suggested installation for Windows users. Install Anaconda or miniconda and then run the following commands.

# create and activate conda environment
conda create -n cpn python=3.8
conda activate cpn

# for CPU-only
pip install torch==1.11.0+cpu torchvision==0.12.0+cpu --extra-index-url https://download.pytorch.org/whl/cpu

# for GPU-accelerated prediction, run this instead:
#
# pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
#
# or see https://pytorch.org/get-started/previous-versions/ for other CUDA versions.

# install other requirements
pip install -r requirements.txt

Installation via Docker

We also provide a Dockerfile to build the environment.

Trained models

We provide Localization and Scoring models for PNNs and PVs structures:

	Latest (v0.5)	v0.3
PNN Localization	FRCNN-640	FRCNN-640 | UNet-320
PNN Scoring	OR | RL	AC | OR | RL
PV Localization	FRCNN-640	-
PV Scoring	OR | RL	-

❗ Ensure you have checked out the correct code version (v0.5 or v0.3) for the model you want to use.

How to do predictions

Download and unzip a localization and a scoring model. Alternatively, you can train your own models (see next section). Then, you can do predictions using the predict.py script by passing the extracted run folders and the paths to data to process. E.g.:

# use both localization and scoring models (tip: set a low localization threshold for high-recall localization)
python predict.py pnn_v2_fasterrcnn_640/ -r pnn_v2_scoring_rank_learning/ -t 0.0 my_images_*.tiff

# use only the localization model (uses default threshold)
python predict.py pnn_v2_fasterrcnn_640/ my_images_*.tiff

# check python predict.py -h for more options

Accepted formats for input data are image formats (TIFF, PNG, JPEG, etc.) and the HDF5 format. We assume a sigle 1-channel (bidimensional) image per input file. For HDF5, we assume the image is stored in the /data dataset.

Visualize predictions

We provide a utility script to visualize the predictions of the network on the input images.

# by default, the script looks for input images in the current folder
python draw_predictions.py localizations.csv

#  if you want to read/write images in folders different from the current one, use:
python draw_predictions.py --root path/to/input_images_dir/ --output path/to/output_dir/ localizations.csv

# check python draw_predictions.py -h for all options

How to train

First, you need to download the datasets in the data/ folder (see these instructions). Then you can use the train.py and train_score.py script to launch training sessions.

Stage 1: Localization Model Training

Train configurations for localization models are specified with Hydra config groups in conf/experiments. You can run a training experiment by passing as argument experiment=<exp_name> to the train.py script, where <exp_name> is the path to a YAML experiment configuration relative to conf/experiments and without the .yaml extension.

Examples:

• Train the detection-based approach (FasterRCNN) with 480x480 patches on PerineuronalNets:

  python train.py experiment=perineuronal-nets/detection/fasterrcnn_480

• Train the density-based approach (CSRNet) on VGG Cells:

  python train.py experiment=vgg-cells/density/csrnet

Runs files will be produced in the runs folder. Once trained, you can evaluate the trained models on the corresponding test sets using the evaluate.py script. E.g.,

# check python evaluate.py -h for more options
python evaluate.py runs/experiment=perineuronal-nets/detection/fasterrcnn_480/

Metrics and predictions will be saved in the run folder under test_predictions/.

Stage 2: Scoring Model Training

Train configurations for scoring models are specified in conf_score/. You can run a training experiment by passing as argument method=<method_name> to the train_score.py script, where <method_name> is the path to a YAML configuration relative to conf_score/method and without the .yaml extension (e.g., ordinal_regression).

Scoring models can be trained only on the Perineuronal Nets dataset that contains multi-rater data.

Examples:

# train scorer with Agreement Ordinal Regression method
python train_score.py method=ordinal_regression seed=87

# train scorer with Agreement Rank Learning method
python train_score.py method=pairwise_balanced seed=67

Runs files will be produced in the runs_score folder. Once trained, you can evaluate the trained models on the corresponding test sets using the evaluate_score.py script. E.g.,

# check python evaluate_score.py -h for more options
python evaluate_score.py runs_score/method=pairwise_balanced,seed=67/

Metrics and predictions will be saved in the run folder under test_predictions/.

Reproducing Experiments

• Check the reproduce.sh script for launching all training and evaluation phases defined in the paper.

• Check the show.ipynb notebook for producing figures and tables presented in the paper.