NOISe: Nuclei-Aware Osteoclast Instance Segmentation for Mouse-to-Human Domain Transfer

NOISe builds on top of YOLOv8 for the task of osteoclast instance segmentation. This repository contains training, validation, and inference scripts to train an instance segmentation model from scratch and finetune on top of the provided checkpoints trained for osteoclast and nuclei detection as proposed in our paper:

Manne, S.K.R., Martin, B., Roy, T., Neilson, R., Peters, R., Chillara, M., Lary, C.W., Motyl, K.J., Wan, M., "NOISe: Nuclei-Aware Osteoclast Instance Segmentation for Mouse-to-Human Domain Transfer.” IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (CVPR 2024), Workshop on Computer Vision for Microscopy Image Analysis (CVMI).

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

Below verified setup uses Pytorch 2.3 with CUDA 12.1 support:

conda create --name noise python=3.8
conda activate noise
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip install ultralytics

For alternative installation methods including Conda, Docker, and Git, please refer to the Quickstart Guide.

Dataset

Our dataset consists of full slide images and corresponding instance segmentation annotations, along with patches used for training and validation in our experiments. Please download the dataset from here, unzip the data and place it in the dataset folder with the following folder structure:

noise
    |...
    |-- dataset
        |-- m1
            |-- images
            |-- labels
        |-- m2
            |-- images
            |-- labels
        |...
        |-- m5
            |-- images
            |-- labels
    |...

Update the dataset folder absolute path in osteo.yaml accordingly.

Whole Slide Inference

Instance segmentation prediction can be done for a whole slide image by creating overlapping patches of 832x832 resolution, that are then merged to generate a full-scale output. Different models trained on various configurations of data are available here. Please download the checkpoints and place them in the checkpoints folder.

Model Name	Info	Recommendation
yolo_mouse_ins.pt (YOLOv8 M→M)	YOLOv8 model trained on entire mouse data for osteoclast instance segmentation	Recommended model to test on mouse data
yolo_mouse_det_pretrain.pt	NOISE pretrain - YOLOv8 model trained on entire mouse data for osteoclast and nuclei detection	Recommended model to finetune on new data
noise_h1_ins_finetune.pt (NOISE H→H)	NOISe model finetuned on H1 dataset for osteoclast instance segmentation	Recommended model to test new human osteoclast data
noise_h2_ins_finetune.pt (NOISE H→H)	NOISe model finetuned on H2 dataset for osteoclast instance segmentation	Recommended model to test new human osteoclast data

Inference can be performed with the following command.

python wholeslide_inference.py --model_path path/to/checkpoint.pt --img_foldername path/to/images --out_foldername path/to/output --ratio r --device dev

Set --ratio to the appropriate $\mu m / pixel$ ratio for your dataset; the image will be patched accordingly, and the patches will then be scaled to 832x832 pixel resolution. The training dataset of our images used a ratio of 0.7784.

You can select a cuda device (default is the cpu) to use for inference with --device.

Outputs will be stored in path/to/output. The output for each image consists of a text file containing all predicted bounding boxes, objectness scores, and segmentation masks as well as an image representing these same results.

Training

Training from scratch or finetuning a specific checkpoint can be done using the following command:

python train.py --ckpt path/to/checkpoint.pt

Sample command to train YOLOv8 M→H model:

python train.py --ckpt checkpoints/yolo_mouse_det_pretrain.pt --data ./osteo.yaml --name YOLOv8MH

Validation

Running evaluation on test data can be done using the following command:

python val.py --ckpt path/to/checkpoint.pt

Sample command to validate the YOLOv8 M→M model:

python train.py --ckpt checkpoints/yolo_mouse_ins.pt --data ./osteo.yaml

Training on your data

NOISe can be further improved by training on custom data following these steps:

Preparing dataset

We provide a helper script to convert whole slide images into overlapping patches of a fixed size for training and validation of models.

python create_patches.py --img_foldername path/to/wsi_imagefolder --roi_foldername path/to/imageJ_rois --out_foldername path/to/patch_folder

Configs

Update the config.yaml according to the dataset created:

path: path/to/patch_data
train: [train_folder1, train_folder2, ...]
val: [test_folder1, test_folder2, ...]

Train

Once the dataset is generated, and the config.yaml file is adjusted accordingly, training can be done using:

python train.py --ckpt path/to/checkpoint.pt --data config.yaml

Acknowledgement

NOISe is largely based on YOLOv8.

Citation

If you find the dataset, model weights, or the code useful for your work, please consider citing us:

@InProceedings{ManneMartin_2023_CVPR,
    author    = {Manne, S.K.R. and Martin, B. and Roy, T. and Neilson, R. and Peters, R. and Chillara, M. and Lary, C.W. and Motyl, K.J. and Wan, M.},
    title     = {NOISe: Nuclei-Aware Osteoclast Instance Segmentation for Mouse-to-Human Domain Transfer},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2024}
}

Note that the first two authors contributed equally.