Self-supervised Neuron Segmentation with Multi-agent Reinforcement Learning (IJCAI 2023)

This repository contains the official implementation of the paper Self-supervised Neuron Segmentation with Multi-agent Reinforcement Learning, presented at IJCAI 2023. You can find the paper here.

Visitor Count

Figure 1: The pipeline of our proposed methods

Figure 2: The framework of our proposed decision module

Environment Setup

To streamline the setup process, we provide a Docker image that can be used to set up the environment with a single command. The Docker image is available at:

docker pull registry.cn-hangzhou.aliyuncs.com/cyd_dl/monai-vit:v26

Dataset Download

The datasets required for pre-training and segmentation are as follows:

Dataset Type	Dataset Name	Description	URL
Pre-training Dataset	Region of FAFB Dataset	Fly brain dataset for pre-training	EM Pretrain Dataset
Segmentation Dataset	CREMI Dataset	Challenge on circuit reconstruction datasets	CREMI Dataset
Segmentation Dataset	AC3/AC4	AC3/AC4 Dataset	Mouse Brain GoogleDrive

Pre-training Dataset: Region of FAFB

The FAFB region dataset is used for pre-training. Please follow the instructions provided in the paper to acquire and preprocess this dataset. You can download it from the Hugging Face EM Pretrain Dataset. Use the subfolder FAFB_hdf to match the paper's settings, or use additional relevant data to achieve better results.

To use this dataset, please refer to the license provided here.

Segmentation Dataset: CREMI

The CREMI dataset is used for the segmentation tasks. Detailed instructions for downloading and preprocessing can be found on the CREMI Challenge website.

Usage Guide

1. Pretraining

python pretrain.py -c pretraining_all -m train

2. Finetuning

python finetune.py -c seg_3d -m train -w [your pretrained path]

License (Important !!!)

Usage Notes

Before the public release of the data, the following usage restrictions must be met:

1. Non-commercial Use: Users do not have the rights to copy, distribute, publish, or use the data for commercial purposes or develop and produce products. Any format or copy of the data is considered the same as the original data. Users may modify the content and convert the data format as needed but are not allowed to publish or provide services using the modified or converted data without permission.

2. Research Purposes Only: Users guarantee that the authorized data will only be used for their own research and will not share the data with third parties in any form.

3. Citation Requirements: Research results based on the authorized data, including books, articles, conference papers, theses, policy reports, and other publications, must cite the data source according to citation norms, including the authors and the publisher of the data.

4. Prohibition of Profit-making Activities: Users are not allowed to use the authorized data for any profit-making activities.

5. Termination of Data Use: Users must terminate all use of the data and destroy the data (e.g., completely delete from computer hard drives and storage devices/spaces) upon leaving their team or organization or when the authorization is revoked by the copyright holder.

Data Information

• Sample Source: Mouse MEC MultiBeam-SEM, Intelligent Institute Brain Imaging Platform (Wafer 4 at layer VI, wafer 25, wafer 26, and wafer 36 at layer II/III)
• Resolution: 8nm x 8nm x 35nm
• Volume Size: 1250 x 1250 x 125
• Annotation Completion Dates: 2023.12.11 (w4), 2024.04.12 (w36)
• Authors: Shi Te, Guo Jun, Yin Chunying, Zhang Ruobing
• Copyright Holder: Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

Acknowledgment Norms

• Chinese Name: 合肥综合性国家科学中心人工智能研究院
• English Name: Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

Citation

If you find this code or dataset useful in your research, please consider citing our paper:

@inproceedings{chen2023self,
  title={Self-supervised neuron segmentation with multi-agent reinforcement learning},
  author={Chen, Yinda and Huang, Wei and Zhou, Shenglong and Chen, Qi and Xiong, Zhiwei},
  booktitle={Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence},
  pages={609--617},
  year={2023}
}

To-Do List

• Open-sourced the core code
• Wrote the README for code usage
• Open-sourced the pre-training dataset
• Upload the pre-trained weights

Contact

If you need any help or are looking for cooperation feel free to contact us. cyd0806@mail.ustc.edu.cn