Median-Nerve-Segmentation

---

Installation

Clone the git-repository first.

git clone https://github.com/karang2606/Median-Nerve-Segmentation.git

Then, install the latest version of Pytorch and Torchvision from this link or below.

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

Install pycocotools

conda install cython scipy
pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
pip install git+https://github.com/youtubevos/cocoapi.git#"egg=pycocotools&subdirectory=PythonAPI"

From the pycocotools folder in the repository, copy ytvos.py and ytvoseval.py to the package installation location in your system, i.e., home/anaconda3/lib/python3.10/site-packages/pycocotools/

Compile DCN module(requires GCC>=5.3, cuda>=10.0)

cd models/VisTR/models/dcn
python setup.py build_ext --inplace

---

Preparation

Set directory structure as follows:

VisTR
├── data
│   ├── train
│   |   ├──patient_01
│   |   |   ├──images
│   |   |   |   ├──0.jpg, ....
│   |   |   ├──masks
│   |   |       ├──0.png, ....
│   ├── test
...

Download the pretrained DETR models Google Drive on COCO and save it to the pretrained path.

---

Training

Training the model requires at least 30GB of GPU memory, so we have utilized two NVIDIA RTX A6000 GPU cards with a memory of 48GB each.

Details of some training arguments:

--model_name: Provide the model name from any of these.

Model name	Argument
UNet1	unet
UNet++2	unetpp
Attention UNet3	attn_unet
Siamese UNet4	siam_unet
LSTM UNet5	lstm_unet
TransUNet6	trans_unet
Video Instance Segmentation with Transformers7	vistr

--no_aug: If true, we augment the data by doing horizontal and vertical flips.
--data_path: Provide a path for training data. The default is "data/train/"
--lr: Learning rate
--num_frames: For LSTM and VisTR, clip length for training the model. The default is 36.
--epochs: Number of epochs for training.
--pretrained_weights: Path to pretrained weights for VisTR backbone. The default is "pretrained/384_coco_r101.pth"

To train the model, run the following:

python train.py --model_name unet --batch_size 16 --val_batch_size 16 --no_aug

---

Testing

Download the trained weights of model from here Details of some test arguments:
--data_path: Provide a path for testing data. The default is "data/test/"
--load_from: Path to pretrained model weights. --save_clip: If given, the model will save each test video with the segmentation result and cross-sectional area of the nerve at the bottom. --perturb_input: Perturbe the input frame with Poisson noise. --perturb_model: Perturbe the model with a given fraction. --no_GT: Testing for video without having ground truth.

To test the model, run the following:

python test.py --model_name vistr --perturb_input 0.05 --perturb_model 0.1

---

Acknowledgement

We would like to thank the DETR and VisTR open-source projects for their awesome work; part of the code is modified from their project.

---

References

Footnotes

1. Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015. ↩

2. Zhou, Zongwei, et al. "Unet++: A nested u-net architecture for medical image segmentation." Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer International Publishing, 2018. ↩

3. Oktay, Ozan, et al. "Attention u-net: Learning where to look for the pancreas." arXiv preprint arXiv:1804.03999 (2018). ↩

4. Dunnhofer, Matteo, et al. "Siam-U-Net: encoder-decoder siamese network for knee cartilage tracking in ultrasound images." Medical Image Analysis 60 (2020): 101631. ↩

5. Hsu, Mu Chien, Jui Chun Shyur, and Hiroshi Watanabe. "Pseudo Ground Truth Segmentation Mask to Improve Video Prediction Quality." 2020 IEEE 9th Global Conference on Consumer Electronics (GCCE). IEEE, 2020. ↩

6. Chen, Jieneng, et al. "Transunet: Transformers make strong encoders for medical image segmentation." arXiv preprint arXiv:2102.04306 (2021). ↩

7. Wang, Yuqing, et al. "End-to-end video instance segmentation with transformers." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021. ↩