Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Related to label deviating position (DeepEdit multi label app) #649

Closed
SachidanandAlle opened this issue Feb 23, 2022 · 21 comments
Closed

Related to label deviating position (DeepEdit multi label app) #649

SachidanandAlle opened this issue Feb 23, 2022 · 21 comments
Labels
0.4.0 Targeted to Release version 0.4 help wanted Extra attention is needed
Projects
MONAILabel-v0.4.0
Milestone
Bugs + Miscellane...

Comments

@SachidanandAlle
Copy link
Collaborator

SachidanandAlle commented Feb 23, 2022 •
edited

Dear Sachidanand,

First of all I would like to thank and congratulate you for your tremendous work. AI for healthcare imaging has never been more accessible.

I am a medical physicist in radiotherapy department in Amsterdam. I am currently investigating the possibilities of deep learning based segmentation in adaptive radiotherapy using MRI images. I've been using/developing monailabel APP (deep_edit multilabel app) for a while and so far I'm very happy with it.

However, I have encountered a few issues and have not been able to resolve them and I hope you can help me with that;

  • This app generates labels with a (default) deviating position ( x: -2, y: -2, z: -2). Maybe this is because of a transform or something else do you have an idea how I can solve this?

-I use this app to segment different organs in the abdomen. The results are very good for 8 of the 9 organs (dice > 0.95), but for the duodenum it is much lower. Is there an opportunity within monailabel to put more focus on improving a single structure.

  • Is there a possibility to automatically export the created labels in Dicom-RT file structure.

Kind Regards

OB
@SachidanandAlle SachidanandAlle added help wanted Extra attention is needed 0.4.0 Targeted to Release version 0.4 labels Feb 23, 2022
@SachidanandAlle SachidanandAlle added this to To do in MONAILabel-v0.4.0 via automation Feb 23, 2022
@SachidanandAlle
Copy link
Collaborator Author

SachidanandAlle commented Feb 23, 2022 •
edited

@diazandr3s can you help on the deepedit issue. Looks like some post-transform issue.
Or may be the problem is already solved in latest main branch.

-I use this app to segment different organs in the abdomen. The results are very good for 8 of the 9 organs (dice > 0.95), but for the duodenum it is much lower. Is there an opportunity within monailabel to put more focus on improving a single structure.
MONAI Label deepedit is one kind of example App. There can be multiple ways to solve certain use-cases. For example you can even train segmentation model per single structure or use other networks. Or use combinations of models (not just one deepedit/deeplearning model)..

If you have DICOM Webserver, we currently support NIFTI/Label masks to get converted into DICOM-Seg using dcmqi tools (itkimage2segimage). We didn't see a use-case of producing DICOM-RT so far.

@diazandr3s
Copy link
Collaborator

Thanks for pinging me here.
Regarding the post transforms, it was solved in the latest branch (https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/deepedit_multilabel/lib/infer.py#L161).

Would it be possible to try with the latest MONAI Label version? Install the weekly version: pip install monailabel-weekly and let us know how that goes :)

@OB-1606
Copy link

OB-1606 commented Feb 23, 2022

****MONAI Label deepedit is one kind of example App. There can be multiple ways to solve certain use-cases. For example you can even train segmentation model per single structure or use other networks. Or use combinations of models (not just one deepedit/deeplearning model).** <-- Thank you this is clear

If you have DICOM Webserver, we currently support NIFTI/Label masks to get converted into DICOM-Seg using dcmqi tools (itkimage2segimage). We didn't see a use-case of producing DICOM-RT so far. <-- I have used OHIF, you can directly import dicom files, but after segmenting there is only the option to export NRRD or NIFTI. Most systems in Radiation oncology work with DICOM-RT files it would be nice if there was a Dicom-RT directly.

Thank you for your quick response and help

@SachidanandAlle
Copy link
Collaborator Author

when you save/submit the label to monailabel.. it gets added as DICOM SEG into your Orthanc/DICOM server..
NRRD or NIFTI is for local download (for debugging purpose)

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

thank you for your last suggestion,

I installed the latest weekly version using [pip install monailabel-weekly] however after installing I got the message "Successfully installed monailabel-weekly-0.4.dev2205". Is this correct? because the latest version I can find is monailabel-weekly 0.4.dev2208.

After installation I ran a new test; x and y direction still a difference of -2, however in the z direction it is now correct so no more deviation.

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022 via email

@diazandr3s
Copy link
Collaborator

Thanks, @OB-1606.
That monailabel-weekly version should be OK. If you like, you could specifically install the latest version using this command:
pip install monailabel-weekly==0.4.dev2208
With regards to the direction difference, would it be possible to see a print screen of the image/label? If you can, please also attach the log file (app.log) you can find in the logs folder.

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022 •
edited

Thanks,

Using --> pip install monailabel-weekly==0.4.dev2208 results in error:

afbeelding

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

regarding direction difference here a print screen:
image

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

hereby the log file

app.log

@diazandr3s
Copy link
Collaborator

Thanks for your quick response, @OB-1606.
From what I can see, the predicted label is a bit blocky and moved because of the image resolution used to train the model.

This is the result of upsampling both image and predicted mask. By default, the deepedit_multilabel App resizes the images to 128x128x128. See here.

This means, images are downsampled before performing inference and then upsampled to the original size. The predicted label is originally 128 cube and then upsample to the original image size.

You can easily change this if you have enough GPU memory. What is the average image size you're using for this?

With regards to the monailabel version, let's use the one you have before (monailabel-weekly-0.4.dev2205) while we investigate the error you've faced. Don't think there are major differences between those.

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

Thanks again,

I checked for several images: 276 x 276 x 80.

I also use heuristic_planner does that matter?

@diazandr3s
Copy link
Collaborator

Yes, heuristic planner is working and it considers the available GPU memory to define the image size for training and inferencing. I take this from the log you attached before:

[2022-02-23 03:56:34,848] [9616] [MainThread] [INFO] (monailabel.utils.others.planner:71) - Available GPU memory: {0: 8785} in MB [2022-02-23 03:56:34,849] [9616] [MainThread] [INFO] (monailabel.utils.others.generic:131) - Using nvidia-smi command [2022-02-23 03:56:34,880] [9616] [MainThread] [INFO] (monailabel.utils.others.planner:75) - Spacing: [2. 2. 3.]; Spatial Size: [128, 128, 128]

I see your GPU has around 9 GB of memory available. I'm afraid it won't be possible to train a model with bigger image size using this GPU card.

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

so if I use a different GPU (larger memory size) solves this problem?

@diazandr3s
Copy link
Collaborator

Yes, it should! You can either leave the heuristic planner to decide the image size or you just disable it and fix the image size yourself here https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/deepedit_multilabel/main.py#L57

@SachidanandAlle
Copy link
Collaborator Author

based on the image and bug description.. this looks like an issue with resize (round) issue.. just a hint from myside.. may be i am wrong..

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

changing current GPU is not straight forward at this moment and I still cant understand why its always the same deviation?

@diazandr3s
Copy link
Collaborator

@SachidanandAlle, I think the same. The prediction is blocky and deviated because of the image resizing done for training and inference.
One thing you could do @OB-1606 is label refinement after getting the prediction. This can be done using the manual tools available in Slicer.

@OB-1606
Copy link

OB-1606 commented Feb 24, 2022

what is the minimum GPU memory required to solve this?

I'm already using some post labeling steps which makes the labels smoother but the deviation makes it a bit confusing

@diazandr3s
Copy link
Collaborator

This is a good question, @OB-1606. This depends on different factors such as image size, data transforms, network architecture, batch size, etc
Just as a reference, I've trained the multilabel deepedit using the BTCV dataset using an image size of 256x256x128, bacth size =1 and it occupies around 17 GB of GPU memory.
As the images you're using are 276 x 276 x 80, my recommendation is to try an image size closer to that (i.e., 256x256x128)

Hope this helps.

@SachidanandAlle
Copy link
Collaborator Author

I see no further comments.. closing the issue for now.

MONAILabel-v0.4.0 automation moved this from To do to Done Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.4.0 Targeted to Release version 0.4 help wanted Extra attention is needed
Projects
No open projects
MONAILabel-v0.4.0
Done
Milestone
Bugs + Miscellaneous [p0]
Development

No branches or pull requests
3 participants
@SachidanandAlle @diazandr3s @OB-1606