Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intervertebral disc labeling using pose estimation #787

Closed
wants to merge 56 commits into from
Closed

Intervertebral disc labeling using pose estimation #787

wants to merge 56 commits into from

Conversation

rezazad68
Copy link
Collaborator

@rezazad68 rezazad68 commented May 13, 2021

Checklist

GitHub

  • I've given this PR a concise, self-descriptive, and meaningful title
  • I've linked relevant issues in the PR body
  • I've applied the relevant labels to this PR
  • I've assigned a reviewer

PR contents

Description

In this pull, we aim to reformulate the semantic intervertebral disc labeling using the pose estimation technique. To this end, we utilize the stacked hourglass network. The proposed deep model takes into account the strength of semantic segmentation and pose estimation technique to handle the missing and falsely additional disc areas.

Linked issues

resolves #539
resolves #766

Run Demo:

  1. Download Spine Generic Public Database (Multi-Subject) dataset
  2. Run (update path to proper location):
    python prepare_dataset_vertebral_labeling.py --path ~/data-multi-subject/ -s _T2w
    

After running the above command, it will generate the mid files. While running it will print the list of subjects that the mid file is generating for them. For example:
image

3- Run ivadomed --train -c config_pose.json --gif 2

ivadomed/transforms.py Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
@jcohenadad
Copy link
Member

@rezazad68 can you please add a description in the body PR

Copy link
Member

@joshuacwnewton joshuacwnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excited to see this! I'm looking forward to seeing how this compares with spinalcordtoolbox/spinalcordtoolbox#2679.

ivadomed/loader/loader.py Outdated Show resolved Hide resolved
ivadomed/training.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/losses.py Show resolved Hide resolved
ivadomed/training.py Outdated Show resolved Hide resolved
ivadomed/main.py Outdated Show resolved Hide resolved
ivadomed/models.py Outdated Show resolved Hide resolved
ivadomed/models.py Outdated Show resolved Hide resolved
ivadomed/models.py Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/main.py Outdated Show resolved Hide resolved
ivadomed/losses.py Show resolved Hide resolved
ivadomed/models.py Outdated Show resolved Hide resolved
the condition on the prediction mask is slightly changed.

Co-authored-by: Joshua Newton <joshuacwnewton@gmail.com>
@joshuacwnewton

This comment has been minimized.

@jcohenadad

This comment has been minimized.

@joshuacwnewton

This comment has been minimized.

@jcohenadad

This comment has been minimized.

@coveralls
Copy link

coveralls commented May 23, 2021

Pull Request Test Coverage Report for Build 870259786

  • 37 of 209 (17.7%) changed or added relevant lines in 7 files are covered.
  • 60 unchanged lines in 4 files lost coverage.
  • Overall coverage decreased (-1.8%) to 68.434%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ivadomed/loader/loader.py 3 4 75.0%
ivadomed/losses.py 3 20 15.0%
ivadomed/training.py 0 21 0.0%
ivadomed/transforms.py 14 35 40.0%
ivadomed/models.py 15 127 11.81%
Files with Coverage Reduction New Missed Lines %
ivadomed/training.py 1 11.11%
ivadomed/main.py 2 13.72%
ivadomed/loader/loader.py 15 85.86%
ivadomed/inference.py 42 74.47%
Totals Coverage Status
Change from base Build 830411545: -1.8%
Covered Lines: 4065
Relevant Lines: 5940

💛 - Coveralls

@jcohenadad
Copy link
Member

@mariehbourget Thanks for the comment. Could you please specify what can be the reason? I mean is it related to bad data copy or some part of the code?

The error is in the structure of the sub-barcelona05_T2w_csfseg-manual.json file in the dataset. For examples: missing quote, misplaced comma/bracket, etc.
So the error doesn't come from the ivadomed code but from the source file in the spine-generic/data-multi-subject dataset.

thank you @mariehbourget ! i'll try again after removing the problematic JSON files

@jcohenadad
Copy link
Member

jcohenadad commented Jun 26, 2021

ok, now i am getting this with ivadomed 8d96972 and data-multi-subject 9d27a7010d8d528e369a7ed86b6278c64a7cdf77 and this config file: config_pose_julien.json.zip

Terminal output
(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2
Output path already exists: /Users/julien/temp/20210625_reza
Cuda is not available.
Working on cpu.

Selected architecture: HourglassNet, with the following parameters:
	dropout_rate: 0.3
	bn_momentum: 0.1
	depth: 3
	is_2d: True
	folder_name: pose_model
	in_channel: 1
	out_channel: 1
Missing derivatives for sub-amu01_T1w.nii.gz. Skipping.
Missing derivatives for sub-amu02_T1w.nii.gz. Skipping.
Missing derivatives for sub-amu03_T1w.nii.gz. Skipping.
Missing derivatives for sub-amu04_T1w.nii.gz. Skipping.
Missing derivatives for sub-amu05_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist01_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist02_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist03_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist04_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist05_T1w.nii.gz. Skipping.
Missing derivatives for sub-balgrist06_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona01_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona02_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona03_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona04_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona05_T1w.nii.gz. Skipping.
Missing derivatives for sub-barcelona06_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE01_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE02_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE03_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE04_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma01_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma02_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma03_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma04_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma05_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio01_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio02_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio03_T1w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio04_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec01_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec02_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec03_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec04_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec05_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec06_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb01_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb02_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb03_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb04_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb05_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb06_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb07_T1w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb08_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff01_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff02_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff03_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff04_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff05_T1w.nii.gz. Skipping.
Missing derivatives for sub-cardiff06_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra01_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra02_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra03_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra04_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra05_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrra06_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb01_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb02_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb03_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb04_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb05_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb06_T1w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb07_T1w.nii.gz. Skipping.
Missing derivatives for sub-dresden01_T1w.nii.gz. Skipping.
Missing derivatives for sub-dresden02_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva01_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva02_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva03_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva04_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva05_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva06_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma01_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma02_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma03_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma04_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma05_T1w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma06_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva01_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva02_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva03_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva04_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva05_T1w.nii.gz. Skipping.
Missing derivatives for sub-geneva06_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg01_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg02_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg03_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg04_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg05_T1w.nii.gz. Skipping.
Missing derivatives for sub-hamburg06_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w01_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w02_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w03_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w04_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w05_T1w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w06_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh01_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh02_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh03_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh04_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh05_T1w.nii.gz. Skipping.
Missing derivatives for sub-mgh06_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan01_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan02_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan03_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan04_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan05_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan06_T1w.nii.gz. Skipping.
Missing derivatives for sub-milan07_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniPilot1_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS01_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS02_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS03_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS04_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS05_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS06_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS07_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS08_T1w.nii.gz. Skipping.
Missing derivatives for sub-mniS09_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai01_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai02_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai03_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai04_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai05_T1w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai06_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs01_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs02_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs03_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs05_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs06_T1w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs07_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil01_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil02_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil03_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil04_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil05_T1w.nii.gz. Skipping.
Missing derivatives for sub-nottwil06_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu01_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu02_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu03_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu04_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu05_T1w.nii.gz. Skipping.
Missing derivatives for sub-nwu06_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib01_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib02_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib03_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib04_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib05_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib06_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib07_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib08_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib09_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib10_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib11_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba01_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba02_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba03_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba04_T1w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba05_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia01_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia02_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia03_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia04_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia05_T1w.nii.gz. Skipping.
Missing derivatives for sub-pavia06_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform01_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform02_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform03_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform04_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform05_T1w.nii.gz. Skipping.
Missing derivatives for sub-perform06_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland01_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland02_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland03_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland04_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland05_T1w.nii.gz. Skipping.
Missing derivatives for sub-queensland06_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke01_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke02_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke03_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke04_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke05_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke06_T1w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke07_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford01_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford02_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford03_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford04_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford05_T1w.nii.gz. Skipping.
Missing derivatives for sub-stanford06_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg01_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg02_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg03_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg04_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg05_T1w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg06_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS01_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS02_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS03_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS04_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS05_T1w.nii.gz. Skipping.
Missing derivatives for sub-tehranS06_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w01_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w02_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w03_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w04_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w05_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w06_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w07_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia01_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia02_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia03_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia04_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia05_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia06_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia07_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra01_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra02_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra03_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra04_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra05_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra06_T1w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra07_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc01_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc02_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc03_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc04_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc05_T1w.nii.gz. Skipping.
Missing derivatives for sub-ubc06_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis01_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis02_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis03_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis04_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis05_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis06_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis07_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl01_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl02_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl03_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl04_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl05_T1w.nii.gz. Skipping.
Missing derivatives for sub-ucl06_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf01_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf02_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf03_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf04_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf05_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf06_T1w.nii.gz. Skipping.
Missing derivatives for sub-unf07_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron01_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron02_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron03_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron04_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron05_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron06_T1w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron07_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva01_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva02_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva03_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva04_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva05_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva06_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia01_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia02_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia03_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia04_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia05_T1w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia06_T1w.nii.gz. Skipping.
Traceback (most recent call last):
  File "/Users/julien/code/ivadomed/venv/bin/ivadomed", line 33, in <module>
    sys.exit(load_entry_point('ivadomed', 'console_scripts', 'ivadomed')())
  File "/Users/julien/code/ivadomed/ivadomed/main.py", line 550, in run_main
    run_command(context=context,
  File "/Users/julien/code/ivadomed/ivadomed/main.py", line 336, in run_command
    bids_df = imed_loader_utils.BidsDataframe(loader_params, path_output, derivatives=True)
  File "/Users/julien/code/ivadomed/ivadomed/loader/utils.py", line 624, in __init__
    self.create_bids_dataframe()
  File "/Users/julien/code/ivadomed/ivadomed/loader/utils.py", line 725, in create_bids_dataframe
    raise RuntimeError("Derivatives not found.")
RuntimeError: Derivatives not found.

@rezazad68 can you pls show exactly your terminal output, and mention your code git commit + database commit.

Update: the issue was caused by the misleading "test this PR demo" in the PR body. It says:

1- Download Spine Generic Public Database (Multi-Subject) dataset
2- Run prepare_dataset_vertebral_labeling.py to generate the mid files. Sample command:
python3 prepare_dataset_vertebral_labeling.py --path /data/data-multi-subject/ -s _T2w

while the file config_pose.json specifies T1w contrast. The issue was fixed by changing T1w for T2w. @rezazad68 this is another example of why it is important to make a consistent documentation so people can properly test the PRs without having to spend hours figuring out what the problem is...

@jcohenadad
Copy link
Member

jcohenadad commented Jun 26, 2021

OK, fixing the JSON config file by changing T1w for T2w, I am still getting "Skipping derivative", but this time because of a discrepancy between the image file name and the GT file name.

Example: image file name:

Missing derivatives for sub-amu01_T2w.nii.gz. Skipping.

While the derivatives is:

sub-amu01_rec-mid_T2w_heatmap0.nii.gz

And the config file states:

        "target_suffix": ["_heatmap0"],

So it no surprise that the GT cannot be associated with the images.

Full terminal output
(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2
Output path already exists: /Users/julien/temp/20210625_reza
Cuda is not available.
Working on cpu.

Selected architecture: HourglassNet, with the following parameters:
	dropout_rate: 0.3
	bn_momentum: 0.1
	depth: 3
	is_2d: True
	folder_name: pose_model
	in_channel: 1
	out_channel: 1
Missing derivatives for sub-amu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist01_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist02_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist03_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist04_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist06_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona01_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona02_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona03_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona04_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona05_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona06_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb08_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden01_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma06_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan01_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan02_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan03_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan04_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan05_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniPilot1_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS08_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS09_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs07_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil06_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib05_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib07_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib08_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib09_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib10_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib11_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform01_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform02_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform03_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform04_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform05_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform06_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland01_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland02_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland03_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland04_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland05_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke01_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke02_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke03_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke04_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke05_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke07_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford01_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford02_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford03_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford04_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford05_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford06_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf01_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf02_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf03_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf04_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf05_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia06_T2w.nii.gz. Skipping.
Dataframe has been saved in /Users/julien/temp/20210625_reza/bids_dataframe.csv.
After splitting: train, validation and test fractions are respectively 0.6, 0.2 and 0.2 of participant_id.

Selected transformations for the ['training'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}

Selected transformations for the ['validation'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}
Loading dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [00:00<00:00, 200.47it/s]
Loaded 51 sagittal slices for the validation set.
Loading dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 156/156 [00:00<00:00, 212.50it/s]
Loaded 151 sagittal slices for the training set.
Creating model directory: /Users/julien/temp/20210625_reza/pose_model

Initialising model's weights from scratch.

Scheduler parameters: {'name': 'CosineAnnealingLR', 'base_lr': 1e-06, 'max_lr': 0.0001}

Selected Loss: JointsMSELoss
	with the parameters: []
Training:  10%|███████████████▊                                                                                                                                              | 1/10 [00:00<?, ?it/s]/Users/julien/code/ivadomed/venv/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "

@rezazad68 I spent enough time trying to figure out things. What I need from you at this point is an EXACT syntax to reproduce your work, including the EXACT git commit of the dataset and ivadomed code.

UPDATE: these "skipping" messages are normal: they appear because there is in fact no GT for these images (e.g. sub-vuiisAchieva06_T2w.nii.gz). The script prepare_dataset_vertebral_labeling.py generates the 2D image files *_rec-mid_T2w.nii.gz (example: sub-vallHebron05_rec-mid_T2w.nii.gz). As mentioned in #787 (comment), having a description of what the script does would prevent users spending time figuring it out themselves.

@jcohenadad
Copy link
Member

This is maybe more a question for the core dev of ivadomed, but I launched this:

(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2

Expecting to get some fancy GIF files, but nothing there after 4 epochs 🤷‍♂️

julien-macbook:~/temp/20210625_reza $ find . -name *.gif
julien-macbook:~/temp/20210625_reza $ 
Terminal output
(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2
Output path already exists: /Users/julien/temp/20210625_reza
Cuda is not available.
Working on cpu.

Selected architecture: HourglassNet, with the following parameters:
	dropout_rate: 0.3
	bn_momentum: 0.1
	depth: 3
	is_2d: True
	folder_name: pose_model
	in_channel: 1
	out_channel: 1
Missing derivatives for sub-amu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist01_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist02_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist03_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist04_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist06_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona01_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona02_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona03_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona04_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona05_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona06_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb08_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden01_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma06_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan01_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan02_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan03_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan04_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan05_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniPilot1_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS08_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS09_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs07_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil06_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib05_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib07_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib08_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib09_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib10_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib11_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform01_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform02_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform03_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform04_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform05_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform06_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland01_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland02_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland03_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland04_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland05_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke01_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke02_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke03_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke04_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke05_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke07_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford01_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford02_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford03_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford04_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford05_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford06_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf01_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf02_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf03_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf04_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf05_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia06_T2w.nii.gz. Skipping.
Dataframe has been saved in /Users/julien/temp/20210625_reza/bids_dataframe.csv.
After splitting: train, validation and test fractions are respectively 0.6, 0.2 and 0.2 of participant_id.

Selected transformations for the ['training'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}

Selected transformations for the ['validation'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}
Loading dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [00:00<00:00, 200.47it/s]
Loaded 51 sagittal slices for the validation set.
Loading dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 156/156 [00:00<00:00, 212.50it/s]
Loaded 151 sagittal slices for the training set.
Creating model directory: /Users/julien/temp/20210625_reza/pose_model

Initialising model's weights from scratch.

Scheduler parameters: {'name': 'CosineAnnealingLR', 'base_lr': 1e-06, 'max_lr': 0.0001}

Selected Loss: JointsMSELoss
	with the parameters: []
Training:  10%|███████████████▊                                                                                                                                              | 1/10 [00:00<?, ?it/s]/Users/julien/code/ivadomed/venv/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "
Epoch 1 training loss: 0.0026.  Dice training loss: -0.0010.                                                                                                                                        
Epoch 1 validation loss: 0.0003.        Dice validation loss: -0.0015.                                                                                                                              
Epoch 1 took 592.04 seconds.                                                                                                                                                                        
Epoch 2 training loss: 0.0002.  Dice training loss: -0.0235.                                                                                                                                        
Epoch 2 validation loss: 0.0002.        Dice validation loss: 0.0016.                                                                                                                               
Epoch 2 took 592.96 seconds.                                                                                                                                                                        
Epoch 3 training loss: 0.0002.  Dice training loss: -0.0004.                                                                                                                                        
Epoch 3 validation loss: 0.0004.        Dice validation loss: -0.0011.                                                                                                                              
Epoch 3 took 597.11 seconds.                                                                                                                                                                        
Epoch 4 training loss: 0.0003.  Dice training loss: -0.0010.                                                                                                                                        
Epoch 4 validation loss: 0.0002.        Dice validation loss: 0.0015.                                                                                                                               
Epoch 4 took 591.33 seconds.                                                                                                                                                                        
Training:  50%|██████████████████████████████████████████████████████████████████████████▌                                                                          | 5/10 [39:35<49:27, 593.59s/it]

@rezazad68
Copy link
Collaborator Author

rezazad68 commented Jun 28, 2021

This is maybe more a question for the core dev of ivadomed, but I launched this:

(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2

Expecting to get some fancy GIF files, but nothing there after 4 epochs 🤷‍♂️

julien-macbook:~/temp/20210625_reza $ find . -name *.gif
julien-macbook:~/temp/20210625_reza $ 
Terminal output
(venv) julien-macbook:~/code/ivadomed $ ivadomed --train -c ivadomed/config/config_pose_julien.json --gif 2
Output path already exists: /Users/julien/temp/20210625_reza
Cuda is not available.
Working on cpu.

Selected architecture: HourglassNet, with the following parameters:
	dropout_rate: 0.3
	bn_momentum: 0.1
	depth: 3
	is_2d: True
	folder_name: pose_model
	in_channel: 1
	out_channel: 1
Missing derivatives for sub-amu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-amu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist01_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist02_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist03_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist04_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist05_T2w.nii.gz. Skipping.
Missing derivatives for sub-balgrist06_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona01_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona02_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona03_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona04_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona05_T2w.nii.gz. Skipping.
Missing derivatives for sub-barcelona06_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingGE04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio01_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio02_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio03_T2w.nii.gz. Skipping.
Missing derivatives for sub-beijingVerio04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoCeitec06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-brnoUhb08_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cardiff06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb01_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb02_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb03_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb04_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb05_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb06_T2w.nii.gz. Skipping.
Missing derivatives for sub-cmrrb07_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden01_T2w.nii.gz. Skipping.
Missing derivatives for sub-dresden02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma01_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma02_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma03_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma04_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma05_T2w.nii.gz. Skipping.
Missing derivatives for sub-fslPrisma06_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-geneva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-hamburg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-juntendo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mgh06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan01_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan02_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan03_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan04_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan05_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan06_T2w.nii.gz. Skipping.
Missing derivatives for sub-milan07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniPilot1_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS07_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS08_T2w.nii.gz. Skipping.
Missing derivatives for sub-mniS09_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai04_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mountSinai06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs01_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs02_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs03_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs05_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs06_T2w.nii.gz. Skipping.
Missing derivatives for sub-mpicbs07_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nottwil06_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu01_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu02_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu03_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu04_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu05_T2w.nii.gz. Skipping.
Missing derivatives for sub-nwu06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib05_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib06_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib07_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib08_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib09_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib10_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordFmrib11_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba01_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba02_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba03_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba04_T2w.nii.gz. Skipping.
Missing derivatives for sub-oxfordOhba05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-pavia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform01_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform02_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform03_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform04_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform05_T2w.nii.gz. Skipping.
Missing derivatives for sub-perform06_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland01_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland02_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland03_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland04_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland05_T2w.nii.gz. Skipping.
Missing derivatives for sub-queensland06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke01_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke02_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke03_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke04_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke05_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke06_T2w.nii.gz. Skipping.
Missing derivatives for sub-sherbrooke07_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford01_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford02_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford03_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford04_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford05_T2w.nii.gz. Skipping.
Missing derivatives for sub-stanford06_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg01_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg02_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg03_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg04_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg05_T2w.nii.gz. Skipping.
Missing derivatives for sub-strasbourg06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tehranS06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyo750w07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoIngenia07_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra01_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra02_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra03_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra04_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra05_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra06_T2w.nii.gz. Skipping.
Missing derivatives for sub-tokyoSkyra07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ubc06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis06_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucdavis07_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl01_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl02_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl03_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl04_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl05_T2w.nii.gz. Skipping.
Missing derivatives for sub-ucl06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf01_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf02_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf03_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf04_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf05_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf06_T2w.nii.gz. Skipping.
Missing derivatives for sub-unf07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vallHebron07_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisAchieva06_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia01_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia02_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia03_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia04_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia05_T2w.nii.gz. Skipping.
Missing derivatives for sub-vuiisIngenia06_T2w.nii.gz. Skipping.
Dataframe has been saved in /Users/julien/temp/20210625_reza/bids_dataframe.csv.
After splitting: train, validation and test fractions are respectively 0.6, 0.2 and 0.2 of participant_id.

Selected transformations for the ['training'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}

Selected transformations for the ['validation'] dataset:
	Resample: {'wspace': 256, 'hspace': 256, 'dspace': 1, 'flag_pixel': True}
	VertebralSplitting: {'max_joint': 11, 'applied_to': ['gt']}
	NumpyToTensor: {}
	NormalizeInstance: {'applied_to': ['im']}
Loading dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 52/52 [00:00<00:00, 200.47it/s]
Loaded 51 sagittal slices for the validation set.
Loading dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 156/156 [00:00<00:00, 212.50it/s]
Loaded 151 sagittal slices for the training set.
Creating model directory: /Users/julien/temp/20210625_reza/pose_model

Initialising model's weights from scratch.

Scheduler parameters: {'name': 'CosineAnnealingLR', 'base_lr': 1e-06, 'max_lr': 0.0001}

Selected Loss: JointsMSELoss
	with the parameters: []
Training:  10%|███████████████▊                                                                                                                                              | 1/10 [00:00<?, ?it/s]/Users/julien/code/ivadomed/venv/lib/python3.8/site-packages/torch/nn/functional.py:2970: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn("Default upsampling behavior when mode={} is changed "
Epoch 1 training loss: 0.0026.  Dice training loss: -0.0010.                                                                                                                                        
Epoch 1 validation loss: 0.0003.        Dice validation loss: -0.0015.                                                                                                                              
Epoch 1 took 592.04 seconds.                                                                                                                                                                        
Epoch 2 training loss: 0.0002.  Dice training loss: -0.0235.                                                                                                                                        
Epoch 2 validation loss: 0.0002.        Dice validation loss: 0.0016.                                                                                                                               
Epoch 2 took 592.96 seconds.                                                                                                                                                                        
Epoch 3 training loss: 0.0002.  Dice training loss: -0.0004.                                                                                                                                        
Epoch 3 validation loss: 0.0004.        Dice validation loss: -0.0011.                                                                                                                              
Epoch 3 took 597.11 seconds.                                                                                                                                                                        
Epoch 4 training loss: 0.0003.  Dice training loss: -0.0010.                                                                                                                                        
Epoch 4 validation loss: 0.0002.        Dice validation loss: 0.0015.                                                                                                                               
Epoch 4 took 591.33 seconds.                                                                                                                                                                        
Training:  50%|██████████████████████████████████████████████████████████████████████████▌                                                                          | 5/10 [39:35<49:27, 593.59s/it]

Ivadomed saves the gif file at the end of the training. It saves the result at path_output

@rezazad68
Copy link
Collaborator Author

rezazad68 commented Jun 28, 2021

From your terminal, it seems no mask did not find for the data. Please double-check you have the generated mid heatmaps. The output should be similar to the following one and at the end, it should generate the gif files.

image

Copy link
Member

@joshuacwnewton joshuacwnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review is to help tidy up the PR. The suggestions here will help make the PR easier to review, so that reviewers can focus on the more important parts.

In the future, I would recommend trying to prepare your PR for review by removing these "noise" changes in advance. The article "How to pull request" also provides some important tips. The most relevant point here is "3. Go back and remove noise":

Often while self reviewing, you come across a file with only white space changes, formatting changes, optimized imports or some text change that has no impact related to the intention of the PR. Make the effort, go back to your branch and revert these files to how they were before, it doesn’t matter that you have slightly improved them — functionally they are the same, and the extra file in the list of ‘changed files’ listed in the PR provokes groans from the reviewer and lowers their motivation to do a proper review — especially on larger PRs.

Once your PR has been cleaned up and prepared for review, please "Re-request review" and I will be happy to provide feedback on the more critical parts of the PR. 🙂

@@ -12,6 +11,7 @@
from scipy.ndimage.measurements import label, center_of_mass
from scipy.ndimage.morphology import binary_dilation, binary_fill_holes, binary_closing
from skimage.exposure import equalize_adapthist
from skimage import measure
from torchvision import transforms as torchvision_transforms

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, your branch is conflicting with the master branch.

Screenshot from 2021-07-02 11-08-22

Could you please merge the changes from the branch ivadomed:master into your branch, and resolve the conflicts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, in the future, could you please check for these conflicts and fix them when you see them?

Ideally, the PR should never have conflicts, so when you see this message on your PR, it is helpful to try fix it immediately (so that the people reviewing your PR won't have to take the time to ask).

ivadomed/utils.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Outdated Show resolved Hide resolved
ivadomed/utils.py Outdated Show resolved Hide resolved
ivadomed/transforms.py Show resolved Hide resolved
ivadomed/scripts/prepare_dataset_vertebral_labeling.py Outdated Show resolved Hide resolved
ivadomed/scripts/prepare_dataset_vertebral_labeling.py Outdated Show resolved Hide resolved
@jcohenadad
Copy link
Member

@rezazad68 can you please fix the conflicts:
image

thank you

@jcohenadad
Copy link
Member

I'm testing 2cdc40a, and the usage showed here does not appear:

(IvadoMedEnv) p101317@romane:~/code/ivadomed$ python3 ivadomed/scripts/prepare_dataset_vertebral_labeling.py  -h
2021-07-05 21:22:23.447 | INFO     | ivadomed.utils:init_ivadomed:401 - 
ivadomed (git-rezazad68-master-2cdc40ad8e73cf0a8cc0aeac419c6dc739a9c325*)

usage: prepare_dataset_vertebral_labeling.py [-h] -p <file> -s <str> [-a <int>]

optional arguments:
  -h, --help            show this help message and exit
  -p <file>, --path <file>
                        Path to bids folder
  -s <str>, --suffix <str>
                        Suffix of the input file as in sub-xxxSUFFIX.nii.gz (E.g., _T2w)
  -a <int>, --aim <int>
                        -1 or positive int. If set to any positive int, only label with this value will be taken into account

@rezazad68 are you sure you pushed your changes with the modified usage?

joshuacwnewton pushed a commit that referenced this pull request Jul 6, 2021
This commit contains all commits between:
  - 2b0b5ba (Start)
  - 8d7d45b (End)

This commit was also rebased onto ccd82d2
from the master branch, because that commit was the last point at which
master was merged into Reza's branch. (See: a502b37)

All former commits are preserved on the branch 'ra/intervertebral-disc-labeling'.

Co-authored-by: Reza Azad <rezazad68@gmail.com>
@rezazad68
Copy link
Collaborator Author

I'm testing 2cdc40a, and the usage showed here does not appear:

(IvadoMedEnv) p101317@romane:~/code/ivadomed$ python3 ivadomed/scripts/prepare_dataset_vertebral_labeling.py  -h
2021-07-05 21:22:23.447 | INFO     | ivadomed.utils:init_ivadomed:401 - 
ivadomed (git-rezazad68-master-2cdc40ad8e73cf0a8cc0aeac419c6dc739a9c325*)

usage: prepare_dataset_vertebral_labeling.py [-h] -p <file> -s <str> [-a <int>]

optional arguments:
  -h, --help            show this help message and exit
  -p <file>, --path <file>
                        Path to bids folder
  -s <str>, --suffix <str>
                        Suffix of the input file as in sub-xxxSUFFIX.nii.gz (E.g., _T2w)
  -a <int>, --aim <int>
                        -1 or positive int. If set to any positive int, only label with this value will be taken into account

@rezazad68 are you sure you pushed your changes with the modified usage?

I think you mentioned the wrong commit. Please use the last version 8de3c1e

@joshuacwnewton
Copy link
Member

As discussed with @jcohenadad, because this PR was created from the master branch on a fork, the easiest way to move forward with the work is to remake the PR using a branch created on ivadomed.

So, I'm going to close this PR, as it is now a duplicate of #852.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deep learning Any topics touching PyTorch and deep learning concepts, requiring additional ML/DL expertise feature category: new functionality refactoring PRs/Issues that solely deal with existing code reorganization
Projects
None yet
8 participants