Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MNI reslicing failure #703

Closed
chrisgorgo opened this issue Sep 14, 2017 · 15 comments
Closed

MNI reslicing failure #703

chrisgorgo opened this issue Sep 14, 2017 · 15 comments
Assignees
@oesteban
Labels
bug qform/sform issue
Milestone
Phase I

Comments

@chrisgorgo
Copy link
Contributor

The output images are not aligned with MNI template
image

The aCompCor ROIs are also offset
image

More details at https://neurostars.org/t/fmriprep-images-not-aligned-to-mni-coordinates/772

Might be related to: https://neurostars.org/t/fmriprep-tcompcorr-crashes-on-empty-array/668/4?u=chrisgorgolewski
@chrisgorgo chrisgorgo added the bug label Sep 14, 2017
@oesteban oesteban added this to the 1.0.0-rc5 milestone Sep 14, 2017
@oesteban oesteban self-assigned this Sep 14, 2017
@oesteban oesteban modified the milestones: 1.0.0-rc5, Phase I Sep 14, 2017
@chrisgorgo
Copy link
Contributor Author

Example data needed to replicate: https://neurostars.org/t/fmriprep-images-not-aligned-to-mni-coordinates/772/10?u=chrisgorgolewski

@oesteban
Copy link
Member

Running that dataset on Stampede2 ...

@oesteban
Copy link
Member

It seems to have gotten stalled here:

         Running: fslmaths /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/atlas_2_ref/fmap_atlas_trans.nii.gz -thr 3 -bin /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/threshold_atlas/fmap_atlas_trans_maths.nii.gz -odt char
170915-10:26:31,91 workflow INFO:
         [Job finished] jobname: threshold_atlas jobid: 412
170915-10:26:39,494 workflow INFO:
         Executing: fixed_image_masks ID: 413
170915-10:26:40,250 workflow INFO:
         Executing node fixed_image_masks in dir: /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/fixed_image_masks
170915-10:26:49,587 workflow INFO:
         [Job finished] jobname: fixed_image_masks jobid: 413
170915-10:26:58,363 workflow INFO:
         [Job finished] jobname: t1_2_ref jobid: 495
170915-10:27:06,199 workflow INFO:
         Executing: syn_i ID: 415
170915-10:27:08,22 workflow INFO:
         Executing node syn_i in dir: /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/syn_i
170915-10:27:10,446 workflow INFO:
         Running: antsRegistration --collapse-output-transforms 1 --dimensionality 3 --float 1 --initialize-transforms-per-stage 0 --interpolation Linear --output [ ants_susceptibility, ants_susceptibility_Warped.nii.gz ] --transform SyN[ 0.8, 2.0, 2.0 ] --metric Mattes[ /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/t1_2_ref/highres001_BrainExtractionBrain_inv_trans.nii.gz, /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/bold_reference_wf/enhance_and_skullstrip_bold_wf/apply_mask/uni_masked.nii.gz, 1, 56 ] --convergence [ 100x50, 1e-08, 20 ] --smoothing-sigmas 1.0x0.0vox --shrink-factors 2x1 --use-estimate-learning-rate-once 1 --use-histogram-matching 1 --restrict-deformation 1x0x0 --masks [ NULL, NULL ] --transform SyN[ 0.8, 2.0, 2.0 ] --metric CC[ /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/t1_2_ref/highres001_BrainExtractionBrain_inv_trans.nii.gz, /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/bold_reference_wf/enhance_and_skullstrip_bold_wf/apply_mask/uni_masked.nii.gz, 1, 5 ] --convergence [ 20x10, 1e-08, 10 ] --smoothing-sigmas 1.0x0.0vox --shrink-factors 1x1 --use-estimate-learning-rate-once 1 --use-histogram-matching 1 --restrict-deformation 1x0x0 --masks [ /work/03843/crn_plab/stampede2/issue-703/work/fmriprep_wf/single_subject_008_wf/func_preproc_ses_01_task_probe_run_02_wf/nonlinear_sdc_wf/threshold_atlas/fmap_atlas_trans_maths.nii.gz, NULL ] --winsorize-image-intensities [ 0.001, 1.0 ]  --write-composite-transform 0

A top on the compute node says:

top - 10:22:02 up 58 days, 16:08,  1 user,  load average: 44.62, 40.81, 39.84
Tasks: 2566 total,   7 running, 2559 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.7 us,  0.0 sy,  0.0 ni, 99.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 98696408 total, 79905656 free, 13082096 used,  5708656 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 81268768 avail Mem 

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                     
234176 oesteban  20   0 2470844 1.716g   3420 R 100.0  1.8 197:56.52 mri_ca_label                                                                                                                                
168739 oesteban  20   0 1063304 276560  17980 R  26.2  0.3  77:57.26 antsRegistratio                                                                                                                             
 98819 oesteban  20   0 1024936 238572  17980 R  24.0  0.2  99:52.01 antsRegistratio                                                                                                                             
128088 oesteban  20   0 1060464 273964  17980 R  24.0  0.3  95:24.97 antsRegistratio                                                                                                                             
190325 oesteban  20   0 1071828 285340  17980 R  24.0  0.3  70:02.92 antsRegistratio                                                                                                                             
263929 oesteban  20   0  164444   4916   1592 R   6.5  0.0   0:02.84 top                                                                                                                                         
197933 oesteban  20   0 1294812 618960  22592 R   3.1  0.6  55:27.13 fmriprep

If I'm not wrong, those load averages for a 48-core system would be okay. It doesn't seem to have hit swap and the %CPU should also be fine for 48-cores. Switching of the Irix mode of top gives me 0.4 usage for mri_ca_label and 0.1 for those antsRegistration processes.

I'll leave it to run for a bit longer and keep you updated.

@oesteban
Copy link
Member

Good news: fmriprep passed that point :)

This was referenced Sep 18, 2017
Closed
Closed
@oesteban
Copy link
Member

@chrisfilo could you point me to the task and run that you used to generate those plots?

@oesteban
Copy link
Member

For some reason I'm not able to replicate this.

2mm_t14

aCompCorr also looks good

@effigies
Copy link
Member

effigies commented Sep 20, 2017 •
edited
Loading

Hmm... I wonder if this was incidentally fixed in #662. Possible if they were running 1.0.0-rc1 or earlier.

@chrisgorgo
Copy link
Contributor Author

chrisgorgo commented Sep 20, 2017 via email

@oesteban
Copy link
Member

oesteban commented Sep 20, 2017 •
edited
Loading

This is a multisession dataset. If the default is using all the T1w images, then yes, I'm using all of them.

EDIT: there are two T1s

@oesteban
Copy link
Member

@effigies they were running 1.0.0-rc2 (https://neurostars.org/t/fmriprep-images-not-aligned-to-mni-coordinates/772/13?u=oesteban)

@effigies
Copy link
Member

Yeah, I saw that. The biggest potentially relevant changes since then were #666 and #695.

@effigies
Copy link
Member

Full comparison, btw: 1.0.0-rc2...1.0.0-rc4

@effigies
Copy link
Member

Would it be worth making a backup copy of the scratch directory (cp -al will preserve timestamps and use hardlinks) and re-running with 1.0.0-rc2, just to verify that the issue was resolved somehow between then and now?

@oesteban
Copy link
Member

Still unable to replicate. Now with FMRIPREP 1.0.0-rc2
issue703-1 0 0-rc2

@effigies
Copy link
Member

Resolved by running with a fresh working directory.

@oesteban oesteban mentioned this issue Oct 10, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@oesteban oesteban

Labels
bug qform/sform issue
Projects
None yet
Milestone
Phase I
Development

No branches or pull requests
3 participants
@effigies @chrisgorgo @oesteban