Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Ignore sourcedata and derivatives when fetching data #1788

Merged
merged 1 commit into from Sep 26, 2019

Conversation

effigies
Copy link
Member

Changes proposed in this pull request

Closes #1787.

Documentation that should be reviewed

@effigies
Copy link
Member Author

@yarikoptic Would you care to give this branch a try?

@yarikoptic
Copy link
Contributor

oh... I run fmriprep as released in a container. would take me a bit to do otherwise. if you have such environment ready, could you please just try running that script (removing singularity shimming) I provided in #1787 -- it should be quite straightforward

@effigies
Copy link
Member Author

If you use fmriprep-docker, you can use the -f flag to point to a live repo. e.g.,

fmriprep-docker <all> <my> <usual> <args> -f /data/projects/fmriprep/fmriprep

(Note that you need to target the Python source directory, not the repository root.)

@effigies
Copy link
Member Author

(I'll give your script a try, though.)

@yarikoptic
Copy link
Contributor

If you use fmriprep-docker, you can use the -f flag to point to a live repo. e.g.,

fmriprep-docker <all> <my> <usual> <args> -f /data/projects/fmriprep/fmriprep

that is neat! haven't used fmriprep-docker yet... will check it out later, thanks!

@effigies
Copy link
Member Author

Updated script to:

> cat replicate-fmriprepkaboom.sh 
#!/bin/bash
set -eu
datalad install ///labs/haxby/life
datalad get life/sub-rid000001  # one subject to start with

export FS_LICENSE=/Users/markiewicz/Documents/freesurfer_license.txt

mkdir -p derivatives/{work,fmriprep-1.5.0}
# manual execution to ease troubleshooting
#  --skip-bids-validation  since we did not get/process all the subjects, and 
#   https://github.com/bids-standard/bids-validator/issues/769 still scares me a lot ;)
fmriprep-docker life derivatives/fmriprep-1.5.0/ participant --skip-bids-validation \
    --participant-label rid000001 -w derivatives/work \
    --notrack --skull-strip-fixed-seed \
    --ignore slicetiming --fs-license-file="$FS_LICENSE" \
    --bold2t1w-dof 9 --output-space fsaverage5 T1w \
    --n_cpus 2 --omp-nthreads 1 $@

To get to the error, I ran:

> bash replicate-fmriprepkaboom.sh --anat-only
Producing (click to expand)
[INFO   ] Cloning http://datasets.datalad.org/labs/haxby/life [1 other candidates] into '/Users/markiewicz/Projects/crn/fmriprep/life' 
install(ok): /Users/markiewicz/Projects/crn/fmriprep/life (dataset)                                                                   
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/anat/sub-rid000001_T1w_defacemask.nii.gz (file) [from origin...]  
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/anat/sub-rid000001_T1w.nii.gz (file) [from origin...]             
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/func/sub-rid000001_task-life_acq-346vol_run-02_bold.nii.gz (file) [from origin...]nc/sub-rid000001_task-life_acq-412vol_run-04_bold.nii.gz:  70%|████████████▌     | 68.6M/98.2M [00:07<00:02, 10.6MB/s]
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/func/sub-rid000001_task-life_acq-377vol_run-03_bold.nii.gz (file) [from origin...]
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/func/sub-rid000001_task-life_acq-374vol_run-01_bold.nii.gz (file) [from origin...]
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001/func/sub-rid000001_task-life_acq-412vol_run-04_bold.nii.gz (file) [from origin...]
get(ok): /Users/markiewicz/Projects/crn/fmriprep/life/sub-rid000001 (directory)
action summary:
  get (ok: 7)
RUNNING: docker run --rm -it -e DOCKER_VERSION_8395080871=19.03.2 -v /Users/markiewicz/Documents/freesurfer_license.txt:/opt/freesurfer/license.txt:ro -v /Users/markiewicz/Projects/crn/fmriprep/life:/data:ro -v /Users/markiewicz/Projects/crn/fmriprep/derivatives/fmriprep-1.5.0:/out -v /Users/markiewicz/Projects/crn/fmriprep/derivatives/work:/scratch poldracklab/fmriprep:1.5.0 /data /out participant --skip-bids-validation --participant-label rid000001 --notrack --skull-strip-fixed-seed --ignore slicetiming --bold2t1w-dof 9 --output-space fsaverage5 T1w --n_cpus 2 --omp-nthreads 1 --anat-only -w /scratch
The ``--output_space`` option has been deprecated in version 1.4.0. Your selection of spaces "fsaverage5, T1w" will be inserted at the front of the ``--output-spaces`` argument list. Please update your scripts to use ``--output-spaces``.
190925-15:30:02,643 nipype.workflow IMPORTANT:
	 
    Running fMRIPREP version 1.5.0:
      * BIDS dataset path: /data.
      * Participant list: ['rid000001'].
      * Run identifier: 20190925-152957_e9748c5d-4333-4942-bc32-daf65ed3c1ca.
    
190925-15:30:08,887 nipype.workflow IMPORTANT:
	 Works derived from this fMRIPrep execution should include the following boilerplate:


Results included in this manuscript come from preprocessing
performed using *fMRIPrep* 1.5.0
(@fmriprep1; @fmriprep2; RRID:SCR_016216),
which is based on *Nipype* 1.2.2
(@nipype1; @nipype2; RRID:SCR_002502).

Anatomical data preprocessing

: A total of 2 T1-weighted (T1w) images were found within the input
BIDS dataset.
All of them were corrected for intensity non-uniformity (INU)
with `N4BiasFieldCorrection` [@n4], distributed with ANTs 2.2.0 [@ants, RRID:SCR_004757].
The T1w-reference was then skull-stripped with a *Nipype* implementation of
the `antsBrainExtraction.sh` workflow (from ANTs), using OASIS30ANTs
as target template.
Brain tissue segmentation of cerebrospinal fluid (CSF),
white-matter (WM) and gray-matter (GM) was performed on
the brain-extracted T1w using `fast` [FSL 5.0.9, RRID:SCR_002823,
@fsl_fast].
A T1w-reference map was computed after registration of
2 T1w images (after INU-correction) using
`mri_robust_template` [FreeSurfer 6.0.1, @fs_template].
Brain surfaces were reconstructed using `recon-all` [FreeSurfer 6.0.1,
RRID:SCR_001847, @fs_reconall], and the brain mask estimated
previously was refined with a custom variation of the method to reconcile
ANTs-derived and FreeSurfer-derived segmentations of the cortical
gray-matter of Mindboggle [RRID:SCR_002438, @mindboggle].
Volume-based spatial normalization to one standard space (MNI152NLin2009cAsym) was performed through
nonlinear registration with `antsRegistration` (ANTs 2.2.0),
using brain-extracted versions of both T1w reference and the T1w template.
The following template was selected for spatial normalization:
*ICBM 152 Nonlinear Asymmetrical template version 2009c* [@mni152nlin2009casym, RRID:SCR_008796; TemplateFlow ID: MNI152NLin2009cAsym].

Many internal operations of *fMRIPrep* use
*Nilearn* 0.5.2 [@nilearn, RRID:SCR_001362],
mostly within the functional processing workflow.
For more details of the pipeline, see [the section corresponding
to workflows in *fMRIPrep*'s documentation](https://fmriprep.readthedocs.io/en/latest/workflows.html "FMRIPrep's documentation").


### Copyright Waiver

The above boilerplate text was automatically generated by fMRIPrep
with the express intention that users should copy and paste this
text into their manuscripts *unchanged*.
It is released under the [CC0](https://creativecommons.org/publicdomain/zero/1.0/) license.

### References


190925-15:30:14,393 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.lap_tmpl" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/anat_preproc_wf/brain_extraction_wf/lap_tmpl".
190925-15:30:14,418 nipype.workflow INFO:
	 [Node] Running "lap_tmpl" ("niworkflows.interfaces.ants.ImageMath"), a CommandLine Interface with command:
ImageMath 3 tpl-OASIS30ANTs_res-01_T1w_maths.nii.gz Laplacian /home/fmriprep/.cache/templateflow/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz 1.5 1
190925-15:30:20,308 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.res_tmpl" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/anat_preproc_wf/brain_extraction_wf/res_tmpl".
190925-15:30:20,326 nipype.workflow INFO:
	 [Node] Running "res_tmpl" ("niworkflows.interfaces.ants.ResampleImageBySpacing"), a CommandLine Interface with command:
ResampleImageBySpacing 3 /home/fmriprep/.cache/templateflow/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz tpl-OASIS30ANTs_res-01_T1w_resampled.nii.gz 4 4 4 1
190925-15:30:22,236 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.res_tmpl".
190925-15:30:23,364 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.lap_tmpl".
190925-15:30:23,855 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.mrg_tmpl" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/anat_preproc_wf/brain_extraction_wf/mrg_tmpl".
190925-15:30:23,877 nipype.workflow INFO:
	 [Node] Running "mrg_tmpl" ("nipype.interfaces.utility.base.Merge")
190925-15:30:23,925 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.brain_extraction_wf.mrg_tmpl".
190925-15:30:23,931 nipype.workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_rid000001_wf.about" (/scratch/fmriprep_wf/single_subject_rid000001_wf/about)
190925-15:30:23,943 nipype.workflow ERROR:
	 Node about failed to run on host 1785a13b45b8.
190925-15:30:23,952 nipype.workflow ERROR:
	 Saving crash info to /out/fmriprep/sub-rid000001/log/20190925-152957_e9748c5d-4333-4942-bc32-daf65ed3c1ca/crash-20190925-153023-root-about-571a4129-b575-44b4-b145-583c4f7c976a.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 316, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 564, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 671, in _run_command
    rebase=str2bool(self.config['execution']['use_relative_paths']))
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 264, in save_resultfile
    setattr(result.outputs, key, val)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 317, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 144, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: The 'out_report' trait of a SummaryOutputSpec instance must be a pathlike object or string representing an existing file, but a value of 'report.html' <class 'str'> was specified.

190925-15:30:24,235 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.bidssrc" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/bidssrc".
190925-15:30:24,254 nipype.workflow INFO:
	 [Node] Running "bidssrc" ("niworkflows.interfaces.bids.BIDSDataGrabber")
190925-15:30:24,257 nipype.interface WARNING:
	 No "t2w" images found for sub-<undefined>
190925-15:30:24,258 nipype.interface WARNING:
	 No "flair" images found for sub-<undefined>
190925-15:30:24,260 nipype.interface WARNING:
	 No "fmap" images found for sub-<undefined>
190925-15:30:24,262 nipype.interface WARNING:
	 No "sbref" images found for sub-<undefined>
190925-15:30:24,263 nipype.interface WARNING:
	 No "roi" images found for sub-<undefined>
190925-15:30:24,312 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_rid000001_wf.bidssrc".
190925-15:30:25,860 nipype.workflow WARNING:
	 Error while checking node hash, forcing re-run. Although this error may not prevent the workflow from running, it could indicate a major problem. Please report a new issue at https://github.com/nipy/nipype/issues adding the following information:

	Node: fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.anat_template_wf.t1_template_dimensions
	Interface: niworkflows.interfaces.images.TemplateDimensions
	Traceback:
Traceback (most recent call last):

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/base.py", line 337, in _local_hash_check
    cached, updated = self.procs[jobid].is_cached()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 297, in is_cached
    hashed_inputs, hashvalue = self._get_hashval()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 494, in _get_hashval
    self._get_inputs()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 541, in _get_inputs
    self.set_input(key, deepcopy(output_value))

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 279, in set_input
    setattr(self.inputs, parameter, deepcopy(val))

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 400, in validate
    value = super(MultiObject, self).validate(objekt, name, newvalue)

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_types.py", line 2336, in validate
    return TraitListObject( self, object, name, value )

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2313, in __init__
    raise excp

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in __init__
    value = [ validate( object, name, val ) for val in value ]

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in <listcomp>
    value = [ validate( object, name, val ) for val in value ]

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 317, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 141, in validate
    self.error(objekt, name, str(value))

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )

traits.trait_errors.TraitError: Each element of the 't1w_list' trait of a TemplateDimensionsInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz' <class 'str'> was specified.

Error setting node input:
Node: t1_template_dimensions
input: t1w_list
results_file: /scratch/fmriprep_wf/single_subject_rid000001_wf/bidssrc/result_bidssrc.pklz
value: ['/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz', '/data/sub-rid000001/anat/sub-rid000001_T1w.nii.gz']

190925-15:30:25,880 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.anat_template_wf.t1_template_dimensions" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/anat_preproc_wf/anat_template_wf/t1_template_dimensions".
190925-15:30:25,887 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_wf.single_subject_rid000001_wf.bids_info" in "/scratch/fmriprep_wf/single_subject_rid000001_wf/bids_info".
190925-15:30:25,926 nipype.workflow INFO:
	 [Node] Running "bids_info" ("niworkflows.interfaces.bids.BIDSInfo")
190925-15:30:27,767 nipype.workflow ERROR:
	 Node t1_template_dimensions failed to run on host 1785a13b45b8.
190925-15:30:27,769 nipype.workflow ERROR:
	 Saving crash info to /out/fmriprep/sub-rid000001/log/20190925-152957_e9748c5d-4333-4942-bc32-daf65ed3c1ca/crash-20190925-153027-root-t1_template_dimensions-7717ba8b-fe5d-4166-a7ff-db41ed0267ec.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 411, in run
    cached, updated = self.is_cached()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 297, in is_cached
    hashed_inputs, hashvalue = self._get_hashval()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 494, in _get_hashval
    self._get_inputs()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 541, in _get_inputs
    self.set_input(key, deepcopy(output_value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 279, in set_input
    setattr(self.inputs, parameter, deepcopy(val))
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 400, in validate
    value = super(MultiObject, self).validate(objekt, name, newvalue)
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_types.py", line 2336, in validate
    return TraitListObject( self, object, name, value )
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2313, in __init__
    raise excp
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in __init__
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in <listcomp>
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 317, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 141, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: Each element of the 't1w_list' trait of a TemplateDimensionsInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz' <class 'str'> was specified.

Error setting node input:
Node: t1_template_dimensions
input: t1w_list
results_file: /scratch/fmriprep_wf/single_subject_rid000001_wf/bidssrc/result_bidssrc.pklz
value: ['/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz', '/data/sub-rid000001/anat/sub-rid000001_T1w.nii.gz']

190925-15:30:29,666 nipype.workflow INFO:
	 [Node] Finished "fmriprep_wf.single_subject_rid000001_wf.bids_info".
190925-15:30:51,160 nipype.workflow WARNING:
	 Error while checking node hash, forcing re-run. Although this error may not prevent the workflow from running, it could indicate a major problem. Please report a new issue at https://github.com/nipy/nipype/issues adding the following information:

	Node: fmriprep_wf.single_subject_rid000001_wf.summary
	Interface: fmriprep.interfaces.reports.SubjectSummary
	Traceback:
Traceback (most recent call last):

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/base.py", line 337, in _local_hash_check
    cached, updated = self.procs[jobid].is_cached()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 297, in is_cached
    hashed_inputs, hashvalue = self._get_hashval()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 494, in _get_hashval
    self._get_inputs()

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 541, in _get_inputs
    self.set_input(key, deepcopy(output_value))

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 279, in set_input
    setattr(self.inputs, parameter, deepcopy(val))

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 400, in validate
    value = super(MultiObject, self).validate(objekt, name, newvalue)

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_types.py", line 2336, in validate
    return TraitListObject( self, object, name, value )

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2313, in __init__
    raise excp

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in __init__
    value = [ validate( object, name, val ) for val in value ]

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in <listcomp>
    value = [ validate( object, name, val ) for val in value ]

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 317, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)

  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 141, in validate
    self.error(objekt, name, str(value))

  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )

traits.trait_errors.TraitError: Each element of the 't1w' trait of a SubjectSummaryInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz' <class 'str'> was specified.

Error setting node input:
Node: summary
input: t1w
results_file: /scratch/fmriprep_wf/single_subject_rid000001_wf/bidssrc/result_bidssrc.pklz
value: ['/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz', '/data/sub-rid000001/anat/sub-rid000001_T1w.nii.gz']

190925-15:30:51,174 nipype.workflow ERROR:
	 Node summary failed to run on host 1785a13b45b8.
190925-15:30:51,177 nipype.workflow ERROR:
	 Saving crash info to /out/fmriprep/sub-rid000001/log/20190925-152957_e9748c5d-4333-4942-bc32-daf65ed3c1ca/crash-20190925-153051-root-summary-2d0d60e5-ec13-4e55-abcc-03e62cc57bf4.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 316, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 411, in run
    cached, updated = self.is_cached()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 297, in is_cached
    hashed_inputs, hashvalue = self._get_hashval()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 494, in _get_hashval
    self._get_inputs()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 541, in _get_inputs
    self.set_input(key, deepcopy(output_value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 279, in set_input
    setattr(self.inputs, parameter, deepcopy(val))
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 400, in validate
    value = super(MultiObject, self).validate(objekt, name, newvalue)
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_types.py", line 2336, in validate
    return TraitListObject( self, object, name, value )
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2313, in __init__
    raise excp
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in __init__
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in <listcomp>
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 317, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 141, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: Each element of the 't1w' trait of a SubjectSummaryInputSpec instance must be a pathlike object or string representing an existing file, but a value of '/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz' <class 'str'> was specified.

Error setting node input:
Node: summary
input: t1w
results_file: /scratch/fmriprep_wf/single_subject_rid000001_wf/bidssrc/result_bidssrc.pklz
value: ['/data/sourcedata/sub-rid000001/anat/sub-rid000001_T1w.nii.gz', '/data/sub-rid000001/anat/sub-rid000001_T1w.nii.gz']

190925-15:30:53,53 nipype.workflow ERROR:
	 could not run node: fmriprep_wf.single_subject_rid000001_wf.anat_preproc_wf.anat_template_wf.t1_template_dimensions
fMRIPrep failed: Workflow did not execute cleanly. Check log for details
Preprocessing did not finish successfully. Errors occurred while processing data from participants: rid000001 (3). Check the HTML reports for details.
fMRIPrep: Please report errors to https://github.com/poldracklab/fmriprep/issues

Running with

> bash replicate-fmriprepkaboom.sh --anat-only -f /Users/markiewicz/Projects/crn/fmriprep/fmriprep

Continues past that point. So I'll go ahead and merge this once tests pass.

@oesteban A note that rerunning with the fix and a working directory that has a crashfile in it produced

190925-15:34:45,437 nipype.workflow ERROR:
	 Saving crash info to /out/fmriprep/sub-rid000001/log/20190925-153421_187a8542-9cd8-4a3c-81c8-e0fecbd06160/crash-20190925-153445-root-t1_conform-9a81870d-d191-41df-8742-af50f01d8c43.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 269, in _send_procs_to_workers
    num_subnodes = self.procs[jobid].num_subnodes()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 1204, in num_subnodes
    self._get_inputs()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 1219, in _get_inputs
    super(MapNode, self)._get_inputs()
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 521, in _get_inputs
    outputs = _load_resultfile(results_file)[0].outputs
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py", line 311, in load_resultfile
    if resolve and result.outputs:
AttributeError: 'NoneType' object has no attribute 'outputs'

Deleting the working directory resolved it. fMRIPrep reports using Nipype 1.2.2.

@effigies effigies merged commit 17e9732 into nipreps:master Sep 26, 2019
@effigies effigies added this to the 1.5.1 milestone Sep 26, 2019
@yarikoptic
Copy link
Contributor

so, the fix came without a test to solidify correct behavior... I guess it might come back to bite us.

@effigies effigies deleted the fix/ignore_sourcedata_etc branch August 24, 2023 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tries to consider sourcedata/ (for T1s)
3 participants