Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: generate CIFTI derivatives #1001

Merged
merged 66 commits into from
Apr 20, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
b12072d
enh: generate CIFTI derivatives
mgxd Feb 21, 2018
e1b57d5
[ci skip] enh: start adding to workflow
mgxd Feb 21, 2018
0230ab4
fix: some missing connections, addressing @effigies review
mgxd Feb 22, 2018
d222aef
test: add cifti to ds005 testing
mgxd Feb 23, 2018
96cff8c
enh+fix: save output as valid CIFTI dtseries
mgxd Feb 27, 2018
270d16e
sty: removed blank line [skip ci]
mgxd Feb 27, 2018
622ce39
Merge branch 'master' into enh/cifti
mgxd Feb 27, 2018
4dd53c8
sty: flake8 fixes [skip ci]
mgxd Feb 28, 2018
00e8e74
Merge branch 'enh/cifti' of https://github.com/mgxd/fmriprep into enh…
mgxd Feb 28, 2018
87f29a4
PIN: poldracklab/niworkflows master
effigies Feb 28, 2018
3606d6e
Merge remote-tracking branch 'upstream/master' into enh/cifti
effigies Feb 28, 2018
f1036be
fix: variable name
mgxd Feb 28, 2018
c214e28
fix: add cifti_output to function docstring
mgxd Feb 28, 2018
1dff620
enh: upgrade to niworkflows 0.3.5
mgxd Feb 28, 2018
da37487
fix: connection syntax
mgxd Feb 28, 2018
f2949bf
PIN: niworkflows 0.3.5 until pip installable
mgxd Feb 28, 2018
a8a9bb7
enh: fix resource name, add docstring to create_cifti_image
mgxd Feb 28, 2018
6af5486
enh: upgrade niworkflows
mgxd Feb 28, 2018
6788741
doc: add documentation for cifti output
mgxd Feb 28, 2018
0835807
fix: download path for template
mgxd Feb 28, 2018
eff36ee
fix: syntax
mgxd Feb 28, 2018
26e5ad8
tst: add missing parameter
mgxd Feb 28, 2018
a6b7f85
fix: ensure BOLD/atlas have same res
mgxd Mar 9, 2018
7a45f34
fix: typo
mgxd Mar 9, 2018
1b2bf32
fix: more flexible approach if dealing with multiple labels
mgxd Mar 9, 2018
4004d3b
fix: output CIFTI derivative
mgxd Mar 9, 2018
1eef747
enh: add option for uncompressed niftis
mgxd Mar 9, 2018
ee1abf5
unpin for niworkflows 0.3.5
mgxd Mar 9, 2018
d86a691
style fix
mgxd Mar 9, 2018
4c4f449
fix: add missing argument
mgxd Mar 9, 2018
e77ba64
fix: add derivatives source file
mgxd Mar 10, 2018
c922a85
sty: improve derivatives (de)compression options
mgxd Mar 12, 2018
20bb2bf
Merge branch 'enh/cifti' of github.com:mgxd/fmriprep into enh/cifti
mgxd Mar 12, 2018
60905a3
fix: ensure compression if unchanged from input
mgxd Mar 15, 2018
0478645
Merge branch 'master' of git://github.com/poldracklab/fmriprep into e…
mgxd Mar 15, 2018
ed8b642
enh: ensure bold resampled to t1w space is compressed
mgxd Mar 15, 2018
be006b3
Merge branch 'master' of git://github.com/poldracklab/fmriprep into e…
mgxd Mar 21, 2018
05bde9c
fix: use correct model type for voxels
mgxd Mar 21, 2018
d47608a
fix: flexible for non-nii files as well
mgxd Mar 21, 2018
417cf99
enh: better naming for CIFTI outputs
mgxd Mar 27, 2018
523b132
Merge branch 'master' of git://github.com/poldracklab/fmriprep into e…
mgxd Mar 27, 2018
d1ece92
fix: convert to staticmethod
mgxd Mar 27, 2018
693e9c5
fix: add cifti variant to outputs
mgxd Mar 27, 2018
01a25c5
fix: return runtime
mgxd Mar 28, 2018
089cad0
fix: sort expected outputs
mgxd Mar 28, 2018
41456aa
tst: clean cache
mgxd Apr 5, 2018
182b10a
Merge remote-tracking branch 'upstream/master' into enh/cifti
oesteban Apr 5, 2018
5dfb902
update niworkflows pinning correctly
oesteban Apr 5, 2018
310a870
fix: dont gen cifti if no fs
mgxd Apr 5, 2018
0c468c7
fix: add argparse mutually exclusive group for xor options
mgxd Apr 6, 2018
2f374d1
Merge branch 'master' of https://github.com/poldracklab/fmriprep into…
mgxd Apr 6, 2018
4f86de2
sty: flake8 indentation
mgxd Apr 6, 2018
6b9d1e9
sty: make surface option exclusivity clearer
mgxd Apr 6, 2018
4f179b6
sty: keep node name consistent with variable
mgxd Apr 6, 2018
61b2de6
fix: remove duplicate arguments
mgxd Apr 6, 2018
6b9bc58
merge upstream
mgxd Apr 11, 2018
cd87258
enh: output supplementary json for cifti variants
mgxd Apr 11, 2018
1e67ce6
sty: forgotten comma
mgxd Apr 11, 2018
6570783
fix: make suffix iterable
mgxd Apr 11, 2018
06bdd61
sty: flake8 indentation
mgxd Apr 11, 2018
277cbc4
fix: missing connection
mgxd Apr 12, 2018
f4d83e3
tst: ds005 outputs
mgxd Apr 12, 2018
e51fa5e
merge with upstream master
mgxd Apr 17, 2018
6f64800
ci: add cifti option to rerun
mgxd Apr 19, 2018
be41721
ci: add cifti derivatives to partial outputs
mgxd Apr 19, 2018
b1ff492
fix: use continuous interpolation when resampling
mgxd Apr 20, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,8 @@ jobs:
steps:
- restore_cache:
keys:
- data-v4-{{ epoch }}
- data-v4-
- data-v5-{{ epoch }}
- data-v5-
- run:
name: Get test data from ds000005
command: |
Expand Down Expand Up @@ -151,7 +151,7 @@ jobs:
- ds054/nipype.cfg
- ds000210/nipype.cfg
- save_cache:
key: data-v4-{{ epoch }}
key: data-v5-{{ epoch }}
paths:
- /tmp/data
- /tmp/ds005/derivatives/freesurfer
Expand Down Expand Up @@ -331,7 +331,7 @@ jobs:
--ignore-aroma-denoising-errors --mem_mb 4096 \
--output-space T1w template fsaverage5 \
--template-resampling-grid native \
--nthreads 2 -vv
--nthreads 2 --cifti-output -vv
- run:
name: Move intermediate results for second run
command: |
Expand All @@ -357,7 +357,7 @@ jobs:
--ignore-aroma-denoising-errors --mem_mb 4096 \
--output-space T1w template fsaverage5 \
--template-resampling-grid native \
--nthreads 2 -vv
--nthreads 2 --cifti-output -vv
- run:
name: Checking outputs of partial fMRIPrep run
command: |
Expand Down Expand Up @@ -553,7 +553,7 @@ jobs:
/tmp/data/ds000210 /tmp/ds000210/derivatives participant \
--fs-no-reconall --t2s-coreg --use-syn-sdc \
--template-resampling-grid native \
--debug --write-graph --mem_mb 4096 --nthreads 2 -vv
--debug --write-graph --mem_mb 4096 --nthreads 2 -vv
- run:
name: Checking outputs of fMRIPrep
command: |
Expand Down
4 changes: 4 additions & 0 deletions .circleci/ds005_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ fmriprep/sub-01/func
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_confounds.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_MELODICmix.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-cifti_variant-space1_preproc.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-cifti_variant-space1_preproc.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-fsaverage5.L.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-fsaverage5.R.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
Expand All @@ -56,6 +58,8 @@ fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-T1w_preproc.
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_confounds.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_MELODICmix.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_space-cifti_variant-space1_preproc.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_space-cifti_variant-space1_preproc.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_space-fsaverage5.L.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_space-fsaverage5.R.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
Expand Down
2 changes: 2 additions & 0 deletions .circleci/ds005_partial_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ fmriprep/sub-01/func
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_confounds.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_MELODICmix.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-cifti_variant-space1_preproc.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-cifti_variant-space1_preproc.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-fsaverage5.L.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-fsaverage5.R.func.gii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz
Expand Down
2 changes: 1 addition & 1 deletion docs/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ dependencies:
- svgutils
- nitime
- nilearn
- niworkflows>=0.3.7
- niworkflows>=0.3.7
23 changes: 14 additions & 9 deletions docs/outputs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ Outputs of FMRIPREP

FMRIPREP generates three broad classes of outcomes:

1. **Visual QA (quality assessment) reports**:
1. **Visual QA (quality assessment) reports**:
one :abbr:`HTML (hypertext markup language)` per subject,
that allows the user a thorough visual assessment of the quality
of processing and ensures the transparency of fMRIPrep operation.

2. **Pre-processed imaging data** which are derivatives of the original
anatomical and functional images after various preparation procedures
have been applied. For example,
have been applied. For example,
:abbr:`INU (intensity non-uniformity)`-corrected versions of the T1-weighted
image (per subject), the brain mask, or :abbr:`BOLD (blood-oxygen level dependent)`
images after head-motion correction, slice-timing correction and aligned into
Expand All @@ -41,7 +41,7 @@ Each report is self contained and thus can be easily shared with collaborators (
Preprocessed data (fMRIPrep *derivatives*)
------------------------------------------

There are additional files, called "Derivatives", written to
There are additional files, called "Derivatives", written to
``<output dir>/fmriprep/sub-<subject_label>/``. See the
`BIDS Derivatives <https://docs.google.com/document/d/1Wwc4A6Mow4ZPPszDIWfCUCRNstn7d_zzaWPcfcHmgI4/edit?usp=sharing>`_
spec for more information.
Expand Down Expand Up @@ -87,6 +87,11 @@ Surface output spaces include ``fsnative`` (full density subject-specific mesh),

- (optional) ``*bold_space-<space>.[LR].func.gii`` Motion-corrected EPI file sampled to surface ``<space>``

EPIs can be saved as a CIFTI dtseries file.

- (optional) ``*bold_space-cifti_variant-<variant>_preproc.dtseries.nii`` Motion-corrected EPI converted to CIFTI filetype. Sub-cortical representations
are volumetric (supported spaces: ``MNI152NLin2009cAsym``), while cortical representations are sampled to surface (supported spaces: ``fsaverage5``, ``fsaverage6``)


.. _fsderivs:

Expand Down Expand Up @@ -144,7 +149,7 @@ These are :abbr:`TSV (tab-separated values)` tables, which look like the example
-1.41 0.29 0.96 13.06 0.99 0.01 -0.04 0.07 0.10 0.31 0.47 0.27 -0.22 0.09 0.11 0.12 0.56 0.14 0.00 0.00 0.00 0.07 0.00 0.00 0.00 -0.67 0.44 0.25 -0.57


Each row of the file corresponds to one time point found in the
Each row of the file corresponds to one time point found in the
corresponding :abbr:`BOLD (blood-oxygen level dependent)` time-series
(stored in ``<output_folder>/fmriprep/sub-<sub_id>/func/sub-<sub_id>_task-<task_id>_run-<run_id>_bold_preproc.nii.gz``).

Expand All @@ -154,13 +159,13 @@ the :abbr:`CSF (cerebro-spinal fluid)` and :abbr:`WM (white matter)` mask across
derivative of RMS variance over voxels (or :abbr:`DVARS (D referring to difference, )`) that can be
standardized (``stdDVARS``), non-standardized (``non-stdDVARS``), and voxel-wise standardized (``vx-wisestdDVARS``);
the ``FrameDisplacement`` is a quantification of the estimated bulk-head motion; ``X``, ``Y``, ``Z``, ``RotX``,
``RotY``, ``RotZ`` are the actual 6 rigid-body transform parameters estimated by FMRIPREP;
``RotY``, ``RotZ`` are the actual 6 rigid-body transform parameters estimated by FMRIPREP;
the ``NonSteadyStateOutlier00`` column is ``1`` when a non-steady state was found (typically at the beginning of
the scan) and ``0`` elsewhere; and finally six noise components ``aCompCorXX`` calculated using
:abbr:`CompCor (Component Based Noise Correction Method)`
and five noise components ``AROMAaggrCompXX`` if
the scan) and ``0`` elsewhere; and finally six noise components ``aCompCorXX`` calculated using
:abbr:`CompCor (Component Based Noise Correction Method)`
and five noise components ``AROMAaggrCompXX`` if
:abbr:`ICA (independent components analysis)`-:abbr:`AROMA (Automatic Removal Of Motion Artifacts)` was enabled.

All these confounds can be used to perform *scrubbing* and *censoring* of outliers,
All these confounds can be used to perform *scrubbing* and *censoring* of outliers,
in the subsequent first-level analysis when building the design matrix,
and in group level analysis.
2 changes: 2 additions & 0 deletions docs/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ is presented below:
output_spaces=['T1w', 'fsnative',
'template', 'fsaverage5'],
medial_surface_nan=False,
cifti_output=False,
ignore=[],
debug=False,
low_mem=False,
Expand Down Expand Up @@ -232,6 +233,7 @@ BOLD preprocessing
output_spaces=['T1w', 'fsnative',
'template', 'fsaverage5'],
medial_surface_nan=False,
cifti_output=False,
debug=False,
low_mem=False,
use_bbr=True,
Expand Down
21 changes: 14 additions & 7 deletions fmriprep/cli/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,18 +175,24 @@ def get_parser():

# FreeSurfer options
g_fs = parser.add_argument_group('Specific options for FreeSurfer preprocessing')
g_fs.add_argument('--fs-no-reconall', '--no-freesurfer',
action='store_false', dest='run_reconall',
help='disable FreeSurfer surface preprocessing.'
' Note : `--no-freesurfer` is deprecated and will be removed in 1.2.'
' Use `--fs-no-reconall` instead.')
g_fs.add_argument('--no-submm-recon', action='store_false', dest='hires',
help='disable sub-millimeter (hires) reconstruction')
g_fs.add_argument(
'--fs-license-file', metavar='PATH', type=os.path.abspath,
help='Path to FreeSurfer license key file. Get it (for free) by registering'
' at https://surfer.nmr.mgh.harvard.edu/registration.html')

# Surface generation xor
g_surfs = parser.add_argument_group('Surface preprocessing options')
g_surfs.add_argument('--no-submm-recon', action='store_false', dest='hires',
help='disable sub-millimeter (hires) reconstruction')
g_surfs_xor = g_surfs.add_mutually_exclusive_group()
g_surfs_xor.add_argument('--cifti-output', action='store_true', default=False,
help='output BOLD files as CIFTI dtseries')
g_surfs_xor.add_argument('--fs-no-reconall', '--no-freesurfer',
action='store_false', dest='run_reconall',
help='disable FreeSurfer surface preprocessing.'
' Note : `--no-freesurfer` is deprecated and will be removed in 1.2.'
' Use `--fs-no-reconall` instead.')

g_other = parser.add_argument_group('Other options')
g_other.add_argument('-w', '--work-dir', action='store',
help='path where intermediate results should be stored')
Expand Down Expand Up @@ -449,6 +455,7 @@ def build_workflow(opts, retval):
output_spaces=opts.output_space,
template=opts.template,
medial_surface_nan=opts.medial_surface_nan,
cifti_output=opts.cifti_output,
template_out_grid=template_out_grid,
hires=opts.hires,
use_bbr=opts.use_bbr,
Expand Down
11 changes: 7 additions & 4 deletions fmriprep/interfaces/bids.py
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,8 @@ class DerivativesDataSinkInputSpec(BaseInterfaceInputSpec):
source_file = File(exists=False, mandatory=True, desc='the input func file')
suffix = traits.Str('', mandatory=True, desc='suffix appended to source_file')
extra_values = traits.List(traits.Str)
compress = traits.Bool(desc="force compression (True) or uncompression (False)"
" of the output file (default: same as input)")


class DerivativesDataSinkOutputSpec(TraitedSpec):
Expand Down Expand Up @@ -202,9 +204,10 @@ def __init__(self, out_path_base=None, **inputs):
def _run_interface(self, runtime):
src_fname, _ = _splitext(self.inputs.source_file)
_, ext = _splitext(self.inputs.in_file[0])
compress = ext == '.nii'
if compress:
ext = '.nii.gz'
if self.inputs.compress is True and not ext.endswith('.gz'):
ext += '.gz'
elif self.inputs.compress is False and ext.endswith('.gz'):
ext = ext[:-3]

m = BIDS_NAME.search(src_fname)

Expand Down Expand Up @@ -246,7 +249,7 @@ def _run_interface(self, runtime):
if isdefined(self.inputs.extra_values):
out_file = out_file.format(extra_value=self.inputs.extra_values[i])
self._results['out_file'].append(out_file)
if compress:
if ext.endswith('.gz'):
with open(fname, 'rb') as f_in:
with gzip.open(out_file, 'wb') as f_out:
copyfileobj(f_in, f_out)
Expand Down
Loading