1.0.1

Release notes

Hotfix release to address the most relevant issues found on the 1.0.0 version. Additionally, a great new feature has been added to processes multi-echo data (thanks to @emdupre).

CHANGES

With thanks to @emdupre for contributions.

• [PIN] Update required niworkflows version to 0.2.3
• [FIX] Refine antsBrainExtraction if recon-all is run (#912)
  With thanks to Arno Klein for his helpful comments here
  (https://github.com/poldracklab/fmriprep/issues/431#issuecomment-299583391)
• [FIX] Use thinner contours in reportlets (#910)
• [FIX] Robuster EPI mask (#911)
• [FIX] Set workflow return value before potential error (#887)
• [DOC] Documentation about FreeSurfer and --fs-no-reconall (#894)
• [DOC] Fix example in installation ants-nthreads -> omp-nthreads (#885)
  With thanks to @mvdoc.
• [ENH] Allow for multiecho data (#875)

1.0.0

Release notes

FMRIPREP 1.0.0, codename "BOLD raccoon", is the first official release of FMRIPREP.

FMRIPREP "BOLD" is the outcome of a thorough testing process to assess that FMRIPREP works reliably on any dataset uploaded to OpenfMRI.

At the time of writing, a second pass of FMRIPREP is being run over 243 participants from 62 datasets of OpenfMRI that contain BOLD images. This accounts for approximately 4 subjects per dataset (with exceptions like My Connectome, which only has one subject but hundreds of BOLD runs). Overall, this release has been tested on more than 320 participants from OpenfMRI, and dozens of other datasets that our users have provided when encountering problems.

Making a thorough pipeline that adapts to many kinds of dataset and associated metadata is a difficult undertaking. Throughout the three testing phases, we have encountered many problems in FMRIPREP (as well as some of the datasets), but the most complicated to deal with were hitting memory limits on high-capacity (>64GB) compute nodes. The effort to profile and limit memory consumption led to many optimizations and bugfixes in FMRIPREP, as well as in Nipype.

This release supports a friendly way of providing FreeSurfer licenses, and comes with FreeSurfer 6.0.1 embedded in the Docker and Singularity containers.

Acknowledgment

We want to thank all the contributors that have added to FMRIPREP in one way or another (see our credits file).
This release also contains modifications to prepare contributions that are in the making, like @madeleinesnyder's pull request for processing only-BOLD datasets or @emdupre's pull request for Multi-echo BOLD images. We also thank these soon-to-be contributors, and look forward to incorporating their additions.

We would also like to thank the FreeSurfer team for their support when moving to 6.0.1, and particularly Andrew Hoopes, who followed-up closely during this effort.

Finally, we want to thank all our users who posted over 80(!) questions about FMRIPREP on neurostars.org, many of them reporting problems that couldn't have been diagnosed otherwise. Likewise, many users directly reported to this GitHub repository. We want to express our utmost appreciation to them and their valuable feedback.

CHANGES

• [ENH] Add --resource-monitor flag (#883)
• [FIX] Collision between Multi-T1w and --no-freesurfer (#880)
• [FIX] Setting use_compression on resampling workflows (#882)
• [ENH] Estimate motion parameters before STC (#876)
• [ENH] Add --stop-on-first-crash option (#865)
• [FIX] Correctly handling xforms (#874)
• [FIX] Combined ROI reportlets (#872)
• [ENH] Strip reportlets out of full report (#867)

1.0.0-rc13 (1st of December 2017)

• [FIX] Broken --fs-license-file argument (#869)

1.0.0-rc12 (29th of November 2017)

• [ENH] Use Nipype MultiProc even for sequential execution (#856)
• [REF] More memory annotations and considerations (#816)
• [FIX] Controlling memory explosion (#854)
• [WRAPPER] Mount nipype repositories as niworkflows submodule (#834)
• [FIX] Reduce image loads in local memory (#839)
• [ENH] Always sync qforms, refactor error messaging (#851)

1.0.0-rc11 (24th of November 2017)

• [ENH] Check for invalid qforms in validation (#847)
• [FIX] Update pybids to include latest bugfixes (#838)
• [FIX] MultiApplyTransforms failed with nthreads=1 (#835)

1.0.0-rc10 (9th of November 2017)

• [FIX] Adopt new FreeSurfer (v6.0.1) license mechanism (#787)
• [ENH] Output affine transforms from original T1w images to preprocessed anatomical (#726)
• [FIX] Correct headers in AFNI-generated NIfTI files (#818)
• [FIX] Normalize T1w image qform/sform matrices (#820)

1.0.0-rc9 (2nd of November 2017)

• [FIX] Fixed #776 (aCompCor - numpy.linalg.linalg.LinAlgError: SVD did not converge) via #807.
• [ENH] Added CSF column to _confounds.tsv (included in #807)
• [DOC] Add more details on the outputs of FMRIPREP and minor fixes (#811)
• [ENH] Processing confounds in BOLD space (#807)
• [ENH] Updated niworkflows and nipype, including the new feature to close all file descriptors (#810)
• [REF] Refactored BOLD workflows module (#805)
• [ENH] Improved memory annotations (#803, #807)

1.0.0-rc8 (27th of October 2017)

• [FIX] Allow missing magnitude2 in phasediff-type fieldmaps (#802)
• [FIX] Lower tolerance deciding t1_merge shapes (#798)
• [FIX] Be robust to 4D T1w images (#797)
• [ENH] Resource annotations (#746)
• [ENH] Use indexed_gzip with nibabel (#788)
• [FIX] Reduce FoV of outputs in T1w space (#785)

1.0.0-rc7 (20th of October 2017)

• [ENH] Update pinned version of nipype to latest master
• [ENH] Added rX permissions to make life easier on Singularity users (#757)
• [DOC] Citation boilerplate (#779)
• [FIX] Patch to remove long filenames after mri_concatenate_lta (#778)
• [FIX] Only use unbiased template with --longitudinal (#771)
• [FIX] Use t1_2_fsnative registration when sampling to surface (#762)
• [ENH] Remove --skull_strip_ants option (#761)
• [DOC] Add reference to beginners guide (#763)

1.0.0-rc6 (11th of October 2017)

• [ENH] Add inverse normalization transform (MNI -> T1w) to derivatives (#754)
• [ENH] Fall back to initial registration if BBR fails (#694)
• [FIX] Header and affine transform updates to resolve intermittent misalignments in reports (#743)
• [FIX] Register FreeSurfer template to FMRIPREP template, handling pre-run FreeSurfer subjects more robustly, saving affine to derivatives (#733)
• [ENH] Add OpenFMRI participant sampler command-line tool (#704)
• [ENH] For SyN-SDC, assume phase-encoding direction of A-P unless specified L-R (#740, #744)
• [ENH] Permit skull-stripping with NKI ANTs template (#729)
• [ENH] Erode aCompCor masks to target volume proportions, instead of fixed distances (#731, #732)
• [DOC] Documentation updates (#748)

1.0.0-rc5 (25th of September 2017)

• [FIX] Skip slice time correction on BOLD series < 5 volumes (#711)
• [FIX] Skip AFNI check for new versions (#723)
• [DOC] Documentation clarification and updates (#698, #711)

1.0.0-rc4 (12th of September 2017)

With thanks to Mathias Goncalves for contributions.

• [ENH] Collapse ITK transforms of head-motion correction in only one file (#695)
• [FIX] Raise error when run.py is called directly (#692)
• [FIX] Parse crash files when they are stored as text (#690)
• [ENH] Replace medial wall values with NaNs (#687)

1.0.0-rc3 (28th of August 2017)

With thanks to Anibal Sólon for contributions.

• [ENH] Add --low-mem option to reduce memory usage for large BOLD series (#663)
• [ENH] Parallelize anatomical conformation step (#666)
• [FIX] Handle missing functional data in SubjectSummary node (#670)
• [FIX] Disable --no-skull-strip-ants (AFNI skull-stripping) (#674)
• [FIX] Initialize SyN SDC more robustly (#680)
• [DOC] Add comprehensive documentation of workflow API (#638)

1.0.0-rc2 (12th of August 2017)

• [ENH] Increased support for partial field-of-view BOLD datasets (#659)
• [FIX] Slice time correction is now being applied to output data (not only to intermediate file used for motion estimation - #662)
• [FIX] Fieldmap unwarping is now being applied to MNI space outputs (not only to T1w space outputs - #662)

1.0.0-rc1 (8th of August 2017)

• [ENH] Include ICA-AROMA confounds in report (#646)
• [ENH] Save non-aggressively denoised BOLD series (#648)
• [ENH] Improved logging messages (#621)
• [ENH] Improved resource management (#622, #629, #640, #641)
• [ENH] Improved confound header names (#634)
• [FIX] Ensure multi-T1w image datasets have RAS-oriented template (#637)
• [FIX] More informative errors for conflicting options (#632)
• [DOC] Improved report summaries (#647)

1.0.0-rc13

Release notes

Quick release with hotfix to the broken --fs-license-file.

CHANGES

• [FIX] Broken --fs-license-file argument (#869)

1.0.0-rc12

Release notes

This release is almost completely devoted to fix the memory-related issues that kept appearing in the issue tracker (#164, #225, #657, #688, #766, #799, #801, #832, #833, #836).

We found a snow-ball problem that may affect any workflow based on Nipype, and that particularly affected FMRIPREP for several reasons. The memory explosion typically occurred after a large number of nodes of the execution graph had been run, and was definitely triggered with a memory-hungry process finished at that point in time. Briefly,

1. The workflow building of FMRIPREP is heavyweight routine, and the garbage collector had some trouble to clear-up its debris.
2. With that extra memory reserved, FMRIPREP called the run() member of the main workflow which uses a multiprocessing.Pool. By default, --nthread workers were spun up using the fork system call. Subprocess generated this way allocate as much memory as the parent had allocated. Only some was free, and this meant around 350MB per worker in the case of FMRIPREP.
3. Nipype uses subprocess.Popen to call the third party software (FSL, AFNI, FreeSurfer, etc.). Again, Popen calls try to allocate as much memory as their parent (the workers, or the main thread for nodes with run_without_submitting=True).
4. The workers and the main thread kept growing slowly until the memory explosion condition happened. From that point on, memory grew linearly with running time.

Changes in this release first address two targets:

• Try to keep the main thread of FMRIPREP as lightweight as possible:
  • Isolating as much as possible the workflow creation wrapping it with a subprocess.
  • Removing run_without_submitting=True from all nodes that opened images (which are memory-hungry).
  • When --nthreads 1 use Nipype MultiProc (as if parallel processing were required) for the reasons below.
• Try to keep the workers as thin as possible:
  • Using multiprocessing in forkserver mode, which prevent subprocess from allocating as much memory as the parent.
  • Clean-up memory as often as possible (forcing garbage collector calls, explicitly deleting objects, etc.)
  • Restart the workers as often as possible (currently, after running just one node)
  • Reduce FoV of images whenever possible.

All these efforts also required some changes to Nipype, that will be released with their version 0.14.0.

CHANGES

• [ENH] Use Nipype MultiProc even for sequential execution (#856)
• [REF] More memory annotations and considerations (#816)
• [FIX] Controlling memory explosion (#854)
• [WRAPPER] Mount nipype repositories as niworkflows submodule (#834)
• [FIX] Reduce image loads in local memory (#839)
• [ENH] Always sync qforms, refactor error messaging (#851)

1.0.0-rc11

CHANGE

• [ENH] Check for invalid qforms in validation (#847)
• [FIX] Update pybids to include latest bugfixes (#838)
• [FIX] MultiApplyTransforms failed with nthreads=1 (#835)

1.0.0-rc10

• [FIX] Adopt new FreeSurfer (v6.0.1) license mechanism (#787)
• [ENH] Output affine transforms from original T1w images to preprocessed anatomical (#726)
• [FIX] Correct headers in AFNI-generated NIfTI files (#818)
• [FIX] Normalize T1w image qform/sform matrices (#820)

1.0.0-rc9

This release is focused on reorganizing the modules of the BOLD processing. Also, the workflow that computes the confound has been refactored and now works in native BOLD space. This new feature is expected to help integrate the --func-only feature (#808), helps keeping memory low (as the input files are a lot smaller) and is more robust to partial-FoV scans.

Also, includes large improvements to the documentation.

CHANGES

• [FIX] Fixed #776 (aCompCor - numpy.linalg.linalg.LinAlgError: SVD did not converge) via #807.
• [ENH] Added CSF column to _confounds.tsv (included in #807)
• [DOC] Add more details on the outputs of FMRIPREP and minor fixes (#811)
• [ENH] Processing confounds in BOLD space (#807)
• [ENH] Updated niworkflows and nipype, including the new feature to close all file descriptors (#810)
• [REF] Refactored BOLD workflows module (#805)
• [ENH] Improved memory annotations (#803, #807)

1.0.0-rc8

CHANGES

• [FIX] Allow missing magnitude2 in phasediff-type fieldmaps (#802)
• [FIX] Lower tolerance deciding t1_merge shapes (#798)
• [FIX] Be robust to 4D T1w images (#797)
• [ENH] Resource annotations (#746)
• [ENH] Use indexed_gzip with nibabel (#788)
• [FIX] Reduce FoV of outputs in T1w space (#785)

1.0.0-rc7

• [ENH] Update pinned version of nipype to latest master
• [ENH] Added rX permissions to make life easier on Singularity users (#757)
• [DOC] Citation boilerplate (#779)
• [FIX] Patch to remove long filenames after mri_concatenate_lta (#778)
• [FIX] Only use unbiased template with --longitudinal (#771)
• [FIX] Use t1_2_fsnative registration when sampling to surface (#762)
• [ENH] Remove --skull_strip_ants option (#761)
• [DOC] Add reference to beginners guide (#763)

1.0.0-rc6

• [ENH] Add inverse normalization transform (MNI -> T1w) to derivatives (#754)
• [ENH] Fall back to initial registration if BBR fails (#694)
• [FIX] Header and affine transform updates to resolve intermittent misalignments in reports (#743)
• [FIX] Register FreeSurfer template to FMRIPREP template, handling pre-run FreeSurfer subjects more robustly, saving affine to derivatives (#733)
• [ENH] Add OpenFMRI participant sampler command-line tool (#704)
• [ENH] For SyN-SDC, assume phase-encoding direction of A-P unless specified L-R (#740, #744)
• [ENH] Permit skull-stripping with NKI ANTs template (#729)
• [ENH] Erode aCompCor masks to target volume proportions, instead of fixed distances (#731, #732)
• [DOC] Documentation updates (#748)