Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Nuisance regression strategies might not be forking properly when despiking is also forking #1370

Closed
sgiavasis opened this issue Oct 8, 2020 · 6 comments
Closed

🐛 Nuisance regression strategies might not be forking properly when despiking is also forking #1370

sgiavasis opened this issue Oct 8, 2020 · 6 comments
Assignees
@shnizzedy
Labels
bug
Milestone
1.7.2 Release

Comments

@sgiavasis
Copy link
Collaborator

Describe the bug
User reports that not every nuisance regression strategy showed up in the output directory. Note the strategies fork via Nipype iterables (instead of the C-PAC pipeline fork points) and show up as multiple sub-directories under the nuisance-regressed functional outputs.

To Reproduce
Steps to reproduce the behavior:

  1. Run a pipeline with multiple nuisance regression strategies (3+ would be good for a test).
  2. After the pipeline completes, check the completed nuisance strategies.

Expected behavior
There should be a sub-directory for each nuisance regression strategy under the nuisance-regressed functional outputs in the output directory. functional_nuisance_residuals is a good directory to check.

Versions

  • C-PAC: 1.7.0
  • Container Platform :
    • Singularity
@sgiavasis sgiavasis added the bug label Oct 8, 2020
@sgiavasis sgiavasis added this to the 1.7.2 Release milestone Oct 8, 2020
@sgiavasis
Copy link
Collaborator Author

sgiavasis commented Oct 8, 2020 •
edited by shnizzedy

Here is the regressor selection from the user who reported this issue:

Regressors :
#1: aCompCor, GSR, no spike regression
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   aCompCor:
     summary:
       method: DetrendPC
       components: 5
     tissues:
       - WhiteMatter
       - CerebrospinalFluid
     extraction_resolution: 2
   GlobalSignal:
     summary: Mean
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
#2 aCompCor, no GSR, no spike regression
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   aCompCor:
     summary:
       method: DetrendPC
       components: 5
     tissues:
       - WhiteMatter
       - CerebrospinalFluid
     extraction_resolution: 2
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
#3 No aCompCor, GSR, no spike regression
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   GlobalSignal:
     summary: Mean
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
#4 No aCompCor, no GSR, no spike regression
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
#5 aCompCor, GSR, spike regression
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   aCompCor:
     summary:
       method: DetrendPC
       components: 5
     tissues:
       - WhiteMatter
       - CerebrospinalFluid
     extraction_resolution: 2
   GlobalSignal:
     summary: Mean
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
   Censor:
       method: SpikeRegression
       thresholds:
        - type: FD_J
          value: 0.2
       number_of_previous_trs_to_censor: 0
       number_of_subsequent_trs_to_censor: 0
#6 aCompCor, no GSR, spike regression 
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   aCompCor:
     summary:
       method: DetrendPC
       components: 5
     tissues:
       - WhiteMatter
       - CerebrospinalFluid
     extraction_resolution: 2
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
   Censor:
       method: SpikeRegression
       thresholds:
        - type: FD_J
          value: 0.2
       number_of_previous_trs_to_censor: 0
       number_of_subsequent_trs_to_censor: 0
#7 No aCompCor, GSR, spike regression 
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   GlobalSignal:
     summary: Mean
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
   Censor:
       method: SpikeRegression
       thresholds:
        - type: FD_J
          value: 0.2
       number_of_previous_trs_to_censor: 0
       number_of_subsequent_trs_to_censor: 0
#8 No aCompCor, no GSR, spike regression 
 - Motion:
     include_delayed: true
     include_squared: true
     include_delayed_squared: true
   PolyOrt:
    degree: 2
   Bandpass:
     bottom_frequency: 0.01
     top_frequency: 0.1
   Censor:
       method: SpikeRegression
       thresholds:
        - type: FD_J
          value: 0.2
       number_of_previous_trs_to_censor: 0
       number_of_subsequent_trs_to_censor: 0

@shnizzedy shnizzedy self-assigned this Oct 12, 2020
@shnizzedy
Copy link
Member

shnizzedy commented Oct 13, 2020 •
edited

At completion with the above Regressors section, all 8 outputs are present:

regressor label present in output directory
#1: aCompCor, GSR, no spike regression aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1 ✔️
#2 aCompCor, no GSR, no spike regression aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1 ✔️
#3 No aCompCor, GSR, no spike regression G-M_M-SDB_P-2_BP-B0.01-T0.1 ✔️
#4 No aCompCor, no GSR, no spike regression M-SDB_P-2_BP-0.01-T0.1 ✔️
#5 aCompCor, GSR, spike regression aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2 ✔️
#6 aCompCor, no GSR, spike regression aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2 ✔️
#7 No aCompCor, GSR, spike regression G-M_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2 ✔️
#8 No aCompCor, no GSR, spike regression M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2 ✔️

@shnizzedy
Copy link
Member

The specific user issue here is this forking in concert with forking from

runDespike: [0,1]

which should generate the 8 outputs above with and without despiking but only one set of 8 outputs is generated

@shnizzedy shnizzedy changed the title 🐛 Nuisance regression strategies might not be forking properly 🐛 Nuisance regression strategies might not be forking properly when despiking is also forking Oct 13, 2020
@sgiavasis
Copy link
Collaborator Author

Bingo, so then this is really this:

@sgiavasis
Copy link
Collaborator Author

Just merged #1372 to develop; can try again.

@shnizzedy
Copy link
Member

Success!

In the output directory, a subdirectory with despiking and one without:

.
├── pipeline_debug_nuisance_despike_freq-filter_nuisance
│   └── sub-{subject}_ses-1
|       ├── …
│       ├── functional_nuisance_residuals
│       │   └── _scan_rest_run-1
│       │       ├── _selector_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1
│       │       ├── _selector_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
│       │       ├── _selector_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1
│       │       ├── _selector_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
│       │       ├── _selector_G-M_M-SDB_P-2_BP-B0.01-T0.1
│       │       ├── _selector_G-M_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
│       │       ├── _selector_M-SDB_P-2_BP-B0.01-T0.1
│       │       └── _selector_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
|       └── …
└── pipeline_debug_nuisance_freq-filter_nuisance
    └── sub-{subject}_ses-1
        ├── …
        ├── functional_nuisance_residuals
        │   └── _scan_rest_run-1
        │       ├── _selector_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1
        │       ├── _selector_aC-CSF+WM-2mm-DPC5_G-M_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
        │       ├── _selector_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1
        │       ├── _selector_aC-CSF+WM-2mm-DPC5_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
        │       ├── _selector_G-M_M-SDB_P-2_BP-B0.01-T0.1
        │       ├── _selector_G-M_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
        │       ├── _selector_M-SDB_P-2_BP-B0.01-T0.1
        │       └── _selector_M-SDB_P-2_BP-B0.01-T0.1_C-S-0+0-FD-J0.2
        └── …

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shnizzedy shnizzedy

Labels
bug
Projects
None yet
Milestone
1.7.2 Release
Development

No branches or pull requests
2 participants
@sgiavasis @shnizzedy