fmriprep output report does not properly handle multiple resting state runs that differ only in contrast #2033
Comments
|
I'm wondering if this is actually a niworkflows bug. Poking around in the fmriprep/interfaces/reports.py line 123 it looks like it gets the information from BIDS_NAME: So that will give the correct number of files, but in niworkflows.utils.bids: BIDS_NAME seems to ignore the ce entity, so the two entries will not be distinct - whichever file it happens to hit first in search will be returned each time. Would fixing this line fix the problem? |
|
Certainly we should accept the |
|
I'm certainly up for giving it a go. I'll go fork niworkflows and poke around. |
|
@jdkent can surely help you with this. |
|
I have a little more information on this now - testing this is slow, especially since every time I change the code, all the caches become invalid and it wants to run the entire analysis again (and report generation is the last step!) As is often the case when debugging, I'm now in the "how did this ever work?" phase. I've fixed BIDS_NAME to find ce_id, dir_id, and echo_id (the tags that were not being parsed), but I don't see anywhere that anything other than the task ID (no run_id or anything) gets passed on to the report generator. It does infer the number of runs (by counting the number of series that match the task_id), but does not use the specific run_id tag anywhere. So the question now is "how is it able to differentiate the runs?" |
|
@bbfrederick If you're using Docker, use Also, one thing you'll probably need to do is update: diff --git a/niworkflows/reports/figures.json b/niworkflows/reports/figures.json
index 7122243d0..4551f1786 100644
--- a/niworkflows/reports/figures.json
+++ b/niworkflows/reports/figures.json
@@ -113,6 +113,6 @@
"default_path_patterns": [
"sub-{subject}[/ses-{session}]/{datatype<anat|figures>}/sub-{subject}[_ses-{session}][_acq-{acquisition}][_ce-{contrast}][_rec-{reconstruction}][_space-{space}][_desc-{desc}]_{suffix<T1w|T2w|T1rho|T1map|T2map|T2star|FLAIR|FLASH|PDmap|PD|PDT2|inplaneT[12]|angio|dseg|mask>}.{extension<html|svg>}",
- "sub-{subject}[/ses-{session}]/{datatype<func|figures>}/sub-{subject}[_ses-{session}]_task-{task}[_acq-{acquisition}][_rec-{reconstruction}][_run-{run}][_echo-{echo}][_space-{space}][_desc-{desc}]_{suffix<bold>}.{extension<html|svg>}"
+ "sub-{subject}[/ses-{session}]/{datatype<func|figures>}/sub-{subject}[_ses-{session}]_task-{task}[_acq-{acquisition}][_ce-{contrast}][_rec-{reconstruction}][_run-{run}][_echo-{echo}][_space-{space}][_desc-{desc}]_{suffix<bold>}.{extension<html|svg>}"
]
}This should allow CE agent to be added to the figure. It's possible that this is all that will be necessary for the report generator to figure it out. |
|
Hi @bbfrederick, one more thing to help isolate this bug, inside the working directory there should be a directory named This will help differentiate: A) the situation where the Thank you for debugging this! |
|
Ah! It's B. I'm currently rerunning the analysis the way Chris suggested, after fixing the figures.json files. |
|
ah, I think this needs to be changed in figures.json to be in line with pybids, that is at least one of the places. diff --git a/niworkflows/reports/figures.json b/niworkflows/reports/figures.json
index 7122243..866d716 100644
--- a/niworkflows/reports/figures.json
+++ b/niworkflows/reports/figures.json
@@ -21,7 +21,7 @@
"pattern": "[_/\\\\]acq-([a-zA-Z0-9]+)"
},
{
- "name": "ce",
+ "name": "ceagent",
"pattern": "[_/\\\\]ce-([a-zA-Z0-9]+)"
},
{ |
|
@bbfrederick, I think I have the fix for your situation, could you try running (yet again), with my branch/fork of niworkflows? (unless if your fix works, then I will need to re-evaluate). And to help answer your question about how did this work in the first place, you should take a look in fmriprep.yml to see that fmriprep is looking at the different keys of a bids file. |
|
I tried running your branch, and it complains: It does run my version, however, which I updated with your fix (I have no idea why). Unfortunately, that didn't seem to fix things either. I've deleted everything (all work directories and derivative directories) and I'm trying again to see if maybe there was something cached that was causing the issue... |
|
sorry about the error, it may be because the repo To check this you can type the following: if you get something that looks like then it should work more as expected 🤞 |
|
Ran clean with my version, no joy. Followed your instructions, switched to your version, deleted the figures directory and most of the associated work directories, again, the problem remains. So now I'm deleting everything and running completely clean with your version. I'll report back when done, but I'm not optimistic... There still seems to be some critical piece missing. |
|
just pushed some new commits, I'm reasonably confident those will address your issue. I locally replicated and repaired the issue on my end. you can get the new commits with: Thank you for your patience! |
|
That did the trick! Thank you! In the end, what was the missing bit? BTW, while you were in there, did you fix it for direction and echo as well? It looked like those were not handled either. |
|
@bbfrederick If you're still referring to the Direction and echo are taken care of in the sections that James was working on that fixed your issue. |
|
I agree. I'll submit a separate PR after James' patch is applied just to satisfy my OCD. |
|
PR merged on the niworkflows side. If you want this patch in the next fMRIPrep release, could you submit ASAP? Base your branch off of the If you don't care about it being in the next release, let me know and I'll start the release process. |
|
I assume that was directed at @jdkent |
|
@bbfrederick Nope that was directed at you. @jdkent's patch will be in this release. |
|
@bbfrederick The missing bit was @effigies suggestion above, I didn't include it earlier since I didn't know where it was being used until recently. The linked code above looks in the reportlets directory and if it finds an Hope that makes sense! @effigies I actually see some other items to clean in |
fmriprep version 20.0.2
I ran an analysis to completion (no errors) that contained a T1w anatomic image and two resting state runs taken in the same session, one before Gd contrast was injected, and one several minutes after. The subject folder was as follows:
As far as I can tell, both files were analyzed properly, and created all the appropriate outputs in the anat and func folders of derivatives/fmriprep /sub-B13293. The output files for ce-none and cd-Gd are different, and appear to reflect the actual input files.
However as you can see in the figures folder, only one set of functional figures was produced - sub-B13293_task-rest_desc*. This matches the html output file - it says there are two input task-rest files, and shows two sets of images for the registration, confounds, and aroma analysis - but they are identical. I've done this with datsets that have multiple _task-rest_run_X files, and those are handled properly - the report shows them as unique files. But somewhere in the generation of the report and figures, it seems to throw away the "ce" BIDS entity and treat the files the same.
The text was updated successfully, but these errors were encountered: