Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dataset_description.json #79

Closed
jkkenley opened this issue May 11, 2022 · 14 comments
Closed

dataset_description.json #79

jkkenley opened this issue May 11, 2022 · 14 comments
Assignees
@madisoth
Labels
help wanted Extra attention is needed

Comments

@jkkenley
Copy link

I am using the following docker call to run our data through:

docker run --rm --user 1191:10
-v /docker_input_2020_09_15:/bids_input:ro
-v /docker_output_2020_09_15:/output
-v /usr/local/pkg/freesurfer6.0/license.txt:/license
-v/subject_specific_code/subID_wcII_ABCD_BIDS_cleaning.json:/cleaning.json
dcanlabs/abcd-hcp-pipeline:v0.0.3 /bids_input /output --freesurfer-license=/license
--participant-label subID --ncpus 10 --bandstop 18.582 25.7263 --custom-clean /cleaning.json

I receive the following error when I run this subject but not 3 others:
/usr/local/lib/python3.6/dist-packages/bids/layout/bids_layout.py:116: UserWarning: 'dataset_description.json' file is missing from project root. You may want to set the root path to a valid BIDS project.
warnings.warn("'dataset_description.json' file is missing from "
Traceback (most recent call last):
File "/app/run.py", line 364, in
_cli()
File "/app/run.py", line 67, in _cli
return interface(**kwargs)
File "/app/run.py", line 263, in interface
session_spec = ParameterSettings(session, out_dir)
File "/app/pipelines.py", line 126, in init
self.echospacing = self.bids_data['fmap_metadata']['positive'][0][
IndexError: list index out of range

I have compared this data set to another that ran through and cannot find any obvious failure reasons.
I had gotten this error in the past and was pointed to this site:
https://bids-specification.readthedocs.io/en/stable/03-modality-agnostic-files.html#dataset_descriptionjson

But I'm not sure what to do with the information since I had 3 other data sets that ran through.

Thank you for the help.
Jeanette
@arueter1
Copy link
Contributor

Hi Jeanette! Thanks for submitting your error to us. Someone from our lab will be in contact with you as soon as we are able.

@arueter1 arueter1 added the help wanted Extra attention is needed label May 11, 2022
@madisoth
Copy link
Collaborator

madisoth commented May 12, 2022 •
edited
Loading

Hi Jeanette,

My best guess is the error is unrelated to dataset_description.json and the actual problem is the "EffectiveEchoSpacing" field is missing from the fmap json files. (I believe older versions of Dcm2bids do not output EffectiveEchoSpacing, so that may be the cause.) Can you check:

  1. If "EffectiveEchoSpacing" is present in the fmap directory json files
  2. If the json files have infomation about the Dcm2bids or dcm2niix version used to convert from DICOM (for Dcm2bids the field is titled "Dcm2bidsVersion")

Thanks!

(edit to add: assuming EffectiveEchoSpacing is the problem, as long as you other datasets used the same fmap sequence it would be fine to copy the EffectiveEchoSpacing info from those to the problem subject, then try rerunning it.)

@jkkenley
Copy link
Author

jkkenley commented May 13, 2022 via email

@madisoth
Copy link
Collaborator

Hi Jeanette,

Apologies for the slow response. Thank you for posting that info, that rules out the EffectiveEchoSpacing issue which was my first guess.

Could you share:

  1. the BIDS directory tree for the subject (removing subject and session IDs is OK)
  2. the command or script used to run the pipeline

Thank you!

@jkkenley
Copy link
Author

jkkenley commented May 24, 2022 via email

@arueter1
Copy link
Contributor

@jkkenley - Were you able to send this over to Thomas? If you no longer need assistance, feel free to close this issue. :)

@jkkenley
Copy link
Author

jkkenley commented Jun 23, 2022 via email

@jarodroland
Copy link

I came across this thread while searching a similar error message. For my data, it turned out that one of my field map json files was missing the IntendedFor entry.
I was confused because when I compared my AP fmap json to a known good subject everything looked fine (similar to the above user), but my PA fmap was missing the IntendedFor entry. Once I corrected that it (so both the AP and PA fmap json had the correct IntendedFor directive) then it resolved the error.
Hope this is helpful.

@jkkenley
Copy link
Author

jkkenley commented Oct 11, 2022 via email

@jkkenley
Copy link
Author

jkkenley commented Oct 11, 2022 via email

@lundq163
Copy link
Member

Hi @jkkenley, were you able to successfully provide @madisoth with what he was looking for? (the BIDS directory tree for the subject, the command or script used to run the pipeline)

@fpichardo-umn
Copy link

fpichardo-umn commented Mar 21, 2023 •
edited
Loading

I am having the same issue described above except that I'm testing running the pipeline on two participants, and it happens for both. Here's the directory structure:
sub-001
sub-001/ses-06
sub-001/ses-06/anat
sub-001/ses-06/anat/sub-001_ses-06_T1w.json
sub-001/ses-06/anat/sub-001_ses-06_T1w.nii.gz
sub-001/ses-06/dwi
sub-001/ses-06/dwi/sub-001_ses-06_dir-PA_dwi.bval
sub-001/ses-06/dwi/sub-001_ses-06_dir-PA_dwi.json
sub-001/ses-06/dwi/sub-001_ses-06_dir-AP_dwi.bval
sub-001/ses-06/dwi/sub-001_ses-06_dir-AP_dwi.bvec
sub-001/ses-06/dwi/sub-001_ses-06_dir-AP_dwi.json
sub-001/ses-06/dwi/sub-001_ses-06_dir-PA_dwi.bvec
sub-001/ses-06/dwi/sub-001_ses-06_dir-PA_dwi.nii.gz
sub-001/ses-06/dwi/sub-001_ses-06_dir-AP_dwi.nii.gz
sub-001/ses-06/fmap
sub-001/ses-06/fmap/sub-001_ses-06_acq-rest_dir-AP_epi.json
sub-001/ses-06/fmap/sub-001_ses-06_acq-hariri_dir-AP_epi.json
sub-001/ses-06/fmap/sub-001_ses-06_acq-hariri_dir-AP_epi.nii.gz
sub-001/ses-06/fmap/sub-001_ses-06_acq-rest_dir-AP_epi.nii.gz
sub-001/ses-06/func
sub-001/ses-06/func/sub-001_ses-06_task-rest_bold.nii.gz
sub-001/ses-06/func/sub-001_ses-06_task-rest_bold.json
sub-001/ses-06/func/sub-001_ses-06_task-hariri_bold.nii.gz
sub-001/ses-06/func/sub-001_ses-06_task-hariri_bold.json
sub-001/ses-06/func/sub-001_ses-06_task-hariri_events.tsv
sub-001/ses-06/sub-001_ses-06_scans.json
sub-001/ses-06/sub-001_ses-06_scans.tsv

EffectiveEchoSpacing is in the fmap jsons, and they all have IntendedFor in the relevant fmap json.

I've run it via as an sbatch script that runs the following command:
env -i singularity run
-B /PATH/TO/DATA:/bids_input
-B /PATH/TO/OUTPUT:/output
-B /PATH/TO/license.txt:/opt/freesurfer/license.txt
./abcd-hcp-pipeline.sif /bids_input /output --freesurfer-license=/opt/freesurfer/license.txt --participant-label 001 --session-id 06

Using version: abcd-hcp-pipeline 0.1.2

Here are the resources I requested when submitting the job:
#SBATCH --time=48:00:00
#SBATCH --ntasks=8
#SBATCH --mem=32g
#SBATCH --tmp=32g

I also loaded the following modules:
module load fsl
module load python3
module load freesurfer
module load singularity

Appreciate the help!

@madisoth
Copy link
Collaborator

madisoth commented Apr 10, 2023 •
edited
Loading

@fpichardo-umn

The pipeline will assume that since your fmap filenames end in "_epi.nii.gz", you have PEPolar fmaps (e.g. both "dir-AP" and "dir-PA") for topup correction. With only "dir-AP" in the fmap dir an error would be expected.

The fieldmap schemes the pipelines can handle are cases 2 and 4 in the BIDS spec here: https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/01-magnetic-resonance-imaging-data.html#types-of-fieldmaps

Another potential issue is that the BIDS "acq-" entity in filenames may go unrecognized (it's a known issue with the pipeline), so you may need to rename files from this:


 sub-001/ses-06/fmap/sub-001_ses-06_acq-hariri_dir-AP_epi.nii.gz
 sub-001/ses-06/fmap/sub-001_ses-06_acq-rest_dir-AP_epi.nii.gz

to this:

 sub-001/ses-06/fmap/sub-001_ses-06_dir-AP_run-01_epi.nii.gz
 sub-001/ses-06/fmap/sub-001_ses-06_dir-AP_run-02_epi.nii.gz

Also, if running on the MSI HPC system via Singularity container then Singularity should be the only module you need to load, all other dependencies (other than the FreeSurfer license) are built into the container.

@fpichardo-umn
Copy link

This fixed the issue (specifically checking on the fieldmaps and making sure there were two there). Thanks for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@madisoth madisoth

Labels
help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@jarodroland @madisoth @fpichardo-umn @arueter1 @jkkenley @lundq163