Empty dataset detected. Clinical data cannot be extracted.

[theguardsgod]

When I convert adni to bids with following command clinica convert adni-to-bids -m T1 ./dataset/ADNI ./clinical ./BIDS. it says Empty dataset detected. Clinical data cannot be extracted.

(CDL) [chenyuliu@pbcmlg01 CDL]$ clinica convert adni-to-bids -m T1 ./dataset/ADNI ./clinical ./BIDS /home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids_cli.py:67: DtypeWarning: Columns (19,20,21,104,105,106) have mixed types.Specify dtype option on import or set low_memory=False. force_new_extraction, /home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids.py:107: DtypeWarning: Columns (19,20,21,104,105,106) have mixed types.Specify dtype option on import or set low_memory=False. "ADNI", clinic_specs_path, clinical_data_dir, bids_ids Traceback (most recent call last): File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/bin/clinica", line 8, in <module> sys.exit(main()) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/cmdline.py", line 78, in main cli() File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids_cli.py", line 74, in cli subjects_list_path=subjects_list, File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/iotools/converters/adni_to_bids/adni_to_bids.py", line 129, in convert_clinical_data bids_ids, clinic_specs_path, clinical_data_dir, bids_subjs_paths File "/home/chenyuliu/anaconda3/anaconda3/envs/CDL/lib/python3.7/site-packages/clinica/iotools/converters/adni_to_bids/adni_utils.py", line 722, in create_adni_sessions_dict raise ValueError("Empty dataset detected. Clinical data cannot be extracted.") ValueError: Empty dataset detected. Clinical data cannot be extracted.

[theguardsgod]

BTW my python==3.7 clinica==0.5.2

[omar-rifai]

Hi @theguardsgod,

This message arises upon trying to extract the sessions info from the clinical file. It means that no images where extracted successfully (Possible because the input path was not found?). In order to narrow down the issue, can you please try converting one subject by specifying the subject ID with -sl subject.txt and using the verbose mode (i.e clinica -v convert adni-to-bids ..).

Thank you

[theguardsgod]

Hi @theguardsgod,

This message arises upon trying to extract the sessions info from the clinical file. It means that no images where extracted successfully (Possible because the input path was not found?). In order to narrow down the issue, can you please try converting one subject by specifying the subject ID with -sl subject.txt and using the verbose mode (i.e clinica -v convert adni-to-bids ..).

Thank you

it still doesn't work, the verbose information is following
2021-10-22 10:41:20,439:INFO:Loading a subjects lists provided by the user... 2021-10-22 10:41:20,446:INFO:Calculating paths of T1 images. Output will be stored in /data/chenyuliu/CDL/BIDS/conversion_info/v9. 2021-10-22 10:41:21,159:INFO:No T1 image path found for subject 003_S_1074 in visit bl with image ID 53395 2021-10-22 10:41:21,161:INFO:No T1 image path found for subject 003_S_1074 in visit m12 with image ID 91404 2021-10-22 10:41:21,163:INFO:No T1 image path found for subject 003_S_1074 in visit m18 with image ID 121117 2021-10-22 10:41:21,165:INFO:No T1 image path found for subject 003_S_1074 in visit m24 with image ID 141167 2021-10-22 10:41:21,167:INFO:No T1 image path found for subject 003_S_1074 in visit m36 with image ID 288936 2021-10-22 10:41:21,169:INFO:No T1 image path found for subject 003_S_1074 in visit m48 with image ID 299332 2021-10-22 10:41:21,171:INFO:No T1 image path found for subject 003_S_1074 in visit m60 with image ID 1293319 2021-10-22 10:41:21,173:INFO:No T1 image path found for subject 003_S_1074 in visit m72 with image ID 394777 2021-10-22 10:41:21,175:INFO:No T1 image path found for subject 003_S_1074 in visit m96 with image ID 498002 2021-10-22 10:41:21,177:INFO:No T1 image path found for subject 003_S_1074 in visit m126 with image ID 912458 2021-10-22 10:41:21,189:INFO:Paths of T1 images found. Exporting images into BIDS ... 2021-10-22 10:41:21,372:INFO:[T1] No path specified for 003_S_1074 in session m24 1/10 2021-10-22 10:41:21,372:INFO:[T1] No path specified for 003_S_1074 in session bl 2/10 2021-10-22 10:41:21,373:INFO:[T1] No path specified for 003_S_1074 in session m12 3/10 2021-10-22 10:41:21,374:INFO:[T1] No path specified for 003_S_1074 in session m18 4/10 2021-10-22 10:41:21,375:INFO:[T1] No path specified for 003_S_1074 in session m36 5/10 2021-10-22 10:41:21,375:INFO:[T1] No path specified for 003_S_1074 in session m126 6/10 2021-10-22 10:41:21,376:INFO:[T1] No path specified for 003_S_1074 in session m48 7/10 2021-10-22 10:41:21,377:INFO:[T1] No path specified for 003_S_1074 in session m72 8/10 2021-10-22 10:41:21,378:INFO:[T1] No path specified for 003_S_1074 in session m60 9/10 2021-10-22 10:41:21,378:INFO:[T1] No path specified for 003_S_1074 in session m96 10/10 2021-10-22 10:41:21,384:INFO:Creating modality agnostic files... 2021-10-22 10:41:21,389:INFO:Creating participants.tsv... 2021-10-22 10:41:26,570:INFO:Creating sessions files... 2021-10-22 10:41:26,574:INFO: Reading clinical data file: ADNIMERGE.csv 2021-10-22 10:41:26,894:INFO: Reading clinical data file: CDR.csv 2021-10-22 10:41:26,949:INFO: Reading clinical data file: ITEM.csv 2021-10-22 10:41:27,484:INFO: Reading clinical data file: NEUROBAT.csv 2021-10-22 10:41:27,618:INFO: Reading clinical data file: ADASSCORES.csv 2021-10-22 10:41:27,638:INFO: Reading clinical data file: CCI.csv 2021-10-22 10:41:27,648:INFO: Reading clinical data file: ECOGPT.csv 2021-10-22 10:41:27,707:INFO: Reading clinical data file: ECOGSP.csv 2021-10-22 10:41:27,759:INFO: Reading clinical data file: FCI.csv 2021-10-22 10:41:27,784:INFO: Reading clinical data file: GDSCALE.csv 2021-10-22 10:41:27,830:INFO: Reading clinical data file: MODHACH.csv 2021-10-22 10:41:27,845:INFO: Reading clinical data file: MOCA.csv 2021-10-22 10:41:27,888:INFO: Reading clinical data file: NPI.csv 2021-10-22 10:41:27,977:INFO: Reading clinical data file: NPIQ.csv 2021-10-22 10:41:28,011:INFO: Reading clinical data file: BHR_EVERYDAY_COGNITION.csv 2021-10-22 10:41:28,018:INFO: Reading clinical data file: BHR_BASELINE_QUESTIONNAIRE.csv 2021-10-22 10:41:28,025:INFO: Reading clinical data file: BHR_LONGITUDINAL_QUESTIONNAIRE.csv 2021-10-22 10:41:28,033:INFO: Reading clinical data file: MEDHIST.csv 2021-10-22 10:41:28,050:INFO: Reading clinical data file: VITALS.csv 2021-10-22 10:41:28,097:INFO: Reading clinical data file: UWNPSYCHSUM_03_07_19.csv 2021-10-22 10:41:28,119:INFO: Reading clinical data file: UWNPSYCHSUM_03_26_20.csv

[theguardsgod]

BTW, I had tried clinica==0.4.1 before, it doesn't work to convert data with the issue in #479

[ghisvail]

Could you please show the list of files for one of the subjects?

[theguardsgod]

└── ./135_S_4356
└── ./135_S_4356/Spatially_Normalized,_Masked_and_N3_corrected_T1_image
└── ./135_S_4356/Spatially_Normalized,_Masked_and_N3_corrected_T1_image/2016-12-12_11_50_51.0
└── ./135_S_4356/Spatially_Normalized,_Masked_and_N3_corrected_T1_image/2016-12-12_11_50_51.0/I812584
└── ./135_S_4356/Spatially_Normalized,_Masked_and_N3_corrected_T1_image/2016-12-12_11_50_51.0/I812584/ADNI_135_S_4356_MR_Spatially_Normalized,_Masked_and_N3_corrected_T1_image_Br_20170125142043176_S519305_I812584.nii

[ghisvail]

duplicate of #479

[ghisvail]

I was able to reproduce, investigation in progress.

[theguardsgod]

I was able to reproduce, investigation in progress.

I have an issue with it. Is this issue relevant to this problem?

[omar-rifai]

hi @theguardsgod,

I downloaded the data again from the adni website and was able to successfully convert the data for participant 003_S_1074. It seems that the data was incomplete for conversion previously (did not have images that passed the quality check?) but that it has been changed recently.

Please make sure to download all three of "original" "preprocessed" and "postprocessed" for conversion to function.

Let me know if this helps.

Thanks!

[theguardsgod]

hi @theguardsgod,

I downloaded the data again from the adni website and was able to successfully convert the data for participant 003_S_1074. It seems that the data was incomplete for conversion previously (did not have images that passed the quality check?) but that it has been changed recently.

Please make sure to download all three of "original" "preprocessed" and "postprocessed" for conversion to function.

Let me know if this helps.

Thanks!

Thank you, it works. But does it mean that I need to download all images of a subject so that I can use clinicalDL framework

[omar-rifai]

Great, than you, I will be closing the issue. The images Clinica chooses for conversion for each modality is explained in the docs. I don't have in mind a minimal set that would function right not but I suppose that it should be possible.

For clinicaDL, you would only need the BIDs or CAPs folders of course, and not the raw data.