MRIQC T1w Classifier

[n2parker]

I have MRIQC Outputs of approximately 1000 T1w images. I would like to produce the default T1w classifier using the mriqc_clf command. Since I'm working with singularity on a HPC server I use the following command after loading the singularity module:

singularity exec -c -B $SCRATCH:/mnt/ $SCRATCH/.scinet/mriqc-sandbox.img mriqc_clf --load-classifier -X /mnt/MRIQC/Study_QC/T1w.csv

with the exception of the following minor warning there are no errors.

Fontconfig warning: ignoring C.UTF-8: not a valid language tag

No new output is generated after the command is run and no new additional column is created in the T1w.csv. I'm not sure what has gone wrong here?

[oesteban]

Hi @n2parker, you should be seeing one column added into the T1w.csv file. Could you share that T1w.csv file with me so that I try to reproduce?

[n2parker]

Hi, sure here is the file. Nadine Parker, M.Sc. Graduate Student, Paus Lab Rotman Research Institute, Baycrest Institute of Medical Science University of Toronto Email: nparker@research.baycrest.org Alternate email: nadine.parker@mail.utoronto.ca Phone: (416) 785-2500 ext: 3130 <%28416%29%20785-2500%20ext%3A%203425>

…

On 12 March 2018 at 17:11, Oscar Esteban ***@***.***> wrote: Hi @n2parker <https://github.com/n2parker>, you should be seeing one column added into the T1w.csv file. Could you share that T1w.csv file with me so that I try to reproduce? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#699 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AR0KdMiiZILnMio0GZ_hhwEGMWx6WVOoks5tduSEgaJpZM4SaNHZ> .

[oesteban]

Hi, the attachment seems to have gotten lost from the reply.

Also, I am wondering if you were using an outdated version of MRIQC, in which you would need to use the -o flag to indicate where to store the predictions. Could you try with latest?

[n2parker]

I did try the -o flag and it wasn't recognized.
I will resend the file.

[n2parker]

Nadine Parker, M.Sc. Graduate Student, Paus Lab Rotman Research Institute, Baycrest Institute of Medical Science University of Toronto Email: nparker@research.baycrest.org Alternate email: nadine.parker@mail.utoronto.ca Phone: (416) 785-2500 ext: 3130 <%28416%29%20785-2500%20ext%3A%203425>

…

On 12 March 2018 at 18:48, Oscar Esteban ***@***.***> wrote: Hi, the attachment seems to have gotten lost from the reply. Also, I am wondering if you were using an outdated version of MRIQC, in which you would need to use the -o flag to indicate where to store the predictions. Could you try with latest? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#699 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AR0KdGvHrleyfiGXdWI1yZmd45KX_lhLks5tdvtBgaJpZM4SaNHZ> .

[oesteban]

I'm afraid that github does not accept attachments via email :(. Could you write directly to my email (oesteban@stanford.edu)

[n2parker]

Hi,
any luck using the classifier with the file I sent?

[nathankchan]

Hi @n2parker and @oesteban,

I ran into a similar issue using mriqc with singularity on the same HPC server. Like @n2parker, I received no warnings or errors when running mriqc_clf (exit status is 0), and the T1w.csv is not modified.

Replicating the original issue

The command I ran is below. Note that:

1. $HOME on the HPC server I work with is readable but not writeable for jobs run on compute nodes. As such, I send -H $SCRATCH/sing_home to assign a writeable $HOME for singularity containers.
2. Singularity imports the environment when running docker containers. To avoid using the environment python, I unset PYTHONPATH to force singularity to use the container's python.

$ PYTHONPATH="" singularity exec \
-H $SCRATCH/sing_home \
-B $SCRATCH/BIDS/mriqc_clf/CN_BIDS_output:/output \
$HOME/BIDS_apps/mriqc/poldracklab_mriqc_0.11.0-2018-06-05-b7b338730c42.img \
mriqc_clf --load-classifier \
-X /output/T1w.csv \

To check whether T1w.csv was modified, I ran ls -l $SCRATCH/BIDS/mriqc_clf/CN_BIDS_output/T1w.csv before and after singularity exec as above. The file modification times are not altered, and no new columns are added to T1w.csv. Additionally, echo $? (the most recent exit code) returns 0 after running singularity exec.

As mentioned by @oesteban above, I tried the -o flag, but as reported by @n2parker, it was not recognized. I suspect this is because I'm using mriqc 0.11.0 where -o is not supported.

Further troubleshooting

I re-ran the command with mriqc_clf -v to increase verbosity. The output with -v states:

> 2018-07-05 18:09:40,840 mriqc.classifier:INFO Results saved as /scratch/a/agraff/channa13/sing_home/mclf_run-20180705-180940*

Navigating to $SCRATCH/sing_home, I found the file mclf_run-20180705-180940_data-unseen_pred.csv. I then ran:

$ cat /scratch/a/agraff/channa13/sing_home/mclf_run-20180705-180940_data-unseen_pred.csv | head -n 5
> subject_id,prob_y,pred_y
002S0295,0.147785410865424,0
002S0413,0.3250233296316222,0
002S1261,0.2381351524431868,0
002S1280,0.308822544009952,0

The .csv contains 3 columns: subject_id, prob_y, and pred_y. The subject_id values correspond with the subject IDs from my BIDS input folder, while prob_y values seem to be propensity scores(?), and pred_y values seem to be the classifier's rating (i.e. 0 or 1).

Remaining issues

Based on the comment from @oesteban here, it seems like the expected behaviour of mriqc_clf is to add values in pred_y from mclf_run-*_data_unseen_pred.csv to the respective subject_id row in T1w.csv. Can someone from Poldrack Lab please confirm whether this is the case?

If the pred_y values in mclf_run-*_data-unseen_pred.csv are the correct values, one approach to resolution might be...

• If $HOME it is not writeable, then mriqc should return an error and stop.
• If $HOME is writeable, then mriqc should direct mriqc_clf to make mclf_run-*_data-unseen_pred.csv and properly append the column pred_y to T1w.csv.

Note that a new mclf_run-*_data-unseen_pred.csv is created in $HOME each time singularity exec is run. This file is not deleted when the singularity command exits. As such, using the same $HOME directory with mriqc_clf may get confusing when running multiple datasets (the .csv files are distinguished by timestamp only).

[oesteban]

Hi @n2parker, @nathankchan.

I can confirm all that @nathankchan described, and add some details:

• the mclf_run-*_data-unseen_pred.csv files are created in the working directory (not $HOME)
• my comment here is wrong. The predictions will not be added to the original T1w.csv file. Sorry for the confusion.
• the pred_y column are the prediction scores (threshold = 0.5) of the random forests.

I'll work in making the interface of the classifier more intuitive. I agree it is quite a mess at this moment. Sorry about that.