Deconvolving HRF step takes a huge amount of time #51
Comments
|
Thanks a lot for your message.
Could you please specify the architecture of the machine you are running
the toolbox on?
Also, can you please specify "very long"?
The Matlab SPM plugin takes a couple of minutes to run on a standard
desktop pc for one subject in native space.
…On Sun, Dec 2, 2018 at 2:00 AM Stephen L. ***@***.***> wrote:
The HRF estimation seems to be quite fast (particularly with microtime
resolution 1 and AR off), but the "deconvolving HRF" step always takes very
very long, even for just 300 volumes.
Is there a way to make this step faster?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#51>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AFEKfr2pipgR_Nm_Zn4QV6ph3n_Jtjdtks5u0yYbgaJpZM4Y9LYd>
.
|
|
Something else came to our minds, it might be that you do not use a mask, and then remove the outliers and replace these missing values (including a huge number of voxels outside the brain). |
|
Dear Daniele, Thank you very much for your fast replies and also for your email! I take this opportunity to link to the solution to combine CONN and rshrf. About the time it takes, I reworked further my pipeline to generate additional info that might be useful to you. Here is the log I get: This is a post-processing on 9 subjects with 2 sessions (S2 and W1) for each. As you can see, the first deconvolution is quite quick, then for each session it becomes longer and longer, approximately 10x more time for each. I tried several times, modifying some parameters until I came back to near the default settings in rshrf, and it still happens the same, I never went beyond subject 3 session S2 (first session), because if the exponential increase is true, it would approximately take 50hours to complete (currently it's been running since 30 hours to no avail). More precisely, here are the deconvolution runtimes so far from the log: 254s, 88s, 5416s, 22804s. Your hypothesis that I don't use a mask is correct, but can this explain why the time is increasing between sessions? If it's because of the mask, I would expect the time increase to be constant for the same number of volumes and voxel resolution, which hold for my dataset. I attach the jobs that my script is dynamically generating, maybe you can find more info from this? jobs.zip I can also make available anything else if it can help, just let me know. Thank you very much for taking the time to help me on this! |
|
Dear Stephan thanks a lot for the update on the interface with CONN, I'm sure it will be useful to others. The info that you provided indeed allows to understand that the issue is in the deconvolution phase. We will look into the jobs. For the moment, can you just try to run one subject/session per time, or change the order? Just to see if there is actually an increasing/accumulating delay (as it seems), or if some subjects/sessions are problematic. |
|
Thank you Daniele for this great idea, I'm running now the pipeline by beginning with the 3rd subject, so far it still did not finish, so it seems something in the data is making rshrf deconvolution go wild. I let it run for the night to make sure and if it did not finish, tomorrow I can send you an excerpt of the dataset if you agree. |
|
BTW my pipeline is here: https://github.com/lrq3000/csg_mri_pipelines/blob/master/preprocessing/fmri/script_preproc_fmri_csg.m#L571 It's temporary as I will change it to process after CONN (currently it's done just before, which is wrong), but the rshrf parameters won't change. |
|
matlabbatch{1}.spm.tools.HRF.vox_rsHRF.rmoutlier = 1; % remove outlier
matlabbatch{1}.spm.tools.HRF.vox_rsHRF.mask = {''}; % no mask included
matlabbatch{1}.spm.tools.HRF.vox_rsHRF.outdir = {''}; % save to current folder
I suggest you try to set
*.rmoutlier = 0;
*.mask = {fullfile(spm(‘Dir’),’tpm’,'mask_ICV.nii')};
… On 2 Dec 2018, at 09:00, Stephen L. ***@***.*** ***@***.***>> wrote:
The HRF estimation seems to be quite fast (particularly with microtime resolution 1 and AR off), but the "deconvolving HRF" step always takes very very long, even for just 300 volumes.
Is there a way to make this step faster?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <#51>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ATmXf0YqoarbLxzA0kZrDnsquLQkJGXoks5u0yYbgaJpZM4Y9LYd>.
|
|
@guorongwu Thank you very much! Adding the mask indeed solved the issue (rmoutlier can be set to 1). May I suggest to add this template as the default value in the rshrf batch? I thought that rmoutlier was necessary, what would you guys advise for a typical resting state functional connectivity analysis? And what about AR correction? Thank you very much! |
|
in your pipeline, it's performed on normalized MNI space, I think you can
add it as default value.
I will also remove outlier, but if too much 'outlier' include, this step
may 'change' the final deconvolved signals (similar to bad EEG channels for
EEG analysis).
we can use AR correction, I do not suggest you set a high AR order. the
code may run faster if no AR correction. it will not achieve convergence in
some signals, then without AR correction are better for these singals.
…On Thu, Dec 6, 2018 at 5:14 AM Stephen L. ***@***.***> wrote:
@guorongwu <https://github.com/guorongwu> Thank you very much! Adding the
mask indeed solved the issue (rmoutlier can be set to 1).
May I suggest to add this template as the default value in the rshrf batch?
I thought that rmoutlier was necessary, what would you guys advise for a
typical resting state functional connectivity analysis? And what about AR
correction?
Thank you very much!
Best regards
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#51 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATmXf2eKio0X8yi9M8WbmNxQH25c7eWPks5u2Dc-gaJpZM4Y9LYd>
.
|
|
Thank you very much @guorongwu for the info! I am not sure I understand about the remove outlier option, if I enabled it (inpainting outliers with NaN) this can change the final deconvolved signal (or is it the other way around?). For the default template, yes my pipeline is using MNI space, but since most SPM based pipelines do the same I think it would really ease newcomers to get a hang of the rshrf toolbox if this template was provided as the default, or if not, at least in the help message? I have a final but very crucial question: can the rshrf toolbox work on smoothed data? Because the way CONN author advised to combine CONN + rshrf is to use rshrf after CONN denoising, which is done on smoothed data. If rshrf cannot work on smoothed data, would it be possible/mathematically meaningful to do the hrf deconvolution calculation on non-smoothed data, save it and apply it later on the smoothed data? What I have in mind is something like a normalization/DARTEL transform? |
|
it only change the outlier signals (outlier detected from response height)
actually, we include several demo jobs, which provide example to include
different masks (native or MNI space).
sure, you can use the smoothed data, it can increase SNR.
…On Fri, Dec 7, 2018 at 9:45 AM Stephen L. ***@***.***> wrote:
Thank you very much @guorongwu <https://github.com/guorongwu> for the
info!
I am not sure I understand about the remove outlier option, if I enabled
it (inpainting outliers with NaN) this can change the final deconvolved
signal (or is it the other way around?).
For the default template, yes my pipeline is using MNI space, but since
most SPM based pipelines do the same I think it would really ease newcomers
to get a hang of the rshrf toolbox if this template was provided as the
default, or if not, at least in the help message?
I have a final but very crucial question: can the rshrf toolbox work on
smoothed data? Because the way CONN author advised to combine CONN + rshrf
is to use rshrf after CONN denoising, which is done on smoothed data. If
rshrf cannot work on smoothed data, would it be possible/mathematically
meaningful to do the hrf deconvolution calculation, save it and apply it on
the smoothed data?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#51 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATmXf9EepuXYaxyL8FdKMqBa05wtvZHtks5u2cgvgaJpZM4Y9LYd>
.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The HRF estimation seems to be quite fast (particularly with microtime resolution 1 and AR off), but the "deconvolving HRF" step always takes very very long, even for just 300 volumes.
Is there a way to make this step faster?
The text was updated successfully, but these errors were encountered: