Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work around "too many open files" by slurping the data #2163

Merged
merged 1 commit into from Feb 21, 2019

Conversation

zougloub
Copy link
Contributor

@zougloub zougloub commented Feb 13, 2019

We're just replacing use of mmap'able .nii files by .nii.gz files, which need to be read entirely, which reduces the number of open files. Obviously now the bottleneck becomes RAM.

Fixes #2149
Related to: https://sourceforge.net/p/spinalcordtoolbox/help/13/

@zougloub
Copy link
Contributor Author

A proper solution still uses mmap()'ed data, so the data is accessed on-demand and the computer RAM is not a limitation.
The moco functions are splitting an input 4D file in 3D files, but it's not necessary to do so, the 3D files could still be looked up from the original data.

@jcohenadad jcohenadad changed the title moco: work around "too many open files" by slurping the data (#2149) moco: work around "too many open files" by slurping the data Feb 13, 2019
@jcohenadad jcohenadad added bug category: fixes an error in the code sct_dmri_moco context: sct_fmri_moco context: labels Feb 13, 2019
@jcohenadad jcohenadad added this to the v4.0.0 milestone Feb 13, 2019
@jcohenadad jcohenadad changed the title moco: work around "too many open files" by slurping the data Work around "too many open files" by slurping the data Feb 13, 2019
@zougloub zougloub force-pushed the cJ/2149-moco-toomanyfiles branch 4 times, most recently from 7f03cf6 to d8a5a86 Compare February 18, 2019 02:04
Copy link
Member

@jcohenadad jcohenadad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

successfully tested 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug category: fixes an error in the code sct_dmri_moco context: sct_fmri_moco context:
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants