Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too much vmem? #824

Closed
soichih opened this issue Dec 18, 2019 · 2 comments
Closed

Too much vmem? #824

soichih opened this issue Dec 18, 2019 · 2 comments
Milestone

Comments

@soichih
Copy link

soichih commented Dec 18, 2019

We are running mriqc 0.14.2. Our jobs are killed often while QC-ing our anat/t1w due to too much vmem consumption. I can increase it, but doing so reduces the resources that we can run this App on.. I'd like to know if it's normal that it consumes so much vmem (>18GB)

=>> PBS: job killed: vmem 18826502144 exceeded limit 17179869184

image

The program called "fast" seems to be using a lot of memory - although other processes also uses a lot of memory.

@oesteban
Copy link
Member

Hi @soichih sorry for the very slow turnaround. FAST is part of FSL, so I have very high expectations on #823 where we will stop using it.

But there are many additional measures we can transfer from fMRIPrep here to keep VMEM under control.

The problem roots from the design of memory handling in Python on Linux platforms. Basically, anything that gets imported at some point is kept along the process.

@oesteban oesteban added this to the 0.16.0 milestone Mar 24, 2020
@oesteban
Copy link
Member

Possibly duplicate of #536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants