Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Severe memory leaks on Amazon AWS Linux platform #2646

Closed
spudthorpe opened this issue Oct 20, 2017 · 1 comment

Comments

Projects
None yet
3 participants
@spudthorpe
Copy link

commented Oct 20, 2017

Please see thread:
https://discourse.pymc.io/t/excessive-memory-usage-in-pymc3-solved-aws-linux-platform-issue-works-on-aws-windows/446/9

PyMC3 3.2 and 3.1, Python 3.5.2, Theano 0.9.0, Ubuntu 16.04

Tried running the notebook at https://github.com/twiecki/WhileMyMCMCGentlySamples/blob/master/content/downloads/notebooks/random_walk_deep_net.ipynb

on an AWS Ubuntu instance. The python process allocated about 100MB of memory per second; allocated memory kept growing until the machine was effectively dead from memory swapping. The notebook exhausted memory on 61GB, 122GB, and 244GB instances. I also saw similar behavior on Amazon Linux but primarily investigated on Ubuntu. I tried the HDF5 backend and it did not make a difference.

The same setup worked fine on an AWS Windows instance.

There are other issues open reporting a similar problem:
#1825
#1959

@junpenglao

This comment has been minimized.

Copy link
Member

commented Jan 13, 2018

Hi @spudthorpe, could you try again from pymc3 master? @ferrine removed a few theano copy and the issue might be fixed now.

@twiecki twiecki closed this Dec 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.