Tracking node memory usage in job scheduler #300

Open
pmundkur opened this Issue Feb 15, 2012 · 0 comments

Comments

Projects
None yet
1 participant
@pmundkur
Member

pmundkur commented Feb 15, 2012

Although we use ulimit to limit an individual tasks memory usage, it is possible for several large tasks to consume all the memory on a node, without consuming all the slots on the node. If we don't leave some memory headroom on the node, strange failures occur, both in tasks as well as in disco/ddfs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment