Tracking node memory usage in job scheduler #300

pmundkur opened this Issue Feb 15, 2012 · 0 comments


None yet
1 participant

pmundkur commented Feb 15, 2012

Although we use ulimit to limit an individual tasks memory usage, it is possible for several large tasks to consume all the memory on a node, without consuming all the slots on the node. If we don't leave some memory headroom on the node, strange failures occur, both in tasks as well as in disco/ddfs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment