"Memory Leak" on load balancer #157

Closed
scrappythekangaroo opened this Issue Oct 28, 2012 · 2 comments

Comments

Projects
None yet
2 participants

In starcluster.balancers.sge.init._get_stats:

qacct = '\n'.join(master.ssh.execute(qacct_cmd, log_output=True,
ignore_exit_status=True))

Setting "log_output=True" results in large (and ever growing) memory usage on the balancer when qacct returns a long job history, regardless of the log level.

A bit more investigation, from the plugins documentation:

"All messages logged with the log.debug() method will always be printed to the debug file, however, these messages will only be printed to the screen if the user passes the –debug flag to the starcluster command."

This is the reason for the memory behaviour. For the load balancer which is likely to be running continuously for longer periods this may not be the best idea.

Owner

jtriley commented Oct 31, 2012

@scrappythekangaroo Good catch, although it's not because of logging to a file it's because I'm storing the session log in memory (via a StringIO log handler) in order to conveniently build crash-reports when uncaught exceptions are thrown. Clearly I wasn't quite thinking when I wrote this :P Fixing now...

jtriley closed this in 2fb3aca Oct 31, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment