qacct = '\n'.join(master.ssh.execute(qacct_cmd, log_output=True,
Setting "log_output=True" results in large (and ever growing) memory usage on the balancer when qacct returns a long job history, regardless of the log level.
A bit more investigation, from the plugins documentation:
"All messages logged with the log.debug() method will always be printed to the debug file, however, these messages will only be printed to the screen if the user passes the –debug flag to the starcluster command."
This is the reason for the memory behaviour. For the load balancer which is likely to be running continuously for longer periods this may not be the best idea.
@scrappythekangaroo Good catch, although it's not because of logging to a file it's because I'm storing the session log in memory (via a StringIO log handler) in order to conveniently build crash-reports when uncaught exceptions are thrown. Clearly I wasn't quite thinking when I wrote this :P Fixing now...