Disable Logging panel when DEBUG is set to False to prevent memory leaks #563

Open
wants to merge 5 commits into
from

Projects

None yet

3 participants

@stanhu
stanhu commented Mar 16, 2014

I upgraded django-debug-toolbar from 0.9.4. to 1.0.1, and I noticed my application was quickly leaking memory even when settings.DEBUG was set to False. Once I removed debug_toolbar from the list of applications, the problem disappeared.

Upon further investigation, I found that LoggingPanel was quietly collecting all my log messages and never releasing them.

This problem only occurs if the Django application is fully initialized (e.g. using runserver), since this causes the LoggingPanel to register its handlers. Here is a simple application that illustrates the problem:

from django.core.management import ManagementUtility
import logging

utility = ManagementUtility()
command = utility.fetch_command('runserver')
command.validate()

logger = logging.getLogger('test')

while True:
    logger.info('This logger will leak memory')

Run this application with the default debug_toolbar settings, and then use 'top' to watch the memory grow without bound. The problem appears that the LoggingPanel does not use the enable_instrumentation() function normally called by the middleware, which does the DEBUG check beforehand.

I think the simplest fix is to check if DEBUG is enabled before initializing the LogCollector. This pull request does just that, and it appears to fix the issue.

In addition, I think we should consider a few other things:

  1. Warn the user that under DEBUG mode, memory used for log messages can grow without bounds (this might be true for other panels already).

  2. Limit the number/age of messages stored (perhaps as a configuration option).

@aaugustin aaugustin added the Bug label Mar 25, 2014
@aaugustin
Contributor

I think we have to bite the bullet and write a logging.Filter that checks a thread-local flag set by enable/disable_instrumentation if we want to avoid running into further problems.

@aaugustin aaugustin closed this Mar 25, 2014
@aaugustin aaugustin reopened this Mar 25, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment