Extremely slow speed (with full comparable profiling files)

[yybot1]

I tested django-debug-toolbar and django silk with a simple rest api (Django Rest Framework browsable API on browsers) which just uses factory boy creating 500 records. And I noticed django-debug-toolbar is extremely slow.

with no django-debug-toolbar no silk = 3 seconds
with silk = 7 seconds
with django-debug-toolbar = 47 seconds

I did run the api several times each, so this is average time.

Maybe there are some bugs with django-debug-toolbar? Or there could be some improvements?

I used Pyinstrument creating full-stack profiling files for the tests and providing them below. They are profiling files for the same api so the results should be comparable. Hope they could be used for debugging.

Python 3.6.4
Windows 10
Sqlite database disk file
Django Rest Framework browsable API on browsers
pip list
astroid (1.6.2)
attrs (17.4.0)
autopep8 (1.3.5)
certifi (2018.1.18)
chardet (3.0.4)
colorama (0.3.9)
decorator (4.2.1)
defusedxml (0.5.0)
Django (2.0.3)
django-allauth (0.35.0)
django-debug-toolbar (1.9.1)
django-extensions (2.0.6)
django-oauth-toolkit (1.0.0)
django-rest-auth (0.9.3)
django-silk (2.0.0)
djangorestframework (3.7.7)
factory-boy (2.10.0)
Faker (0.8.12)
gprof2dot (2016.10.13)
idna (2.6)
ipython (6.2.1)
ipython-genutils (0.2.0)
isort (4.3.4)
jedi (0.11.1)
Jinja2 (2.10)
lazy-object-proxy (1.3.1)
line-profiler (2.1.2)
MarkupSafe (1.0)
mccabe (0.6.1)
more-itertools (4.1.0)
oauthlib (2.0.7)
parso (0.1.1)
pickleshare (0.7.4)
pip (9.0.3)
pluggy (0.6.0)
prompt-toolkit (1.0.15)
py (1.5.3)
pycodestyle (2.3.1)
Pygments (2.2.0)
pyinstrument (2.0.2)
pyinstrument-cext (0.1.2)
pylint (1.8.3)
pylint-django (0.9.4)
pylint-plugin-utils (0.2.6)
pytest (3.5.0)
pytest-django (3.1.2)
python-dateutil (2.7.2)
python3-openid (3.1.0)
pytz (2018.3)
requests (2.18.4)
requests-oauthlib (0.8.0)
rope (0.10.7)
selenium (3.11.0)
setuptools (39.0.1)
simplegeneric (0.8.1)
six (1.11.0)
snakeviz (0.4.2)
sqlparse (0.2.4)
text-unidecode (1.2)
tornado (5.0.1)
traitlets (4.3.2)
urllib3 (1.22)
wcwidth (0.1.7)
Werkzeug (0.14.1)
wheel (0.30.0)
wrapt (1.10.11)

django-debug-toolbar-silk-pyinstrument-profiling.zip

[mmmcorpsvit]

i have too problems on windows, check: #1080

[notanumber]

I have what I assume is a similar issue. If I run Django Debug Toolbar in a Linux VM on Windows it runs fine but if I try it in a Docker container it's insanely slow. We're talking upwards of 190s to render a single page. Profiling seems to indicate it's making over 100,000 calls to pprint and that's where a good 90% of the time is spent.

Is there a setting or configuration I should be changing? It seems odd to me that the same page works fine in Linux but slows to a crawl in a Docker container when there's no change to any of the source files.

[notanumber]

After further testing, it seems to be something in the Templates panel. If I uncheck that, I have a page that takes 4790ms to render, not great, but bearable. If I check this panel on the same page it takes 102,020ms to render.

If I also enable to the non-default profiling panel, the render time on this page is upwards of 728,850ms but is showing me debug_toolbar/panels/templates/panel.py in _store_template_info making 14,679 call to pprint.

[matthiask]

We have some problems with the template panel especially since Django changed to render form widgets using templates. We try caching (memoizing) the result of pprint resp. pformat calls now but obviously this isn't enough. I'd very much appreciate pull requests improving the speed of the templates panel.

I'm a bit surprised that the speed is still slow after deactivating the templates panel though. Maybe there's also something else amiss?

[notanumber]

Oh, I'm not blaming the Debug Toolbar for the 4790ms time when the template panel is turned off. That's on me. The page I'm looking at with these times is a beast and in desperate need of a refactor to improve speed. Unfortunately, it's something that just keeps getting pushed back in the queue.

Good to hear that it's not just me that seeing the issue though. If I can carve out some time to see if I can help to improve the template panel, I'd be happy to submit a pull request. Thanks!

[viwal]

Hi guys, I seem to have similar problem running my Django app in docker on MaOS. I tried some configurations and found that debug_toolbar.panels.profiling.ProfilingPanel is the one causing problem. So kind of workaround could be just disable it.