Memory leak #17

chetan opened this Issue Dec 14, 2012 · 8 comments


None yet
4 participants

chetan commented Dec 14, 2012

We observed hue/django using more than 3gb memory today -

hue       3649  0.0 24.1 3900596 3711988 ?     S    Dec12   0:41 /usr/bin/python2.6

It was up for only about 3 days. We're running via CDH 4.1.2 on CentOS 6:


Not sure what other info I can provide. I'll update the ticket if we see it happening again.


romainr commented Dec 15, 2012

It is the first time we heard about this. What exact command are you using for displaying the memory usage?

Is django_debug_mode=0 in hue.ini?

chetan commented Dec 15, 2012

The output above is from ps auxw | grep django. Debug is disabled.

I'm a coworker of chetan. I checked the hue process today and saw that it was consuming 52.1 % of the RAM on the box (7.8 gigs).

hue        878  0.0 52.1 8199192 8010788 ?     S    Dec14   3:40 /usr/bin/python2.6 -c import sys; from spawning import spawning_child; spawning_child.main() 849 3 15 spawning.django_factory.config_factory {"app_factory": "spawning.django_factory.app_factory", "access_log_file": "/dev/null", "status_port": 0, "port": 8888, "verbose": null, "deadman_timeout": 1, "source_directories": ["/usr/share/hue/desktop/core/src/desktop"], "pidfile": null, "args": ["desktop.settings"], "max_age": null, "num_processes": 1, "watch": null, "host": "**************", "coverage": null, "ssl_private_key": null, "sysinfo": null, "status_host": "**************", "ssl_certificate": null, "argv_str": "--factory=spawning.django_factory.config_factory desktop.settings --port 8888 -s 1 -t 0", "no_keepalive": null, "reload": null, "django_settings_module": "desktop.settings", "threadpool_workers": 0}

romainr commented Dec 19, 2012

What is the typical usage of Hue? Any logs?

Are you using Beeswax and downloading the results directly? Opening file in JobBrowser?

I mostly use Hue for the HDFS file manager to browse and look at files. It seems that the memory usage jumps the most when just previewing files. I just tried viewing two separate 13mb text files through the file browser and memory usage jumped up to 2.4% (around 360mb) from .2% and didn't drop back down.

I've also been using Beeswax to query impala but that doesn't seem to impact the memory usage as much as just browsing files. I do use the job browser as well but haven't noticed if that is impacting memory consumption.

Let me know if there's specific log files you'd like me to upload.

romainr was assigned Dec 20, 2012


romainr commented Dec 20, 2012

I uploaded a 500MB file and when I change the length GET parameter to 40096 instead of 4096 and press F5 for a bit I indeed see the memory going up and now decreasing after.

I created


abec commented Jan 18, 2013

I've updated

Basically, spawning is not letting go of memory for requests that do not finish (IE: when a page is refreshed multiple times very quickly). This is likely do to usage... clicking next or previous block several times accidentally.

We may try to address the issue with Spawning in the future... until then, a quick solution is to use cherrypy server.


romainr commented Sep 6, 2013

Cherrypy is now the webserver in Hue 3.0

romainr closed this Sep 6, 2013

@jfota jfota pushed a commit to mapr/hue that referenced this issue Nov 17, 2015

IVarha Merge pull request #17 from mapr/mapr-20440
[MAPR-20440][Hue 3.9] AttributeError at /oozie/submit_workflow/1 'mod…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment