High RAM memory consume previewing gif

[SalahAdDin]

Loading a gif inside the Administration makes very slow the computer: First time i see this performance inside a Docker container, and i reviewed it in local environment and i got the same result.

Issue Summary

Each time that we upload animated gif and after we go to preview it(after creating the new image or since image section), the computer's memory becomes full and the SO have to kill django server for get back normally.

[30/Apr/2017 10:41:49] "GET /admin/tag-autocomplete/?term=soluci%C3%B3n HTTP/1.1" 200 2
[30/Apr/2017 10:41:53] "POST /admin/images/multiple/2/ HTTP/1.1" 200 32
[30/Apr/2017 10:42:06] "GET /admin/images/ HTTP/1.1" 200 4071
[30/Apr/2017 10:42:06] "GET /media/images/problem.max-165x165.gif HTTP/1.1" 200 522647
[30/Apr/2017 10:42:06] "GET /static/wagtailadmin/images/transparency.svg HTTP/1.1" 304 0
[1]    13384 killed     ./manage.py runserver

Steps to Reproduce

1. Create a new image
2. Full all fields
3. Load a gif image
4. Save.

Or:

1. Open and page or image in the admin

Technical details

• Python version: 3.6.1.
• Django version: 1.10.7
• Wagtail version: Hover over the Wagtail bird in the admin, or look in the Settings menu, look in your requirements.txt, or run 1.9.
• Browser version: Chrome
• Sistema operativo: x86_64 Linux 4.10.11-1-ARCH
• Wand version: 0.4.4
• Pillow version: 4.1.1
• Imagemagick version: 6.9.8.4-2

[gasman]

@SalahAdDin What's the file size of the GIF you're uploading?

[SalahAdDin]

1.0 Mb.

[BertrandBordage]

Which is somewhat reasonable given the heavy weight of most GIFs.

[SalahAdDin]

Yes.

[LuisFGomez]

This also bit me just as I was preparing to deploy my first wagtail site! I never had a problem under the development server (Windows 7).

Mine is an animated gif, if that matters. Size if 58kB. RAM consumption got so high that my shared host had to kill other processes associated with my account (with my production django app along with it). I can't have wagtail causing downtime.

lgomez - 399MB - 22:11:44 - 30746 - /home/lgomez/webapps/labwerx_staging/apache2/bin/httpd.worker -f /home/lgomez/webapps/labwerx_staging/apache2/conf/httpd.conf -k start

lgomez - 402MB - 22:11:44 - 30747 - /home/lgomez/webapps/labwerx_staging/apache2/bin/httpd.worker -f /home/lgomez/webapps/labwerx_staging/apache2/conf/httpd.conf -k start

Using:

• wagtail 1.9.1
• Wand 0.4.4
• ImageMagick 6.7.8-9 2016-06-16 Q16
• Django 1.10.7

So, this is now the only thing stopping me from going live with wagtail. Let me know if I can help in any way.

[kxxoling]

I built a wagtail site to serve images, some of the images can be more than 10MB, the site would take memory in GBs and an unbearable CPU time which leading to a 504 response.

I guess this is because gif image resizing takes too much resource, but I really don't care if animated images were compressed or not. Can I disable resize for gif images separately?

[gasman]

Given that these reports have just started appearing recently (and as far as I'm aware there haven't been any recent changes on the Wagtail / Willow side regarding .gif support), I suspect this is a performance regression in the latest version of Wand or Imagemagick. I'd recommend downgrading to an earlier version to try and pinpoint when the regression occurred.

[kxxoling]

My problem seems to be solved after upgrading Wand to new version:

        "Wand": {
            "version": "==0.4.4",
            "hash": "sha256:fb50d0ad4cad995f0b59b13f76bced22682ab80b4299084a2c3c535b225850c9"
        },

[SalahAdDin]

@kxxoling I will test it.

[kaedroho]

@SalahAdDin Did the upgrade fix the problem for you as well?

[SalahAdDin]

@kaedroho It still happens:

[12/Nov/2017 18:02:30] "POST /admin/images/chooser/upload/ HTTP/1.1" 200 231
[12/Nov/2017 18:02:36] "GET /media/images/deepin-screen-recorder_Select_area_201709020.max-165x165.gif HTTP/1.1" 200 762554
[12/Nov/2017 18:04:21] "GET /admin/ HTTP/1.1" 302 0
[12/Nov/2017 18:04:24] "GET /admin/pages/4/edit/ HTTP/1.1" 302 0
Not Found: /blogs/feed/$
[12/Nov/2017 18:04:41] "GET /admin/login/?next=/admin/pages/4/edit/ HTTP/1.1" 200 3155
[12/Nov/2017 18:04:41] "GET /blogs/feed/$ HTTP/1.1" 404 2548

[12/Nov/2017 18:09:31] "GET /media/images/deepin-screen-recorder_Select_area_201709020.max-165x165.gif HTTP/1.1" 304 0

@kaedroho Ok, it works now, but it have other bug:
I tried to add a gif inside the editor throuth a stream block BustOut, it takes all browser memory and finally i have to re open itself, restoration. After it i had to entered again in the editor: login, and edit again.
It was a double surprise, now i can see the gif in rendered pages, in the editor's images section, it takes time, and in the page's editor itself, yes, but when i try to add a new BustOut stream block to the page i get errors, one of them you can see in the next gif: i can edid text field, delete these block, and delete the black section, i can delete other blocks, and when i want save it, i got an error. It only happens with these block; and the worst thing is that seems that this event corrupted the page and now i get this bug all time, it doesn't matter if i clear the cache, stop the server and etc.

[BertrandBordage]

It also happened to me with an 8 MB JPEG, leading to a 400 MB RAM consumption until Linux stops the process, leading to a 502 error on nginx for a few seconds.

[japhiaolson]

Have the same issue here.

Using the following:
Wand 0.4.4
Local windows: ImageMagick-6.9.9-50-Q8-x86-dl exe
Ubuntu staging: sudo apt-get install libmagickwand-dev
Wagtail 2.1
Uploading a 1mb file uses 100% CPU and takes a few minutes to finish locally. On my staging server it spikes ram/cpu/disk usage for a minute or so before giving an nginx 502 error on 200kb+ animated gif files.

[evantbyrne]

We encountered what appears to be the same issue. After uploading a 600kb animated gif, the images picker in the admin wouldn't even load. Timeouts and memory errors. The solution was to uninstall Wand.

[JaredOzzy]

Also Ran into this issue today.
Our clusters RAM was maxing out after adding a GIF.

We removed Wand as @evantbyrne suggested, but it seems now any subsequent gif that is added will not work, as all the previous gifs that were added will work.(what i expected).

[kaedroho]

There's a couple of fixes to this upcoming:

• [5225] Take number of frames into account when limiting image size #5227 Which will limit the total number of pixels in animated GIFs to 100 million (this is adjustable)
• Pillow animated gifs Willow#76 Which adds support for resizing animated GIFs using Pillow, which is generally more efficient on memory usage than Wand

[BertrandBordage]

That’s great! We’ll finally have meme websites in Wagtail!

[phette23]

I'm wondering if anyone has advice on working around this issue. I've tried updating to the 2.7rc, updating Willow, setting WAGTAILIMAGES_MAX_IMAGE_PIXELS to low values (which, after all, doesn't fix the issue it just stops it from being debilitating). But no matter what it seems that even fairly small GIFs (128*50000 max pixels, file sizes well under 1mb) cause 502 errors and their derivatives are never generated. Does anyone have a site that successfully does this? Wagtail works fine on my macbook localhost it's just on our (Ubuntu) server where I cannot make it work.

[jkevingutierrez]

Having the same issue, I was wondering if there has been any progress on this?