All files/folders missing after upgrade >= 1.0 with DEBUG == False

[jgoettsch]

The files are still there on the filesystem and show up in the admin if I set DEBUG = True or revert to an older version.

Here are my package versions:

$ pip freeze
Django==1.7.11
django-filer==1.0.4
django-mptt==0.6.1
django-polymorphic==0.7.2
easy-thumbnails==2.2.1
Pillow==3.0.0

[skirsdeda]

Confirmed. Actually this happens in a template, so you can have DEBUG=False; TEMPLATE_DEBUG=True. Something to do with this ('filer/icons/plainfolder_16x16.png' not found while rendering 'directory_listing.html' template):

[Thu Dec 03 15:42:40.108978 2015] [:error] [pid 18649] Traceback (most recent call last):
[Thu Dec 03 15:42:40.108990 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/core/handlers/base.py", line 111, in get_response
[Thu Dec 03 15:42:40.109003 2015] [:error] [pid 18649]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Thu Dec 03 15:42:40.109015 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/utils/decorators.py", line 105, in _wrapped_view
[Thu Dec 03 15:42:40.109026 2015] [:error] [pid 18649]     response = view_func(request, *args, **kwargs)
[Thu Dec 03 15:42:40.109035 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/views/decorators/cache.py", line 52, in _wrapped_view_func
[Thu Dec 03 15:42:40.109042 2015] [:error] [pid 18649]     response = view_func(request, *args, **kwargs)
[Thu Dec 03 15:42:40.109049 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/contrib/admin/sites.py", line 206, in inner
[Thu Dec 03 15:42:40.109056 2015] [:error] [pid 18649]     return view(request, *args, **kwargs)
[Thu Dec 03 15:42:40.109063 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/filer/filer/admin/folderadmin.py", line 416, in directory_listing
[Thu Dec 03 15:42:40.109072 2015] [:error] [pid 18649]     return render(request, self.directory_listing_template, context)
[Thu Dec 03 15:42:40.109101 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/shortcuts.py", line 50, in render
[Thu Dec 03 15:42:40.109115 2015] [:error] [pid 18649]     return HttpResponse(loader.render_to_string(*args, **kwargs),
[Thu Dec 03 15:42:40.109127 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/loader.py", line 178, in render_to_string
[Thu Dec 03 15:42:40.109139 2015] [:error] [pid 18649]     return t.render(context_instance)
[Thu Dec 03 15:42:40.109152 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 148, in render
[Thu Dec 03 15:42:40.109163 2015] [:error] [pid 18649]     return self._render(context)
[Thu Dec 03 15:42:40.109173 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 142, in _render
[Thu Dec 03 15:42:40.109184 2015] [:error] [pid 18649]     return self.nodelist.render(context)
[Thu Dec 03 15:42:40.109196 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109208 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109220 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109239 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109251 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/loader_tags.py", line 126, in render
[Thu Dec 03 15:42:40.109262 2015] [:error] [pid 18649]     return compiled_parent._render(context)
[Thu Dec 03 15:42:40.109274 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 142, in _render
[Thu Dec 03 15:42:40.109285 2015] [:error] [pid 18649]     return self.nodelist.render(context)
[Thu Dec 03 15:42:40.109297 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109310 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109321 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109333 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109346 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/loader_tags.py", line 126, in render
[Thu Dec 03 15:42:40.109357 2015] [:error] [pid 18649]     return compiled_parent._render(context)
[Thu Dec 03 15:42:40.109369 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 142, in _render
[Thu Dec 03 15:42:40.109380 2015] [:error] [pid 18649]     return self.nodelist.render(context)
[Thu Dec 03 15:42:40.109392 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109404 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109416 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109428 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109439 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/loader_tags.py", line 126, in render
[Thu Dec 03 15:42:40.109451 2015] [:error] [pid 18649]     return compiled_parent._render(context)
[Thu Dec 03 15:42:40.109463 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 142, in _render
[Thu Dec 03 15:42:40.109475 2015] [:error] [pid 18649]     return self.nodelist.render(context)
[Thu Dec 03 15:42:40.109486 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109504 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109516 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109528 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109541 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/loader_tags.py", line 65, in render
[Thu Dec 03 15:42:40.109552 2015] [:error] [pid 18649]     result = block.nodelist.render(context)
[Thu Dec 03 15:42:40.109564 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109576 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109587 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109599 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109611 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/defaulttags.py", line 312, in render
[Thu Dec 03 15:42:40.109622 2015] [:error] [pid 18649]     return nodelist.render(context)
[Thu Dec 03 15:42:40.109633 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 844, in render
[Thu Dec 03 15:42:40.109646 2015] [:error] [pid 18649]     bit = self.render_node(node, context)
[Thu Dec 03 15:42:40.109657 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 858, in render_node
[Thu Dec 03 15:42:40.109669 2015] [:error] [pid 18649]     return node.render(context)
[Thu Dec 03 15:42:40.109681 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 898, in render
[Thu Dec 03 15:42:40.109693 2015] [:error] [pid 18649]     output = self.filter_expression.resolve(context)
[Thu Dec 03 15:42:40.109704 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 596, in resolve
[Thu Dec 03 15:42:40.109715 2015] [:error] [pid 18649]     obj = self.var.resolve(context)
[Thu Dec 03 15:42:40.109725 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 734, in resolve
[Thu Dec 03 15:42:40.109736 2015] [:error] [pid 18649]     value = self._resolve_lookup(context)
[Thu Dec 03 15:42:40.109746 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/template/base.py", line 770, in _resolve_lookup
[Thu Dec 03 15:42:40.109756 2015] [:error] [pid 18649]     current = getattr(current, bit)
[Thu Dec 03 15:42:40.109766 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/filer/filer/models/mixins.py", line 19, in icons
[Thu Dec 03 15:42:40.109776 2015] [:error] [pid 18649]     self._icon, size, size))
[Thu Dec 03 15:42:40.109786 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/contrib/staticfiles/templatetags/staticfiles.py", line 9, in static
[Thu Dec 03 15:42:40.109797 2015] [:error] [pid 18649]     return staticfiles_storage.url(path)
[Thu Dec 03 15:42:40.109806 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/contrib/staticfiles/storage.py", line 128, in url
[Thu Dec 03 15:42:40.109816 2015] [:error] [pid 18649]     hashed_name = self.stored_name(clean_name)
[Thu Dec 03 15:42:40.109826 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/contrib/staticfiles/storage.py", line 277, in stored_name
[Thu Dec 03 15:42:40.109836 2015] [:error] [pid 18649]     cache_name = self.clean_name(self.hashed_name(name))
[Thu Dec 03 15:42:40.109846 2015] [:error] [pid 18649]   File "/home/mic/.virtualenvs/lmic_prod/src/django/django/contrib/staticfiles/storage.py", line 91, in hashed_name
[Thu Dec 03 15:42:40.109865 2015] [:error] [pid 18649]     (clean_name, self))
[Thu Dec 03 15:42:40.109880 2015] [:error] [pid 18649] ValueError: The file 'filer/icons/plainfolder_16x16.png' could not be found with <lmic.storages.StaticFilesStorage object at 0x7f762bb2ce90>.

[skirsdeda]

After further investigation I found that problem lies in IconsMixin which adds icons attribute on filer models. Currently it just gets FILER_ADMIN_ICON_SIZES settings, iterates through sizes and returns links to actual icons of different sizes.

The problem is that by default FILER_ADMIN_ICON_SIZES=('16', '32', '48', '64',) and some icons are missing (like 'plainfolder' 16x16). So a quick fix for now is to change FILER_ADMIN_ICON_SIZES in your own settings like this:

FILER_ADMIN_ICON_SIZES = ('32', '48', '64')

[jgoettsch]

I can confirm this fix works. Thanks for the quick response!

[vstoykov]

We also have the same problem. It occurs with DEBUG = False and one of CachedStaticFilesStorage or ManifestStaticFilesStorage are used.

I see from the traceback provided by @skirsdeda that he uses custom StaticFilesStorage and probably this storage inherits on of CachedStaticFilesStorage or ManifestStaticFilesStorage.

There is few possible solutions of the problem:

1. Simple solutions will be to use try:..except: around settings the icon url in the IconMixin. This will ensure that if there is no 16x16 icon of given type it will not fail. Something like:

   @property
   def icons(self):
       r = {}
       if getattr(self, '_icon', False):
           for size in FILER_ADMIN_ICON_SIZES:
               try:
                   r[size] = static("filer/icons/%s_%sx%s.png" % (
                       self._icon, size, size))
               except ValueError:
                   pass
       return r
   

2. Other solutions is just to add this icons for all types

3. As last resort all 16x16 icons can be deleted and default settings for FILER_ADMIN_ICON_SIZES to be set to ('32', '48', '64') as @skirsdeda said.

I think that core developers need to decide which solution they prefer.

[skirsdeda]

@vstoykov Sure, I am inheriting from ManifestStaticFilesStorage but I am also hosting static files on S3 so this for size in FILER_ADMIN_ICON_SIZES: cycle doesn't make me happy either :)

Not only should it be wrapped in try:..except: but it also should be lazy (no unnecessary static calls). This could be simply implemented with another class with __getattr__ which would contain the try:...except: block.

[vstoykov]

@skirsdeda you are using ManifestStaticFilesStorage then it will cache all urls in memory from the manifest file and calls to storage.url method are not so expensive.

For reference https://github.com/django/django/blob/stable/1.9.x/django/contrib/staticfiles/storage.py#L293

Still when list many files from the same type there is no need for every file to calculate urls of all possible icons and then use only one of them. The best will be to cache them in the class itself.

[skirsdeda]

@vstoykov I agree.

[stefanfoulis]

I removed the 16x16 icons and removed the 16px size from the default settings. I hope this resolves the issue. Otherwise please re-open.