Ignore missing files

[chrisspen]

In large projects that use dozens of third-party libraries, invariably, some library will reference a url(../../..somefile) which doesn't exist. When running manage.py collectstatic with django-pipeline, this will result in an incredibly unhelpful error like:

ValueError: The file 'tinymce/js/tinymce/plugins/compat3x/img/icons.gif' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x1ca6750>.

Not only does pipeline not mention which file attempts to include this file, searching for img/icons.gif myself results in 42 matches. Even if I could locate the issue, I can't easily fix the missing file, since it's a third-party library. This means pipeline is effectively blocking all my deployments because of a missing file I don't even use.

Ideally, pipeline should report which resource is referencing a missing file and allow specifying an option to ignore it. This is very similar to the https://github.com/adamcharnock/django-pipeline-forgiving fork, although I think this would be a good idea for the main branch.

[edwinlunando]

Yes. me too, gets this error while using django-grappelli for admin interface. This error is not happening if you use pipeline.storage.PipelineStorage.

[cyberdelia]

Could you provide the full traceback?

[edwinlunando]

Here.

Post-processing 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.8.18.custom.css' failed!

Traceback (most recent call last):
  File "/home/edwinlunando/Development/pycharm/helpers/pycharm/django_manage.py", line 23, in <module>
    run_module(manage_file, None, '__main__', True)
  File "/usr/lib/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/edwinlunando/Projects/sabilulungan/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 167, in handle_noargs
    collected = self.collect()
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 124, in collect
    raise processed
ValueError: The file 'grappelli/jquery/ui/css/custom-theme/images/ui-bg_flat_75_ffffff_40x100.png' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x30ad850>.

[cyberdelia]

Could you try with storage.CachedStaticFilesStorage only?

[edwinlunando]

It gives the same error. Does it mean the problem is not django-pipeline?

Post-processing 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.8.18.custom.css' failed!

Traceback (most recent call last):
  File "/home/edwinlunando/Development/pycharm/helpers/pycharm/django_manage.py", line 23, in <module>
Post-processed 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.10.3.custom.min.css' as 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.10.3.custom.min.5a4566a3bbf8.css'
    run_module(manage_file, None, '__main__', True)
Post-processed 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.10.3.custom.css' as 'grappelli/jquery/ui/css/custom-theme/jquery-ui-1.10.3.custom.2d7d6027205e.css'
  File "/usr/lib/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/edwinlunando/Projects/sabilulungan/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 167, in handle_noargs
    collected = self.collect()
  File "/home/edwinlunando/.virtualenvs/sabilulungan/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 124, in collect
    raise processed
ValueError: The file 'grappelli/jquery/ui/css/custom-theme/images/ui-bg_flat_75_ffffff_40x100.png' could not be found with <django.contrib.staticfiles.storage.CachedStaticFilesStorage object at 0x21a2190>.

[cyberdelia]

staticfiles rewrite url() in CSS so that they match the hashed name (like ui-bg_flat_75_ffffff_40x100.1h2a3s4h.png).
This is the expected behavior and recent django versions indicate properly the file where it originated from (see the first line of your traceback).

My main problem with this change, is that it's a double edged sword, where all missing files goes silent, and 404 are really a nasty thing.

But I'm sure that other projects would love to know that their are shipping CSS that point to non-bundled images!

Thanks again for the help @edwinlunando!

[edwinlunando]

Sure @cyberdelia! Thank you.

[oliverseal]

Wish there was a way to make this "warn" rather than "panic!"

In the meantime, Googlers, use this:
manage.py collectstatic --ignore=grappelli
and copy the grappelli files to your static directory manually.

[karolyi]

@cyberdelia, can we have warnings instead of errors on compilation? As you can see, I'm bumping into more and more errors because of this, and we can't expect every one of them get corrected. So I have to introduce more and more -i flags with comments that they should be removed when those bugs get fixed.

The problem is, since I can't specify paths explicitly (-i only accepts filenames without paths), sooner or later we will run into problems when having a complex project with many packages having overlapping (and needed) filenames.

[karolyi]

yeah, on a side note, some people like @Eonasdan are even unwilling to resolve the problem, against my best intentions. he just deleted the issue I created and banned me in his repo, after making a phlegmatic comment. what a dick move.

so because of this, the problem here won't be solved on his behalf, since he is unwilling to help.

[Eonasdan]

actually @karolyi I disabled all the issues. I don't have time to work on the picker at the moment.

"No is a complete sentence".

Guess you'll have to get over yourself

[yairchu]

We're using "slider revolution" which has this line in a css:

.tp-bannertimer { visibility: hidden; width:100%; height:5px; /*background:url(../assets/timer.png);*/ background: #fff; background: rgba(0,0,0,0.15); position:absolute; z-index:200; top:0px}

It seems to reference ../asserts/timer.png (which doesn't exist) but it's actually just in a comment, not a real reference.

This also triggers this problem. In our case we've removed the comment to work around this and then everything worked fine..

[czue]

Had this same issue using lodash and (via dependency) firebug-lite. The error I was getting was

ValueError: The file 'lodash/vendor/firebug-lite/skin/xp/condBorders.png' could not be found with <django.contrib.staticfiles.storage.ManifestStaticFilesStorage object at 0x7fd3ed9b9ad0>.

And the solution as mentioned above is to run:

./manage.py collectstatic --ignore=firebug-lite

[diegueus9]

I just had the same issue with jquery-compiler, as @czue suggested, the flag --ignore did the trick

./manage.py collectstatic --no-input -c -v 3 --ignore=jquery-confirm

However, it would be nice to have this issue fixed