New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Video processing fails with 0.8.0 #110

Closed
sebiii opened this Issue Sep 4, 2014 · 9 comments

Comments

2 participants
@sebiii

sebiii commented Sep 4, 2014

Hi,

after upgrading to 0.8.0 i found that unfortunately sigal fails while processing a video. With 0.7.0 everything just worked fine.

here the stacktrace:

.ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
Traceback (most recent call last):
  File "/usr/local/bin/sigal", line 9, in <module>
    load_entry_point('sigal==0.8.0', 'console_scripts', 'sigal')()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 609, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 589, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 935, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 781, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/sigal/__init__.py", line 133, in build
    gal.build(force=force)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 528, in build
    processor(files)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 581, in process_file
    return process_video(src_path, dst_path, settings)
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 152, in process_video
    fit=settings['thumb_fit'], options=settings['jpg_options'])
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 130, in generate_thumbnail
    check_subprocess(cmd, source, outname)
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 55, in check_subprocess
    os.remove(outname)
OSError: [Errno 2] No such file or directory: '/home/pi/gallery/mygallery/Valencia 2013/thumbnails/CIMG4433.jpg'
@sebiii

This comment has been minimized.

sebiii commented Sep 5, 2014

Apparently the video was ignored in 0.7.0 (the video is in Motion JPEG format). Is there a plan to support this format? or can the video be also ignored in 0.8?

@saimn

This comment has been minimized.

Owner

saimn commented Sep 7, 2014

Hi @sebiii
Before 0.8.0, if the ffmpeg subprocess failed or was interrupted, it was failling silently, and could have created an output file which was not desirable (see #90). So I added a line of code to remove this file, without checking that the file actually exists, which is bad :s. So I will add a check for this.
Concerning the MJPEG support, I don't see why it should work but I haven't tested. Can you run sigal in debug mode (-d) and paste the output to see where it is failing ?

@saimn saimn closed this in fa7f9d9 Sep 7, 2014

@sebiii

This comment has been minimized.

sebiii commented Sep 8, 2014

hi @saimn ,

thank you for the answer! Unfortunately, sigal still fails when trying to open the temporary tumbnail file (see trace below). I have also attached the output of sigal in debug mode.

ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
Traceback (most recent call last):
  File "/usr/local/bin/sigal", line 9, in <module>
    load_entry_point('sigal==0.8.0', 'console_scripts', 'sigal')()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 610, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 590, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 936, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 782, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/sigal/__init__.py", line 133, in build
    gal.build(force=force)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 528, in build
    processor(files)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 581, in process_file
    return process_video(src_path, dst_path, settings)
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 154, in process_video
    fit=settings['thumb_fit'], options=settings['jpg_options'])
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 135, in generate_thumbnail
    image.generate_thumbnail(tmpfile, outname, box, fit, options)
  File "/usr/local/lib/python2.7/dist-packages/sigal/image.py", line 114, in generate_thumbnail
    img = PILImage.open(source)
  File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 2219, in open
    fp = builtins.open(fp, "rb")
IOError: [Errno 2] No such file or directory: u'/home/pi/gallery/mygallery/Valencia 2013/thumbnails/CIMG4433.jpg.tmp.jpg'
INFO - Processing /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI
DEBUG - Video size: 640, 480 -> 480, 360
DEBUG - Processing video: ffmpeg -i /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI -y -crf 10 -b:v 1.6M -qmin 4 -qmax 63 -vf scale=480:trunc(ow/a/2)*2 /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
ERROR - Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI
DEBUG - STDOUT:

DEBUG - STDERR:
 ffmpeg version 0.8.13-6:0.8.13-1+rpi1, Copyright (c) 2000-2014 the Libav developers
  built on Jul 29 2014 02:09:52 with gcc 4.6.3
*** THIS PROGRAM IS DEPRECATED ***
This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.
[avi @ 0x1232660] non-interleaved AVI
Input #0, avi, from '/home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI':
  Metadata:
    creation_time   : 2013-08-29/ 23:13
    encoder         : CASIO EX-S200
  Duration: 00:00:18.61, start: 0.000000, bitrate: 3596 kb/s
    Stream #0.0: Video: mjpeg, yuvj422p, 640x480, 29.97 tbr, 29.97 tbn, 29.97 tbc
    Stream #0.1: Audio: adpcm_ima_wav, 44100 Hz, 1 channels, s16, 176 kb/s
Incompatible pixel format 'yuvj422p' for codec 'libvpx', auto-selecting format 'yuv420p'
[buffer @ 0x1234560] w:640 h:480 pixfmt:yuvj422p
[scale @ 0x1233fe0] w:640 h:480 fmt:yuvj422p -> w:480 h:360 fmt:yuv420p flags:0x4
[libvpx @ 0x1233580] v1.1.0
Output #0, webm, to '/home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm':
  Metadata:
    creation_time   : 2013-08-29/ 23:13
    encoder         : Lavf53.21.1
    Stream #0.0: Video: libvpx, yuv420p, 480x360, q=4-63, 200 kb/s, 1k tbn, 29.97 tbc
    Stream #0.1: Audio: libvorbis, 44100 Hz, 1 channels, s16
Stream mapping:
  Stream #0.0 -> #0.0
  Stream #0.1 -> #0.1
Press ctrl-c to stop encoding
[webm @ 0x1232ea0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 33 >= 33
av_interleaved_write_frame(): Invalid argument

DEBUG - Process failed, removing file /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
DEBUG - Create thumbnail for video: ffmpeg -i /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm -an -r 1 -vframes 1 -y /home/pi/gallery/mygallery/Valencia 2013/thumbnails/CIMG4433.jpg.tmp.jpg
ERROR - Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
DEBUG - STDOUT:

DEBUG - STDERR:
 ffmpeg version 0.8.13-6:0.8.13-1+rpi1, Copyright (c) 2000-2014 the Libav developers
  built on Jul 29 2014 02:09:52 with gcc 4.6.3
*** THIS PROGRAM IS DEPRECATED ***
This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.
/home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm: No such file or directory

DEBUG - Process failed, removing file /home/pi/gallery/mygallery/Valencia 2013/thumbnails/CIMG4433.jpg
Traceback (most recent call last):
  File "/usr/local/bin/sigal", line 9, in <module>
    load_entry_point('sigal==0.8.0', 'console_scripts', 'sigal')()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 610, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 590, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 936, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 782, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/sigal/__init__.py", line 133, in build
    gal.build(force=force)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 528, in build
    processor(files)
  File "/usr/local/lib/python2.7/dist-packages/sigal/gallery.py", line 581, in process_file
    return process_video(src_path, dst_path, settings)
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 152, in process_video
    fit=settings['thumb_fit'], options=settings['jpg_options'])
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 130, in generate_thumbnail
    check_subprocess(cmd, source, outname)
  File "/usr/local/lib/python2.7/dist-packages/sigal/video.py", line 55, in check_subprocess
    os.remove(outname)
OSError: [Errno 2] No such file or directory: '/home/pi/gallery/mygallery/Valencia 2013/thumbnails/CIMG4433.jpg'

t-animal added a commit to t-animal/sigal that referenced this issue Sep 8, 2014

@saimn saimn reopened this Sep 9, 2014

@saimn

This comment has been minimized.

Owner

saimn commented Sep 9, 2014

Are you sure you have updated the code with the master branch from github ? I can't reproduce it, but I have added an extra check in 6997209. Please check if it is better now.

@sebiii

This comment has been minimized.

sebiii commented Sep 10, 2014

Dear @saimn ,

I have tested the code with the extra check and it now skips the videos just fine, but seems to fail in the end at the zip generation (i have zip_media_format = 'resized').

 Collecting files  [####################################]  100%                  
  [#######-----------------------------]  1/5ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4433.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm
  [##############----------------------]  2/5  00:00:08ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4434.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4434.webm
  [#####################---------------]  3/5  00:00:06ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4432.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4432.webm
  [############################--------]  4/5  00:00:03ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4435.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4435.webm
  [####################################]  5/5          ERROR: Failed to process /home/pi/gallery/pictures/Valencia 2013/CIMG4431.AVI
ERROR: Failed to process /home/pi/gallery/mygallery/Valencia 2013/CIMG4431.webm
  [####################################]  5/5
Traceback (most recent call last):
  File "/usr/local/bin/sigal", line 9, in <module>
    load_entry_point('sigal==0.9.0-dev', 'console_scripts', 'sigal')()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 610, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 590, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 936, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 782, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/sigal-0.9.0_dev-py2.7.egg/sigal/__init__.py", line 133, in build
    gal.build(force=force)
  File "/usr/local/lib/python2.7/dist-packages/sigal-0.9.0_dev-py2.7.egg/sigal/gallery.py", line 628, in build
    writer.write(album)
  File "/usr/local/lib/python2.7/dist-packages/sigal-0.9.0_dev-py2.7.egg/sigal/writer.py", line 110, in write
    page = self.template.render(**self.generate_context(album))
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/sigal-0.9.0_dev-py2.7.egg/sigal/themes/colorbox/templates/index.html", line 119, in top-level template code
    {% if album.zip %}
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 397, in getattr
    return getattr(obj, attribute)
  File "/usr/local/lib/python2.7/dist-packages/sigal-0.9.0_dev-py2.7.egg/sigal/gallery.py", line 452, in zip
    archive.write(p.dst_path, os.path.split(p.dst_path)[1])
  File "/usr/lib/python2.7/zipfile.py", line 1049, in write
    st = os.stat(filename)
OSError: [Errno 2] No such file or directory: '/home/pi/gallery/mygallery/Valencia 2013/CIMG4433.webm'
@saimn

This comment has been minimized.

Owner

saimn commented Sep 11, 2014

Ah yeah, if the video have not been generated they can't be used in the zip process ... I will also add a check there. The main issue is that currently the failed images/videos are not removed from the list of files of an album, I must to look at this.

saimn added a commit that referenced this issue Sep 15, 2014

@saimn

This comment has been minimized.

Owner

saimn commented Sep 15, 2014

@sebiii : I have added the check, can you test again to see if it is ok ? Thanks.

@sebiii

This comment has been minimized.

sebiii commented Sep 16, 2014

Dear @saimn ,

the build process now exits without errors, but I have the videos in my gallery as items (which obviously cannot be played).

@saimn

This comment has been minimized.

Owner

saimn commented Sep 16, 2014

Yeah, currently the solution is to remove these files from the source directory, but I will try to find a better solution. It was on my todo list but I have created an issue to track this (#112). Thanks for your help on tracking these failures !

@saimn saimn closed this Sep 16, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment