avconv broken for vesti.ru -- av_interleaved_write_frame(): Invalid argument

[zwerfvogel]

The official list of supported sites includes vesti.ru, but downloading from it causes a traceback related to avconv.

$ python -V
Python 2.7.3
$ uname -a
Linux myhost 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u1 x86_64 GNU/Linux
user@myhost:$ youtube-dl -U
Updating to version 2014.12.04.2 ...
Updated youtube-dl. Restart youtube-dl to use the new version.
user@myhost:$ youtube-dl --verbose http://www.vesti.ru/videos/show/vid/628838/cid/5/#
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.vesti.ru/videos/show/vid/628838/cid/5/#']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.12.04.2
[debug] Python version 2.7.3 - Linux-3.2.0-4-amd64-x86_64-with-debian-7.7
[debug] exe versions: avconv 0.8.16-6, avprobe 0.8.16-6, ffmpeg 0.8.16-6, ffprobe 0.8.16-6, rtmpdump 2.4
[debug] Proxy map: {}
[Vesti] videos/show/vid/628838/cid/5/#: Downloading page
[RUTV] 1149495: Downloading JSON
[debug] Invoking downloader on u'http://cdnmg.v.rtr-vesti.ru/_cdn_auth/secure/v/vh/vod_hls/definst/smil:vh/smil/001/107/714_d20141204170254.smil/playlist.m3u8?auth=mh&vid=1107714'
[download] Destination: Политолог - мы получили подтверждение, что Крым российский раз и навсегда-1149495.mp4
avconv version 0.8.16-6:0.8.16-1, Copyright (c) 2000-2014 the Libav developers
built on Sep 16 2014 23:10:48 with gcc 4.7.2
[applehttp @ 0xabd300] max_analyze_duration reached
[applehttp @ 0xabd300] Estimating duration from bitrate, this may be inaccurate
Input #0, applehttp, from 'http://cdnmg.v.rtr-vesti.ru/_cdn_auth/secure/v/vh/vod_hls/definst/smil:vh/smil/001/107/714_d20141204170254.smil/playlist.m3u8?auth=mh&vid=1107714':
Duration: 00:11:54.00, start: 22.000000, bitrate: N/A
Stream #0.0: Data: [21][0][0][0] / 0x0015
Metadata:
variant_bitrate : 896000
Stream #0.1: Video: h264 (High), yuv420p, 720x404 [PAR 1:1 DAR 180:101], 25 tbr, 90k tbn, 50 tbc
Metadata:
variant_bitrate : 896000
Stream #0.2: Audio: aac, 48000 Hz, stereo, s16
Metadata:
variant_bitrate : 896000
Stream #0.3: Data: [21][0][0][0] / 0x0015
Metadata:
variant_bitrate : 895000
Stream #0.4: Video: h264 (Main), yuv420p, 640x360 [PAR 1:1 DAR 16:9], 25 tbr, 90k tbn, 50 tbc
Metadata:
variant_bitrate : 895000
Stream #0.5: Audio: aac, 48000 Hz, stereo, s16
Metadata:
variant_bitrate : 895000
Stream #0.6: Data: [21][0][0][0] / 0x0015
Metadata:
variant_bitrate : 496000
Stream #0.7: Video: h264 (Constrained Baseline), yuv420p, 432x242 [PAR 1:1 DAR 216:121], 25 tbr, 90k tbn, 50 tbc
Metadata:
variant_bitrate : 496000
Stream #0.8: Audio: aac, 48000 Hz, stereo, s16
Metadata:
variant_bitrate : 496000
Output #0, mp4, to 'Политолог - мы получили подтверждение, что Крым российский раз и навсегда-1149495.mp4.part':
Metadata:
encoder : Lavf53.21.1
Stream #0.0: Video: libx264, yuv420p, 720x404 [PAR 1:1 DAR 180:101], q=2-31, 90k tbn, 90k tbc
Metadata:
variant_bitrate : 896000
Stream #0.1: Audio: aac, 48000 Hz, stereo
Metadata:
variant_bitrate : 896000
Stream mapping:
Stream #0:1 -> #0:0 (copy)
Stream #0:2 -> #0:1 (copy)
Press ctrl-c to stop encoding
[mp4 @ 0xcb6840] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 512 >= 512
av_interleaved_write_frame(): Invalid argument

ERROR: avconv exited with code 1
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/bin/youtube-dl/main.py", line 19, in
youtube_dl.main()
File "/usr/bin/youtube-dl/youtube_dl/init.py", line 355, in main
_real_main(argv)
File "/usr/bin/youtube-dl/youtube_dl/init.py", line 345, in _real_main
retcode = ydl.download(all_urls)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1121, in download
res = self.extract_info(url)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 564, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 617, in process_ie_result
extra_info=extra_info)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 564, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 610, in process_ie_result
return self.process_video_result(ie_result, download=download)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 897, in process_video_result
self.process_info(new_info)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1091, in process_info
success = dl(filename, info_dict)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1067, in dl
return fd.download(name, info)
File "/usr/bin/youtube-dl/youtube_dl/downloader/common.py", line 294, in download
return self.real_download(filename, info_dict)
File "/usr/bin/youtube-dl/youtube_dl/downloader/hls.py", line 49, in real_download
self.report_error('%s exited with code %d' % (program, retval))
File "/usr/bin/youtube-dl/youtube_dl/downloader/common.py", line 147, in report_error
self.ydl.report_error(_args, *_kargs)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 439, in report_error
self.trouble(error_message, tb)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 401, in trouble
tb_data = traceback.format_list(traceback.extract_stack())

[dstftw]

This is most likely due to avconv from 0.8.x branch. Try one from 10.x.

[zwerfvogel]

Yes, I tried avconv from 10.x and it worked. I suggest adding a version check for this. I have written a patch for that -- http://codepad.org/hyiQjULw or you can find it pasted below. While I was doing it, I fixed a minor bug relating to finding the correct version tuple -- version_tuple could not deal with versions like '10.1-6' (as was reported in my case).

diff --git a/youtube_dl/downloader/hls.py b/youtube_dl/downloader/hls.py
index 954beff..e16debc 100644
--- a/youtube_dl/downloader/hls.py
+++ b/youtube_dl/downloader/hls.py
@@ -4,6 +4,7 @@ import os
 import re
 import subprocess

+from ..postprocessor.ffmpeg import FFmpegPostProcessor
 from .common import FileDownloader
 from ..utils import (
     compat_urlparse,
@@ -30,6 +31,10 @@ class HlsFD(FileDownloader):
         else:
             self.report_error('m3u8 download detected but ffmpeg or avconv could not be found. Please install one.')
             return False
+
+        ffpp = FFmpegPostProcessor(downloader=self)
+        ffpp.check_version()
+
         cmd = [program] + args

         retval = subprocess.call(cmd)
diff --git a/youtube_dl/postprocessor/ffmpeg.py b/youtube_dl/postprocessor/ffmpeg.py
index 9303b83..029da58 100644
--- a/youtube_dl/postprocessor/ffmpeg.py
+++ b/youtube_dl/postprocessor/ffmpeg.py
@@ -38,6 +38,9 @@ class FFmpegPostProcessor(PostProcessor):
             raise FFmpegPostProcessorError('ffmpeg or avconv not found. Please install one.')

         REQUIRED_VERSION = '1.0'
+        if self._uses_avconv():
+            REQUIRED_VERSION = '10.0'
+
         if is_outdated_version(
                 self._versions[self._executable], REQUIRED_VERSION):
             warning = 'Your copy of %s is outdated, update %s to version %s or newer if you encounter any errors.' % (
diff --git a/youtube_dl/utils.py b/youtube_dl/utils.py
index 5efb4c5..c972c02 100644
--- a/youtube_dl/utils.py
+++ b/youtube_dl/utils.py
@@ -1502,7 +1502,15 @@ def limit_length(s, length):


 def version_tuple(v):
-    return [int(e) for e in v.split('.')]
+    o = []
+    for e in v.split('.'):
+        if '-' in e:
+            # fix for versions like 10.1-6
+            ms = [int(m) for m in e.split('-')]
+            o.append(ms)
+        else:
+            o.append(int(e))
+    return o


 def is_outdated_version(version, limit, assume_new=True):

[zwerfvogel]

I just noticed that o.append(ms) in version_tuple in that patch should be o.extend(ms) otherwise it adds the list as a sublist.

[phihag]

Thanks @zwerfvogel, I've added equivalent code to mainline youtube-dl.

[zwerfvogel]

Your test for avconv style should be '10.1-6' not '10-6'.

[phihag]

It's just a test, so it does not really matter that much, but I've amended it in 47d7c64.

[zwerfvogel]

Also required_version = '10-0' is wrong -- should be '10.0' -- I think it only uses '-' for minor versions.

[phihag]

That's fine too; 10.0. and 10-0 are equivalent.