[Ketnet.be] Bug when downloading a video

[MichaelDeBoey]

• I've verified and I assure that I'm running youtube-dl 2016.09.24
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)

---

$ youtube-dl -v https://www.ketnet.be/kijken/nachtwacht/de-bermadoe
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.ketnet.be/kijken/nachtwacht/de-bermadoe']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.09.24
[debug] Python version 2.7.12 - Darwin-16.0.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1
[debug] Proxy map: {}
[Ketnet] de-bermadoe: Downloading webpage
ERROR: An extractor error has occurred. (caused by KeyError(u'source',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 355, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/ketnet.py", line 40, in _real_extract
    config['source']['hls'], video_id, 'mp4',
KeyError: u'source'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 361, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
ExtractorError: An extractor error has occurred. (caused by KeyError(u'source',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

---

When downloading https://www.ketnet.be/kijken/nachtwacht/de-trol via the Ketnet extractor (created for #10343) there's no problem, but when I want to download https://www.ketnet.be/kijken/nachtwacht/de-bermadoe, it gives the above error.

[MichaelDeBoey]

@dstftw This one can be closed I guess?

[dstftw]

I can't test due to geo restriction.

[MichaelDeBoey]

Still can't download the video 😞

$ youtube-dl -v https://www.ketnet.be/kijken/nachtwacht/de-bermadoe
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.ketnet.be/kijken/nachtwacht/de-bermadoe']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.10.02
[debug] Python version 2.7.12 - Darwin-16.0.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.4, ffprobe 3.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[Ketnet] de-bermadoe: Downloading webpage
[Ketnet] de-bermadoe: Downloading f4m manifest
WARNING: Unable to download f4m manifest: HTTP Error 404: Not Found
[Ketnet] de-bermadoe: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[debug] Invoking downloader on u'rtmp://vod.stream.vrt.be/mediazone_ketnet_geo/_definst_/mp4:2016/09/mz-ast-138b35e4-364c-4f00-a681-9e9366a9fa67/video_1296.mp4'
[download] Destination: Nachtwacht - De Bermadoe-de-bermadoe.mp4
[debug] rtmpdump command line: rtmpdump --verbose -r rtmp://vod.stream.vrt.be/mediazone_ketnet_geo/_definst_/mp4:2016/09/mz-ast-138b35e4-364c-4f00-a681-9e9366a9fa67/video_1296.mp4 -o 'Nachtwacht - De Bermadoe-de-bermadoe.mp4.part' --resume --skip 1
[rtmpdump] RTMPDump v2.4
[rtmpdump] (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
[rtmpdump] DEBUG: Parsing...
[rtmpdump] DEBUG: Parsed protocol: 0
[rtmpdump] DEBUG: Parsed host    : vod.stream.vrt.be
[rtmpdump] DEBUG: Parsed app     : mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: Number of skipped key frames for resume: 1
[rtmpdump] DEBUG: Protocol : RTMP
[rtmpdump] DEBUG: Hostname : vod.stream.vrt.be
[rtmpdump] DEBUG: Port     : 1935
[rtmpdump] DEBUG: Playpath : mp4:09/mz-ast-138b35e4-364c-4f00-a681-9e9366a9fa67/video_1296
[rtmpdump] DEBUG: tcUrl    : rtmp://vod.stream.vrt.be:1935/mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: app      : mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: live     : no
[rtmpdump] DEBUG: timeout  : 30 sec
[rtmpdump] DEBUG: Failed to get last keyframe.
[rtmpdump] DEBUG: Closing connection.
[rtmpdump] 0 bytes
[rtmpdump] RTMPDump v2.4
[rtmpdump] (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
[rtmpdump] DEBUG: Parsing...
[rtmpdump] DEBUG: Parsed protocol: 0
[rtmpdump] DEBUG: Parsed host    : vod.stream.vrt.be
[rtmpdump] DEBUG: Parsed app     : mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: Number of skipped key frames for resume: 1
[rtmpdump] DEBUG: Number of skipped key frames for resume: 1
[rtmpdump] DEBUG: Protocol : RTMP
[rtmpdump] DEBUG: Hostname : vod.stream.vrt.be
[rtmpdump] DEBUG: Port     : 1935
[rtmpdump] DEBUG: Playpath : mp4:09/mz-ast-138b35e4-364c-4f00-a681-9e9366a9fa67/video_1296
[rtmpdump] DEBUG: tcUrl    : rtmp://vod.stream.vrt.be:1935/mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: app      : mediazone_ketnet_geo/_definst_/mp4:2016
[rtmpdump] DEBUG: live     : no
[rtmpdump] DEBUG: timeout  : 30 sec
[rtmpdump] DEBUG: Failed to get last keyframe.
[rtmpdump] DEBUG: Closing connection.


ERROR: rtmpdump exited with code 1
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 445, in main
    _real_main(argv)
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 435, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1791, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 705, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 751, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1435, in process_video_result
    self.process_info(new_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/rtmp.py", line 202, in real_download
    self.report_error('rtmpdump exited with code %d' % retval)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 162, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 568, in report_error
    self.trouble(error_message, tb)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 530, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

[MichaelDeBoey]

@dstftw Don't know if my previous post was any helpfull?

[kenavera]

For me the normal video's work.
So, I guess it's due to the geo restriction.
Can you play the video itself, so are you outside the geo area?

If you can play it, could you go to the network tab in chrome (right click inspect, in ff you need httpfox).
In the network tab you search for chunklist and then post one of the links here.
http://vod.stream.vrt.be/mediazone_ketnet/_definst_/smil:2016/10/mz-ast-b2e22257-2c78-438c-ac5b-fbcf22bd27d3-1/video.smil/chunklist_w967508416_b64000_ao.m3u8.
The only thing I can imagine is that they do some 'magic', because it's geo restricted (and not just adding the _geo), because the m3u8 file can not be found. With a direct url to a chunklist I could try and figure out if they added something.

[MichaelDeBoey]

@kenavera I'm from Belgium, so I'm able to watch the videos.

This video has the following chuncklist: http://vod.stream.vrt.be/mediazone_ketnet_geo/_definst_/smil:2016/10/mz-ast-3d6c508a-d243-4941-84b8-d7d18c7b35df-2/video.smil/chunklist_w872545264_b1196000_slnl.m3u8

I didn't update Youtube-dl, but the download for that link is working.
The other links (De Trol and De Bermadoe) aren't available anymore 😞

[peturdainn]

Came across this issue by accident while googling URLs for the above shows that I was dumping manually with rtmpdump.

To capture those videos I needed the -v argument (and patience)

So the above video works if you do it like this:
rtmpdump -v -r rtmp://vod.stream.vrt.be/mediazone_ketnet_geo/_definst_/mp4:2016/10/mz-ast-3d6c508a-d243-4941-84b8-d7d18c7b35df/video_1296.mp4 -y MP4:2016/10/mz-ast-3d6c508a-d243-4941-84b8-d7d18c7b35df/video_1296.mp4 -o de_domovoj.mp4