Add support for www.kijk.nl

[pvdl]

Via this site you can download tv series from these Dutch commercial TV stations.
NET5 http://www.net5.nl/
SBS6 http://www.sbs6.nl/
Veronica http://www.veronicatv.nl/

Maybe you can decide to make three different extractors.

Basically SBS (www.sbs.nl) is a media company which broadcasts these three mentioned TV-stations

[pvdl]

I'll try to do some investigations to find the JSON file or other relevant info.

[pvdl]

Bas Rieter did awsome work for XBMC to get Dutch streams working.
Python code can be examined here: http://www.rieter.net/content/
File: /net.rieter.xot/deploy/net.rieter.xot.channel.sbsnl/kijknl/

[pvdl]

URL: http://www.kijk.nl/video/da6WVCLYiFKo
ID = da6WVCLYiFKo
You can embed the url in an iframe
http://embed.kijk.nl/video=da6WVCLYiFKo

[MatrixView]

www.kijk.nl uses the brightcove player, which youtube-dl already has a plugin for.
i.e. with the following link the video downloads just fine: http://www.kijk.nl/video/jbgOrDOHCVuW

...however, some links don't work, like the one mentioned bij PVDL above:

youtube-dl http://www.kijk.nl/video/da6WVCLYiFKo

[generic] da6WVCLYiFKo: Requesting header
WARNING: Falling back on generic information extractor.
[generic] da6WVCLYiFKo: Downloading webpage
[generic] da6WVCLYiFKo: Extracting information
[generic] Brightcove video detected.
[download] Downloading playlist: Kijk nu online: Aflevering 1, Shopping & Lifestyle (17-08-2014)
[generic] playlist Kijk nu online: Aflevering 1, Shopping & Lifestyle (17-08-2014): Collected 1 video ids (downloading 1 of them)
[download] Downloading video #1 of 1
[Brightcove] 3731602412001: Downloading webpage
[Brightcove] 3731602412001: Extracting information
Traceback (most recent call last):
File "main.py", line 18, in
File "youtube_dl__init__.pyo", line 911, in main
File "youtube_dl__init__.pyo", line 901, in _real_main
File "youtube_dl\YoutubeDL.pyo", line 1063, in download
File "youtube_dl\YoutubeDL.pyo", line 532, in extract_info
File "youtube_dl\YoutubeDL.pyo", line 656, in process_ie_result
File "youtube_dl\YoutubeDL.pyo", line 581, in process_ie_result
File "youtube_dl\YoutubeDL.pyo", line 521, in extract_info
File "youtube_dl\extractor\common.pyo", line 178, in extract
File "youtube_dl\extractor\brightcove.pyo", line 192, in _real_extract
File "youtube_dl\extractor\brightcove.pyo", line 224, in _get_video_info
File "youtube_dl\extractor\brightcove.pyo", line 257, in _extract_video_info

AttributeError: 'NoneType' object has no attribute 'replace'

This xbmc addon can handle it: http://xot.hamans.com/net.rieter.xot-3.5.0.zip
It uses the net.rieter.xot\deploy\net.rieter.xot.channel.sbsnl\kijknl\chn_kijknl.py script to handle the site and it uses its own brightcove plugin to get the stream/file which functions you can find in:
net.rieter.xot\resources\libs\streams\brightcove.py

Unfortunately I'm not a programmer, so I understand very little of it, but maybe it's useful to someone to get the other vids working too in Youtube-DL.

Thanx!

[Reino17]

Sorry to bump this issue, but I too would appreciate it if you could have a look.
At the moment afaik only GemistDownloader can download videos (and generate video-urls) from kijk.nl, so it would be really great if you could add support to youtube-dl.
I'm not sure, but it could be that kijk.nl broadcasts only one format. At least when I watch the video in my browser it never changes to a lower quality version.

GemistDownloader translates http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1 for example to http://l3c.cdn.sbsnet.nl/ra/01/3023772f9f1281ab45cce7dee470ca39b8ab23/hls/r8/sbs6_leermijvrouwenkennen_S2014E01_141030111501302377-hls-1885.m3u8, which is a H.264,868x488+AAC,44.1kHz videostream.

youtube-dl at the moment:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vF', 'http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2014.10.30
[debug] Python version 2.7.8 - Windows-XP-5.1.2600-SP3
[debug] exe versions: ffmpeg N-65810-
[debug] Proxy map: {}
[generic] aflevering-1: Requesting header
WARNING: Falling back on generic information extractor.
[generic] aflevering-1: Downloading webpage
[generic] aflevering-1: Extracting information
[generic] Brightcove video detected.
[download] Downloading playlist: KIJK - Leer mij vrouwen kennen
[generic] playlist KIJK - Leer mij vrouwen kennen: Collected 1 video ids (downloading 1 of them)
[download] Downloading video #1 of 1
ERROR: Cannot find playerKey= variable. Did you forget quotes in a shell invocation?
Traceback (most recent call last):
  File "youtube_dl\YoutubeDL.pyo", line 533, in extract_info
  File "youtube_dl\extractor\common.pyo", line 198, in extract
  File "youtube_dl\extractor\brightcove.pyo", line 208, in _real_extract
ExtractorError: Cannot find playerKey= variable. Did you forget quotes in a shell invocation?

As you can see youtube-dl already picks up the program-url rather well, because should you enter the short program-url, youtube-dl redirects it anyway:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vF', 'http://www.kijk.nl/video/jqMiXKAYan2S']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2014.10.30
[debug] Python version 2.7.8 - Windows-XP-5.1.2600-SP3
[debug] exe versions: ffmpeg N-65810-
[debug] Proxy map: {}
[generic] jqMiXKAYan2S: Requesting header
[redirect] Following redirect to http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/leer-mij-vrouwen-kennen-aflevering-1
[generic] leer-mij-vrouwen-kennen-aflevering-1: Requesting header
WARNING: Falling back on generic information extractor.
[generic] leer-mij-vrouwen-kennen-aflevering-1: Downloading webpage
[generic] leer-mij-vrouwen-kennen-aflevering-1: Extracting information
[generic] Brightcove video detected.
[download] Downloading playlist: KIJK - Leer mij vrouwen kennen
[generic] playlist KIJK - Leer mij vrouwen kennen: Collected 1 video ids (downloading 1 of them)
[download] Downloading video #1 of 1
ERROR: Cannot find playerKey= variable. Did you forget quotes in a shell invocation?
Traceback (most recent call last):
  File "youtube_dl\YoutubeDL.pyo", line 533, in extract_info
  File "youtube_dl\extractor\common.pyo", line 198, in extract
  File "youtube_dl\extractor\brightcove.pyo", line 208, in _real_extract
ExtractorError: Cannot find playerKey= variable. Did you forget quotes in a shell invocation?

[phihag]

Thank you for the report. I believe this issue to be fixed in youtube-dl 2014.11.04. Refer to our FAQ to find out how to update. Please comment or open a new issue if the issue persists with the new version.

[Reino17]

youtube-dl.exe -F http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1
[generic] aflevering-1: Requesting header
WARNING: Falling back on generic information extractor.
[generic] aflevering-1: Downloading webpage
[generic] aflevering-1: Extracting information
[generic] Brightcove video detected.
[download] Downloading playlist: KIJK - Leer mij vrouwen kennen
[generic] playlist KIJK - Leer mij vrouwen kennen: Collected 1 video ids (downloading 1 of them)
[download] Downloading video #1 of 1
[Brightcove] 3866516442001: Downloading webpage
[Brightcove] 3866516442001: Extracting information
[info] Available formats for 3866516442001:
format code extension resolution  note
0           flv       480x360

youtube-dl.exe -g http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1
WARNING: Falling back on generic information extractor.
http://l3c.cdn.sbsnet.nl/ra/01/3023772f9f1281ab45cce7dee470ca39b8ab23/hls/sbs6_leermijvrouwenkennen_
S2014E01_141030111909302377-hls-main.m3u8?&v=3.3.0&fp=13&r=FEEFJ&g=RTSJIMBMPFPB

sbs6_leermijvrouwenkennen_S2014E01_141030111909302377-hls-main.m3u8:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=610664,RESOLUTION=320x180,CODECS="mp4a.40.2,avc1.4d401f"
r1/sbs6_leermijvrouwenkennen_S2014E01_141030111229302377-hls-350.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=928708,RESOLUTION=512x288,CODECS="mp4a.40.2,avc1.4d401f"
r2/sbs6_leermijvrouwenkennen_S2014E01_141030111101302377-hls-650.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1288969,RESOLUTION=640x360,CODECS="mp4a.40.2,avc1.4d401f"
r3/sbs6_leermijvrouwenkennen_S2014E01_141030111012302377-hls-900.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1643382,RESOLUTION=768x432,CODECS="mp4a.40.2,avc1.4d401f"
r4/sbs6_leermijvrouwenkennen_S2014E01_141030111237302377-hls-1200.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2179034,RESOLUTION=868x488,CODECS="mp4a.40.2,avc1.4d401f"
r8/sbs6_leermijvrouwenkennen_S2014E01_141030111501302377-hls-1885.m3u8

Thanks for the initial support. As you can see above, it's not a 480x360 flv-file, but the 'manifest-playlist' which youtube-dl now detects.
Could you please add support for the actual videostreams (as illustrated below), including the full url for --get-url?

youtube-dl.exe -F http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1
...
[info] Available formats for 3866516442001:
format code extension resolution  note
m3u8-meta   m3u8      multiple    Quality selection URL  (worst)
m3u8-350    m3u8      320x180      350k
m3u8-650    m3u8      512x288      650k
m3u8-900    m3u8      640x360      900k
m3u8-1200   m3u8      768x432      1200k
m3u8-1885   m3u8      868x488      1885k  (best)

youtube-dl.exe -g -f m3u8-1200 http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1
WARNING: Falling back on generic information extractor.
http://l3c.cdn.sbsnet.nl/ra/01/3023772f9f1281ab45cce7dee470ca39b8ab23/hls/r4/sbs6_leermijvrouwenkennen_S2014E01_141030111237302377-hls-1200.m3u8

I've had a look at several 'meta-playlists' from other videos and all streams appeared to be 350k,650k,900k,1200k and 1885k. So I believe it's save to name them "m3u8-350", "m3u8-650", etc.

[naglis]

Yes, it's on my TODO list.

It would also be helpful if someone could provide an example of NET5 and Veronica TV videos on kijk.nl as mentioned in the first post by @pvdl, to see if they work. Because currently only SBS6 is known to work.

[Reino17]

kijk.nl is a platform for videos from all 3 mentioned tv stations, so naturally they all use the same brightcove player. I just tested a couple of tvshows from Net5 and Veronica and although I first had to consult the 'manifest-playlist' for the 868x488 videostream-url, they all work fine.

[naglis]

Ok, thanks a bunch for checking!

[naglis]

Different m3u8 formats should be extracted in the next version of youtube-dl.

[Reino17]

Confirmed:

youtube-dl.exe -F http://www.kijk.nl/sbs6/leermijvrouwenkennen/videos/jqMiXKAYan2S/aflevering-1
...
[info] Available formats for 3866516442001:
format code extension resolution  note
m3u8-meta   mp4       multiple    Quality selection URL  (worst)
m3u8-610    mp4       320x180      610k , mp4a, avc1
m3u8-928    mp4       512x288      928k , mp4a, avc1
m3u8-1288   mp4       640x360     1288k , mp4a, avc1
m3u8-1643   mp4       768x432     1643k , mp4a, avc1
m3u8-2179   mp4       868x488     2179k , mp4a, avc1  (best)

Thanks a lot! Though I would've prefered a static format code, without the bitrate in it. It's unpractical for batch-scripts, because the bitrates fluctuates with every other video.

[naglis]

@corone17, the m3u8 format extraction is done by a common helper function, not related to kijk.nl. You can open a new issue regarding that.