Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[npo] Update subtitles url #11865

Merged
merged 1 commit into from Jan 28, 2017
Merged

[npo] Update subtitles url #11865

merged 1 commit into from Jan 28, 2017

Conversation

andrewalker
Copy link
Contributor

NPO websites changed the domain they used for subtitles, from e.omroep.nl to tt888.omroep.nl.

I have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

I noticed I couldn't download subtitles anymore with the npo extractor. In the browser I noticed the subtitles were there in the video. So I just inspected my network, found out the new URL, tested on youtube-dl, and it worked.

Example before my patch:

$ youtube-dl -v --all-subs http://www.npo.nl/jinek/25-01-2017/KN_1687315
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--all-subs', u'http://www.npo.nl/jinek/25-01-2017/KN_1687315']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.02
[debug] Python version 2.7.13 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 11.4, avprobe 11.4, ffmpeg 3.2.2, ffprobe 3.2.2
[debug] Proxy map: {}
[npo] KN_1687315: Downloading JSON metadata
[npo] KN_1687315: Downloading token
[npo] KN_1687315: Downloading adaptive JSON
[npo] KN_1687315: Downloading adaptive stream JSON
[npo] KN_1687315: Downloading m3u8 information
[npo] KN_1687315: Downloading h264_bb JSON
[npo] KN_1687315: Downloading h264_bb stream JSON
[npo] KN_1687315: Downloading h264_sb JSON
[npo] KN_1687315: Downloading h264_sb stream JSON
[npo] KN_1687315: Downloading h264_std JSON
[npo] KN_1687315: Downloading h264_std stream JSON
WARNING: Unable to download subtitle for "nl": HTTP Error 410: Gone
[debug] Invoking downloader on u'http://content10c5a.omroep.nl/urishieldv2/l27m14012d8630d8e1b200588cecad000000.a40e9ddd2907e0df3e9a7c4384157669/ceresodi/h264/p/34/10/10/33/std_KN_1687315.m4v?odiredirecturl=%2Fvideo%2Fida%2Fh264_std%2Fbd09e29e238945d34bf619d0a854101d%2F588cecad%2FKN_1687315%2F1%3Ftype%3Djsonp%26callback%3D%3F%26type%3Djson'
[download] Resuming download at byte 402222809
[download] Destination: Jinek - Menno Bentveld, Charles Groenhuijsen, Sarah Thurlings-Heijse, Jules Deelder, Petra Stienen, Connie Palmen en Marcia Luyten-KN_1687315.m4v
[download] 100% of 428.50MiB in 00:05

After my patch:

$ python3.6 -m youtube_dl --verbose --all-subs 'http://www.npo.nl/jinek/25-01-2017/KN_1687315' 
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--all-subs', 'http://www.npo.nl/jinek/25-01-2017/KN_1687315']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.28
[debug] Git HEAD: 47242d1
[debug] Python version 3.6.0 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 11.4, avprobe 11.4, ffmpeg 3.2.2, ffprobe 3.2.2
[debug] Proxy map: {}
[npo] KN_1687315: Downloading JSON metadata
[npo] KN_1687315: Downloading token
[npo] KN_1687315: Downloading adaptive JSON
[npo] KN_1687315: Downloading adaptive stream JSON
[npo] KN_1687315: Downloading m3u8 information
[npo] KN_1687315: Downloading h264_bb JSON
[npo] KN_1687315: Downloading h264_bb stream JSON
[npo] KN_1687315: Downloading h264_sb JSON
[npo] KN_1687315: Downloading h264_sb stream JSON
[npo] KN_1687315: Downloading h264_std JSON
[npo] KN_1687315: Downloading h264_std stream JSON
[info] Writing video subtitles to: Jinek - Menno Bentveld, Charles Groenhuijsen, Sarah Thurlings-Heijse, Jules Deelder, Petra Stienen, Connie Palmen en Marcia Luyten-KN_1687315.nl.vtt
[debug] Invoking downloader on 'http://content10c5c.omroep.nl/urishieldv2/l27m04f15e8e1f0c22b700588ced14000000.7cac8c4616c3ff91a1f3881319a96b01/ceresodi/h264/p/34/10/10/33/std_KN_1687315.m4v?odiredirecturl=%2Fvideo%2Fida%2Fh264_std%2Ffbee0c0865fcb2d2920f31a0bda9d371%2F588ced14%2FKN_1687315%2F1%3Ftype%3Djsonp%26callback%3D%3F%26type%3Djson'
[download] Destination: Jinek - Menno Bentveld, Charles Groenhuijsen, Sarah Thurlings-Heijse, Jules Deelder, Petra Stienen, Connie Palmen en Marcia Luyten-KN_1687315.m4v
[download] 100% of 428.50MiB in 00:43

For a simpler example, with curl, with the old URL:

$ curl -v 'http://e.omroep.nl/tt888/KN_1687315' >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 145.58.33.91...
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to e.omroep.nl (145.58.33.91) port 80 (#0)
> GET /tt888/KN_1687315 HTTP/1.1
> Host: e.omroep.nl
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 410 Gone
< Date: Sat, 28 Jan 2017 19:19:20 GMT
< Server: Apache
< Content-Length: 311
< Content-Type: text/html; charset=iso-8859-1
<
{ [311 bytes data]
100   311  100   311    0     0   3802      0 --:--:-- --:--:-- --:--:--  3792
* Connection #0 to host e.omroep.nl left intact

With the URL I just introduced:

$ curl -v 'http://tt888.omroep.nl/tt888/KN_1687315' >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 145.58.33.96...
* Connected to tt888.omroep.nl (145.58.33.96) port 80 (#0)
> GET /tt888/KN_1687315 HTTP/1.1
> Host: tt888.omroep.nl
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 28 Jan 2017 19:20:15 GMT
< Server: Apache
< Cache-Control: private, max-age=300, must-revalidate
< Expires: Sat, 28 Jan 2017 19:25:15 GMT
< Access-Control-Allow-Origin: *
< Last-Modified: Sat, 28 Jan 2017 19:20:15 GMT
< X-Node: tab-as56
< Transfer-Encoding: chunked
< Content-Type: text/vtt
<
{ [2566 bytes data]
100 59562    0 59562    0     0   665k      0 --:--:-- --:--:-- --:--:--  668k
* Connection #0 to host tt888.omroep.nl left intact

NPO websites changed the domain they used for subtitles, from
e.omroep.nl to tt888.omroep.nl.
@dstftw dstftw merged commit 56fc078 into ytdl-org:master Jan 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants