[TenPlay] ERROR: Unable to extract video_id

[porjo]

I'm getting this error with current version (2015.09.22)

$ ./youtube-dl --verbose http://tenplay.com.au/channel-ten/the-bolt-report/2015/9/20
[debug] System config: [u'--prefer-free-formats']
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://tenplay.com.au/channel-ten/the-bolt-report/2015/9/20']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.09.22
[debug] Python version 2.7.10 - Linux-4.1.6-201.fc22.x86_64-x86_64-with-fedora-22-Twenty_Two
[debug] exe versions: ffmpeg 2.6.4, ffprobe 2.6.4, rtmpdump 2.4
[debug] Proxy map: {}
[TenPlay] http://tenplay.com.au/channel-ten/the-bolt-report/2015/9/20: Downloading webpage
ERROR: Unable to extract video_id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 660, in extract_info
    ie_result = ie.extract(url)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 287, in extract
    return self._real_extract(url)
  File "./youtube-dl/youtube_dl/extractor/tenplay.py", line 38, in _real_extract
    r'videoID: "(\d+?)"', webpage, 'video_id')
  File "./youtube-dl/youtube_dl/extractor/common.py", line 590, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 581, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract video_id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[ChanderG]

A check of other videos on the site shows that: almost all videos in http://tenplay.com.au/browse-by-genre?genre=News either have the videoID value empty or don't even have it. However the videos on http://tenplay.com.au/news/national seem to have videoID and it's value.

[Mike-Skinner]

I believe something has changed in the markup on the TenPlay site over the past week.

If I use youtube-dl to attempt to download episodes I downloaded (with youtube-dl) last week, I get the same videoID error.

I also rolled back to a previous version with "sudo curl https://yt-dl.org/downloads/2015.08.16.1/youtube-dl -o /usr/local/bin/youtube-dl && sudo chmod a+rx /usr/local/bin/youtube-dl" - same result.

HTH :-)

[remitamine]

it looks like they change to use the brightcove in page embed.
i can't test it because it is georestricted.
i get this message from the api request: Access to this resource is forbidden by access policy.

[porjo]

I've just tried this again and it is still broken for youtube-dl. However it appears that the web page is now downloading the entire MP4 file directly via Akamai CDN. I can right-click the video and 'save as' in from my browser, so a fix for Youtube-dl should be simple I hope. I checked a few other videos on the Tenplay site and they are working the same way.

[ScriptSmith]

Yes it's embedded:

<video id="vjs-e81cb479c487488f897e5ccdf7eec2a9"
data-video-id="4821193673001"
data-account="2199827728001"
data-embed="default"
poster="http://iprx.ten.com.au:80/ImageHandler.ashx?f=jpg&amp;u=https%3a%2f%2fnetworkten-a.akamaihd.net%2fnew%2f2199827728001%2f201603%2f793%2f2199827728001_4821272806001_4821193673001-vs.jpg%3fpubId%3d2199827728001"
data-player="4kPfcMUoe"

autoplay

class="video-js video-player video-player-5"
controls width="auto" height="auto">
</video>

What can I do to adjust tenplay.py?

[remitamine]

can any one check if this works?
youtube-dl -F http://players.brightcove.net/2199827728001/4kPfcMUoe_default/index.html?videoId=4821193673001

[ScriptSmith]

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-F', 'http://players.brightcove.net/2199827728001/4kPfcMUoe_default/index.html?videoId=4822678992001', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.03.26
[debug] Python version 3.5.1 - Linux-4.4.5-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 3.0, ffprobe 3.0, rtmpdump 2.4
[debug] Proxy map: {}
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 668, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 320, in extract
    return self._real_extract(url)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/brightcove.py", line 485, in _real_extract
    formats.extend(self._extract_mpd_formats(src, video_id, 'dash', fatal=False))
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 1395, in _extract_mpd_formats
    compat_etree_fromstring(mpd.encode('utf-8')), mpd_id, mpd_base_url, formats_dict=formats_dict)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 1539, in _parse_mpd_formats
    self._sort_formats(formats)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 839, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[remitamine]

try to use this instead and post the output:
youtube-dl --dump-pages http://players.brightcove.net/2199827728001/4kPfcMUoe_default/index.html?videoId=4821193673001

[remitamine]

i can't test it directly because the site is geo restricted and coudn't find a working AU proxy.

youtube-dl -F http://players.brightcove.net/2199827728001/4kPfcMUoe_default/index.html?videoId=4821193673001
[brightcove:new] 4821193673001: Downloading webpage
[brightcove:new] 4821193673001: Downloading JSON metadata
ERROR: Access to this resource is forbidden by access policy.

[ScriptSmith]

No worries mate ;)
Here's the result: https://u.pomf.is/odgnai

[ScriptSmith]

Fixed the link

[remitamine]

try to update to 2016.03.27 and try to download with the previous url(http://players.brightcove.net/2199827728001/4kPfcMUoe_default/index.html?videoId=4821193673001) if it works then removing the TenPlayIE will fix the problem(the generic extractor can extract brightcove embeds).

[ScriptSmith]

Output for 2016.3.27:

[brightcove:new] 4821193673001: Downloading webpage
[brightcove:new] 4821193673001: Downloading JSON metadata
[brightcove:new] 4821193673001: Downloading m3u8 information
[brightcove:new] 4821193673001: Downloading m3u8 information
[brightcove:new] 4821193673001: Downloading MPD manifest
[info] Available formats for 4821193673001:
format code      extension  resolution note
hls-meta-0       mp4        multiple   Quality selection URL 
hls-meta-1       mp4        multiple   Quality selection URL 
hls-62-0         mp4        unknown      62k 
hls-62-1         mp4        unknown      62k 
hls-159-0        mp4        320x180     159k 
hls-159-1        mp4        320x180     159k 
rtmp-398k-180p   mp4        320x180     398k , MP4 container, H264, 119.16MiB
hls-414-0        mp4        320x180     414k 
hls-414-1        mp4        320x180     414k 
hls-696-0        mp4        640x360     696k 
hls-696-1        mp4        640x360     696k 
rtmp-697k-360p   mp4        640x360     697k , MP4 container, H264, 208.15MiB
hls-1194-0       mp4        1024x576   1194k 
hls-1194-1       mp4        1024x576   1194k 
rtmp-1194k-576p  mp4        1024x576   1194k , MP4 container, H264, 356.36MiB
http-398k-180p   mp4        320x180     398k , MP4 container, H264, 119.16MiB
http-697k-360p   mp4        640x360     697k , MP4 container, H264, 208.15MiB
http-1194k-576p  mp4        1024x576   1194k , MP4 container, H264, 356.36MiB (best)

[ScriptSmith]

If you like, I'll dive in and figure out / change the code

[remitamine]

@ScriptSmith thanks for the help.
it should be fixed in the next version.

[ScriptSmith]

Yep, just checked and it already is