Unsupported URL: 10play.com.au

[flobbby]

• I'm reporting a new site support request
• I've verified that I'm running youtube-dl version 2019.06.08
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that none of provided URLs violate any copyrights
• I've searched the bugtracker for similar site support requests including closed ones

This is a verbose log from youtube-dl:

D:\Youtube-dl-win>youtube-dl.exe -v https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2019.06.08
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763
[debug] exe versions: none
[debug] Proxy map: {}
[generic] tpv190617avtsh: Requesting header
WARNING: Falling back on generic information extractor.
[generic] tpv190617avtsh: Downloading webpage
[generic] tpv190617avtsh: Extracting information
ERROR: Unsupported URL: https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp3esl0iyj\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp3esl0iyj\build\youtube_dl\extractor\common.py", line 530, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp3esl0iyj\build\youtube_dl\extractor\generic.py", line 3316, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh

Example URLs

These are a season-level pages, each of which list the 5 most recent episodes of the show. However, to get more episodes in the season, the user has to click on an arrow to get the next 5 episodes, and so on.
https://10play.com.au/masterchef/episodes/season-9/
https://10play.com.au/masterchef/episodes/season-11

Individual episodes have their own links, but the URLs have a strange pattern of characters on the end, which can only be found on the season-level page:
https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh
https://10play.com.au/masterchef/episodes/season-11/episode-37/tpv190614nckjz
https://10play.com.au/masterchef/episodes/season-11/episode-36/tpv190614rkbgz
https://10play.com.au/masterchef/episodes/season-11/episode-35/tpv190613biphv
https://10play.com.au/masterchef/episodes/season-11/episode-34/tpv190613nrztm

Description

The Australian 10play.com.au TV website is not supported.

Trying to grab arbitrary episodes from a season-level page is not necessary.

The website is geo-restricted to Australia.

[flobbby]

For reference, this is the source of the episode page for Season 11, Episode 38 of the Masterchef TV show.

10play Masterchef.txt

[WolfganP]

I suffered the same isse today. I was downloading the previous 37 episodes daily without any hiccups, but it failed today with a similar message (I even used -no-check-certificate and --geo-bypass-country AU to avoid side issues with https or geolocation)
The previous URL template was something like https://tenplay.com.au/masterchef/episodes/season-11/episode-37/ without the tpv* part, so it seems they introduced a page change today.

[ElectronicPunk]

The URL for an episode used to look like, for example:
https://tenplay.com.au/channel-ten/masterchef/season-11/episode-18

It changed to the new URL today as well.
I was able to grab it with another app called Tubedigger, so I don't think it impossible at least.

[WolfganP]

I found the verbose log of a recent session for the previous site. Pasting here for easier reference:

$ youtube-dl -v --geo-bypass-country AU --no-check-certificate https://tenplay.com.au/channel-ten/masterchef/season-11/episode-36 -f 'bestvideo+hls-audio-0-en__Main_-0' -o "/mnt/usb_12/xTransfer/%(title)s-%(id)s.%(ext)s"         [debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--geo-bypass-country', u'AU', u'--no-check-certificate', u'https://tenplay.com.au/channel-ten/masterchef/season-11/episode-36', u'-f', u'bestvideo+hls-audio-0-en__Main_-0', u'-o', u'/mnt/usb_12/xTransfer/%(title)s-%(id)s.%(ext)s']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.06.08
[debug] Python version 2.7.13 (CPython) - Linux-4.9.62-v7+-armv7l-with-debian-9.9
[debug] exe versions: ffmpeg 3.2.14-1, ffprobe 3.2.14-1, phantomjs .
[debug] Proxy map: {}
[debug] Using fake IP 1.136.24.252 (AU) as X-Forwarded-For.
[generic] episode-36: Requesting header
WARNING: Falling back on generic information extractor.
[generic] episode-36: Downloading webpage
[generic] episode-36: Extracting information
[download] Downloading playlist: MasterChef - S11 Ep. 36
[generic] playlist MasterChef - S11 Ep. 36: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Using fake IP 1.159.44.239 (AU) as X-Forwarded-For.
[brightcove:new] 6048168884001: Downloading webpage
[brightcove:new] 6048168884001: Downloading JSON metadata
[brightcove:new] 6048168884001: Downloading m3u8 information
[brightcove:new] 6048168884001: Downloading m3u8 information
[brightcove:new] 6048168884001: Downloading MPD manifest
[brightcove:new] 6048168884001: Downloading MPD manifest
[brightcove:new] 6048168884001: Downloading MPD manifest
[brightcove:new] 6048168884001: Downloading MPD manifest
[debug] Invoking downloader on u'https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/aafe3bbf-d41a-4556-879e-335f70aca6a4/f748b189-e664-4ec8-8133-2923e34b2469/10s/rendition.m3u8?fastly_token=NWQwODA2NGNfMzVmYmU4YzRlMmQxZTAyNjdiMzRjYWIxN2MxZDk2M2YzNTJjNDExNmUwNzllMjI1MDNhN2M2MjdlYTM5ZGQzNg%3D%3D'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 291
[download] Destination: /mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-1786-1.mp4
[download] 100% of 539.77MiB in 13:46
[debug] Invoking downloader on u'http://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/aafe3bbf-d41a-4556-879e-335f70aca6a4/dd8e35df-7f9a-4532-9450-f70af2e6376f/10s/rendition.m3u8?fastly_token=NWQwODA2NGNfMzIxMjhjODc3ZmVkZWQ3YTI2NDlmZWY3OGQxY2NjNmZkMWM5ZjUzZGNmN2MwOWJiMTBlZDNhNTQyMGRlYzUzYg%3D%3D'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 292
[download] Destination: /mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-audio-0-en__Main_-0.mp4
[download] 100% of 64.00MiB in 08:07
[ffmpeg] Merging formats into "/mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:/mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-1786-1.mp4' -i 'file:/mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-audio-0-en__Main_-0.mp4' -c copy -map '0:v:0' -map '1:a:0' 'file:/mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.temp.mp4'
Deleting original file /mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-1786-1.mp4 (pass -k to keep)
Deleting original file /mnt/usb_12/xTransfer/MasterChef - S11 Ep. 36-6048168884001.fhls-audio-0-en__Main_-0.mp4 (pass -k to keep)
[download] Finished downloading playlist: MasterChef - S11 Ep. 36

[flobbby]

Using youtube-dl version 2019.06.21 still fails:

D:\Youtube-dl-win>youtube-dl.exe --verbose https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2019.06.21
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763
[debug] exe versions: none
[debug] Proxy map: {}
[generic] tpv190617avtsh: Requesting header
WARNING: Falling back on generic information extractor.
[generic] tpv190617avtsh: Downloading webpage
[generic] tpv190617avtsh: Extracting information
ERROR: Unsupported URL: https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmppgbwk_g4\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmppgbwk_g4\build\youtube_dl\extractor\common.py", line 530, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmppgbwk_g4\build\youtube_dl\extractor\generic.py", line 3316, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: https://10play.com.au/masterchef/episodes/season-11/episode-38/tpv190617avtsh

[flobbby]

Got the direct master.m3u8 link (used the Firefox addon "The Stream Detector" for this), and put that through youtube-dl:

D:\Youtube-dl-win>youtube-dl.exe -F https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/887b73a9-5034-4362-9856-466fe83720c6/10s/master.m3u8?fastly_token=NWQwZmY2ZWFfNDg3YjE2MmU1NGM0YmMzNGViM2I3ZThjMTZhMTgxMWQ1NzJmMWY5NmUxMTc4MmQ0Y2Y1NzIwNjk4ODA5MDNjMQ%3D%3D -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/887b73a9-5034-4362-9856-466fe83720c6/10s/master.m3u8?fastly_token=NWQwZmY2ZWFfNDg3YjE2MmU1NGM0YmMzNGViM2I3ZThjMTZhMTgxMWQ1NzJmMWY5NmUxMTc4MmQ0Y2Y1NzIwNjk4ODA5MDNjMQ%3D%3D', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2019.06.21
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763
[debug] exe versions: none
[debug] Proxy map: {}
[generic] master: Requesting header
[generic] master: Downloading m3u8 information
[info] Available formats for master:
format code extension resolution note
audio-0-en__Main_ mp4 audio only [en]
audio-1-en__Main_ mp4 audio only [en]
245 mp4 284x160 245k , avc1.4d000c, video only
633 mp4 480x270 633k , avc1.420015, video only
908 mp4 640x360 908k , avc1.42001e, video only
1457 mp4 960x540 1457k , avc1.4d001f, video only
1787 mp4 1024x576 1787k , avc1.4d001f, video only (best)

I don't understand why the audio tracks have the mp4 extension, but that's not much of an issue.

At least I know I want to get the audio track id "audio-0-en__Main_", and the video track id "1787"

For the audio track:
D:\Youtube-dl-win>youtube-dl.exe --format audio-0-en__Main_ https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/887b73a9-5034-4362-9856-466fe83720c6/10s/master.m3u8?fastly_token=NWQwZmY2ZWFfNDg3YjE2MmU1NGM0YmMzNGViM2I3ZThjMTZhMTgxMWQ1NzJmMWY5NmUxMTc4MmQ0Y2Y1NzIwNjk4ODA5MDNjMQ%3D%3D

Renamed the downloaded file to something else before doing the next step...

For the video track:
D:\Youtube-dl-win>youtube-dl.exe --format 1787 https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/887b73a9-5034-4362-9856-466fe83720c6/10s/master.m3u8?fastly_token=NWQwZmY2ZWFfNDg3YjE2MmU1NGM0YmMzNGViM2I3ZThjMTZhMTgxMWQ1NzJmMWY5NmUxMTc4MmQ0Y2Y1NzIwNjk4ODA5MDNjMQ%3D%3D

Then I muxed the two files together, to get the final video+audio file.

Rather roundabout way, but it works.

There must be a better way, but my brain is fried.

[flobbby]

For what it's worth, using ffmpeg gets both the video and audio, and muxes it all into a single file:

ffmpeg.exe -i "https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/2199827728001/887b73a9-5034-4362-9856-466fe83720c6/10s/master.m3u8?fastly_token=NWQwZmViY2NfZGJlMmNhMTdmNzY1OTFiMTZiNTljOGE4NTM5MmZlMDY5ZDBlNTc0MjA4YzMwZTE0YjU1YzI0ZTYzMjIyZTU3MQ%3D%3D" -c copy S11E38.mp4

[ringofyre]

Can confirm

$ youtube-dl -v https://10play.com.au/have-you-been-paying-attention/episodes/2019/episode-7/tpv190624uwibp
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://10play.com.au/have-you-been-paying-attention/episodes/2019/episode-7/tpv190624uwibp']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.06.21
[debug] Python version 2.7.13 (CPython) - Linux-4.19.0-kali5-amd64-x86_64-with-debian-9.9
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] tpv190624uwibp: Requesting header
WARNING: Falling back on generic information extractor.
[generic] tpv190624uwibp: Downloading webpage
[generic] tpv190624uwibp: Extracting information
ERROR: Unsupported URL: https://10play.com.au/have-you-been-paying-attention/episodes/2019/episode-7/tpv190624uwibp
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2340, in _real_extract
doc = compat_etree_fromstring(webpage.encode('utf-8'))
File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2551, in compat_etree_fromstring
doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2540, in _XML
parser.feed(text)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
raise err
ParseError: not well-formed (invalid token): line 44, column 18
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 3316, in _real_extract
raise UnsupportedError(url)
UnsupportedError: Unsupported URL: https://10play.com.au/have-you-been-paying-attention/episodes/2019/episode-7/tpv190624uwibp