The Highwire https://thehighwire.com - Site Support Request

[coop555]

Checklist

• [ x] I'm reporting a new site support request
• [ x] I've verified that I'm running youtube-dl version 2021.06.06
• [ x] I've checked that all provided URLs are alive and playable in a browser
• [ x] I've checked that none of provided URLs violate any copyrights
• [ x] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

• Single video: https://thehighwire.com/videos/risky-business/
• Single video: https://thehighwire.com/videos/episode-219-in-harms-way/
• Single video: https://thehighwire.com/videos/live-from-event-2021-in-dallas-tx/
• Single video: https://thehighwire.com/videos/concealing-casualties/
• Single video: https://thehighwire.com/videos/the-definition-of-a-bio-weapon/

Description

The videos can be downloaded by youtube-dl if you discover their real URLs, for example: https://media.livecast365.com/highwire/thehighwire/content/1622927384709.mp4

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://thehighwire.com/videos/live-from-event-2021-in-dallas-tx/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.9.2 (CPython) - Linux-5.10.25-linuxkit-x86_64-with-glibc2.31
[debug] exe versions: ffmpeg 4.3.2-0, ffprobe 4.3.2-0, rtmpdump 2.4
[debug] Proxy map: {}
[generic] live-from-event-2021-in-dallas-tx: Requesting header
WARNING: Falling back on generic information extractor.
[generic] live-from-event-2021-in-dallas-tx: Downloading webpage
[generic] live-from-event-2021-in-dallas-tx: Extracting information
ERROR: Unsupported URL: https://thehighwire.com/videos/live-from-event-2021-in-dallas-tx/
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
ie_result = self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 3520, in _real_extract
raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://thehighwire.com/videos/live-from-event-2021-in-dallas-tx/

[dirkf]

The video URL (or URLs) is listed in the sources element of the JSON object that is the value of the data-vbwa attribute of the <video> element.

[SuperSonicHub1]

How to Crack The Highwire

URL Parsing

https://thehighwire.com/videos/the-definition-of-a-bio-weapon/
https://thehighwire.com/videos/<slug>/

Download Video

• DL webpage
• Get JSON+LD on webpage (script.yoast-schema-graph) for metadata
  • for each item in root["@graph"] [array]
    • where item["@type"] == "ImageObject" (thumbnail)
      • "url"
      • "width"
      • "height"
    • where item["@type"] == "WebPage" (video)
      • "datePublished"
      • "name"
• Get JSON from attribute video[data-vbwa] for getting URL to video (thanks @dirkf!)
  • root["sources"][0] (MP4 URL)

Super easy extractor to implement. If I don't implement this extractor myself, I'll leave it as an exercise to the reader.