[abc.net.au] Unable to extract video urls

[raphaelmerx]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• I'm reporting that a supported site is broken
• I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

France

Provide a description that is worded well enough to be understood

Getting error Unable to extract video urls when trying to extract videos from abc.net.au. Error occurs for the test URLs listed in the extractor (e.g. http://www.abc.net.au/news/2014-11-05/australia-to-staff-ebola-treatment-centre-in-sierra-leone/5868334), despite the video working in browser.

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• If using API, add 'verbose': True to YoutubeDL params instead
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['http://www.abc.net.au/news/2014-11-05/australia-to-staff-ebola-treatment-centre-in-sierra-leone/5868334', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (pip)
[debug] Python 3.9.13 (CPython arm64 64bit) - macOS-13.1-arm64-arm-64bit (OpenSSL 1.1.1t  7 Feb 2023)
[debug] exe versions: ffmpeg 5.1.2 (setts), ffprobe 5.1.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.05.18.1, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[abc.net.au] Extracting URL: http://www.abc.net.au/news/2014-11-05/australia-to-staff-ebola-treatment-centre-in-sierra-leone/5868334
[abc.net.au] 5868334: Downloading webpage
ERROR: [abc.net.au] 5868334: Unable to extract video urls; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "envs/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "envs/lib/python3.9/site-packages/yt_dlp/extractor/abc.py", line 119, in _real_extract
    raise ExtractorError('Unable to extract video urls')

[gamer191]

related: ytdl-org/youtube-dl#30887 #3968

Does --ies generic,default still workaround this issue?

Error occurs for the test URLs listed in the extractor (e.g. http://www.abc.net.au/news/2014-11-05/australia-to-staff-ebola-treatment-centre-in-sierra-leone/5868334), despite the video working in browser.

Um, where's the video?

[gamer191]

Did the video get removed or something?

Anyway, here's a new verbose log that definitely has a video (actually 2 videos):

[debug] Command-line config: ['https://www.abc.net.au/news/2023-03-05/nsw-votes-labor-focus-skills-young-people-at-campaign-launch/102054006', '--verbose']
[debug] User config "C:\Users\jaybu\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\jaybu\\ffmpeg\\bin', '-P', 'C:\\Users\\jaybu\\youtube.dl']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.03.04.220756 [392389b7d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[abc.net.au] Extracting URL: https://www.abc.net.au/news/2023-03-05/nsw-votes-labor-focus-skills-young-people-at-campaign-launch/102054006
[abc.net.au] 102054006: Downloading webpage
ERROR: [abc.net.au] 102054006: Unable to extract video urls; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 694, in extract
  File "yt_dlp\extractor\abc.py", line 119, in _real_extract

And my workaround does, in fact, work:

[debug] Command-line config: ['https://www.abc.net.au/news/2023-03-05/nsw-votes-labor-focus-skills-young-people-at-campaign-launch/102054006', '--verbose', '--ies', 'generic,default']
[debug] User config "C:\Users\jaybu\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\jaybu\\ffmpeg\\bin', '-P', 'C:\\Users\\jaybu\\youtube.dl']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.03.04.220756 [392389b7d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[generic] Extracting URL: https://www.abc.net.au/news/2023-03-05/nsw-votes-labor-focus-skills-young-people-at-campaign-launch/102054006
[generic] 102054006: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 102054006: Extracting information
[debug] Looking for embeds
[debug] Identified 2 html5 embeds
[download] Downloading playlist: 'Plagued by scandal': PM attacks Perrottet government at NSW Labor election launch
[generic] Playlist 'Plagued by scandal': PM attacks Perrottet government at NSW Labor election launch: Downloading 2 items of 2
[download] Downloading item 1 of 2
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 102054006-1: Downloading 1 format(s): 0
[debug] Invoking http downloader on "https://mediacore-live-production.akamaized.net/video/01/g4/Z/71.mp4"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: C:\Users\jaybu\youtube.dl\'Plagued by scandal'： PM attacks Perrottet government at NSW Labor election launch (1) [102054006-1].mp4
[download] 100% of    5.32MiB in 00:00:01 at 4.71MiB/s
[download] Downloading item 2 of 2
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 102054006-2: Downloading 1 format(s): 0
[debug] Invoking http downloader on "https://mediacore-live-production.akamaized.net/video/01/fv/Z/3b.mp4"
[download] Destination: C:\Users\jaybu\youtube.dl\'Plagued by scandal'： PM attacks Perrottet government at NSW Labor election launch (2) [102054006-2].mp4
[download] 100% of  294.97MiB in 00:00:54 at 5.37MiB/s
[download] Finished downloading playlist: 'Plagued by scandal': PM attacks Perrottet government at NSW Labor election launch

[raphaelmerx]

Um, where's the video?

Ah yep, sorry, didn't realise this video had been removed. Initially tried with https://www.abc.net.au/news/2023-02-13/carbon-colonialism/101968870 which failed with "Unable to extract video urls"

Can confirm that --ies generic,default works

[meliber]

Um, where's the video?

Ah yep, sorry, didn't realise this video had been removed. Initially tried with https://www.abc.net.au/news/2023-02-13/carbon-colonialism/101968870 which failed with "Unable to extract video urls"

Can confirm that --ies generic,default works

This bug happens again.

[debug] Command-line config: ['https://www.abc.net.au/news/2023-06-25/wagner-boss-orders-troops-back-to-bases-to-avoid-bloodshed/102520540', '--verbose', '--ies', 'generic,default']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.06.22 [812cdfa06]
[debug] Python 3.11.3 (CPython x86_64 64bit) - Linux-6.3.8-arch1-1-x86_64-with-glibc2.37 (OpenSSL 3.1.1 30 May 2023, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, brotlicffi-1.0.9.2, certifi-2023.05.07, secretstorage-3.3.3, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[generic] Extracting URL: https://www.abc.net.au/news/2023-06-25/wagner-boss-orders-troops-back-to-bases-to-avoid-bloodshed/102520540
[generic] 102520540: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 102520540: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.abc.net.au/news/2023-06-25/wagner-boss-orders-troops-back-to-bases-to-avoid-bloodshed/102520540
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1555, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1631, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 708, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/generic.py", line 2568, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.abc.net.au/news/2023-06-25/wagner-boss-orders-troops-back-to-bases-to-avoid-bloodshed/102520540