ABC iView HTTP Error 404: Not found

[xvids121]

youtube-dl reports HTTP:404 errors on iView video URLS. other sites work fine.

• can successfully download youtube videos + yahoo news videos (eg, https://au.news.yahoo.com/york-cats-become-first-us-pets-contract-coronavirus-210024387--spt.html and https://au.tv.yahoo.com/person-dresses-t-rex-shops-055357444.html)

Tried all the options I can think of in a number of combinations.

--no c:\users\user1.cache dir found, even hidden or system.
--ignore-config
--deleted c:\users\user1\youtube-dl.conf
--no-cache and --rm-cache-dir
--config-location as a freshly created zero-sized file.

-U update switch

--no-geo-bypass --no-cache-dir

• tried SSL switching off certificate checking

• overwriting youtube-dl.exe with the latest version downloaded
  -g yields same error HTTP:404.

• tried installed ffmpeg latest version and testing path calling.

• typing python at command prompt. launches a windows store python stub/bootstrapper which apparently was added to the May 10 Windows 10 update https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/

• manually installed python3.8 and successfully tested correct version running by typing python. This no longer launches the windows store stub.

• youtube-dl verbose information reports python v3.4, but the static version I installed is v3.8.2(tested calling path via command line was v3.8.2). Python was not installed prior to this. I was using the static version of python, do I need to use the shared object/dll version instead?

--cookies file
--disabled firewall and tested. socket error reported so this s a different error and ∴ probably not related to firewall.

This used to work seamlessly with iView but something has changed. Is there a way to get more verbosity so I can troubleshoot which (if any settings have changed)?

C:\TEMP\aBC-IVIEW>youtube-dl --no-check-certificate --no-geo-bypass --ignore-config --no-cache-dir --verbose https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--no-check-certificate', '--no-geo-bypass', '--ignore-config', '--no-cache-dir', '--verbose', 'https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.03.24
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: none
[debug] Proxy map: {}
[abc.net.au:iview] SC1802H004S00: Downloading JSON metadata
[abc.net.au:iview] SC1802H004S00: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

---

ffmpeg version git-2020-04-22-2e38c63 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9.3.1 (GCC) 20200328
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 42.102 / 56. 42.102
libavcodec 58. 80.100 / 58. 80.100
libavformat 58. 42.101 / 58. 42.101
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 79.100 / 7. 79.100
libswscale 5. 6.101 / 5. 6.101
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

[ilike2burnthing]

Remove --no-geo-bypass

Working here with:

• youtube-dl.exe https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00
• youtube-dl.exe --newline -i -o "D:\Downloads\%(title)s.%(ext)s" --ignore-config --hls-prefer-native --add-metadata "https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00"
• youtube-dl --no-check-certificate --ignore-config --no-cache-dir --verbose https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00

[ilike2burnthing]

Similar issue #24680 - may be related to Python being on the system, given that's where the error appears in the log

[xvids121]

Similar issue #24680 - may be related to Python being on the system, given that's where the error appears in the log

Disabled the windows python.exe stub in settings->apps->app execution aliases and uninstalled python. Same issue.

Also tried both --no-geo-bypass and normal mode.
Perhaps this is related to SSL or some other settings? Is there a way to get more verbosity/information on specifically what youtube-dl is communicating to see why it is not found?

[ilike2burnthing]

--print-traffic, --write-pages, and -j would be your best options:
https://github.com/ytdl-org/youtube-dl#verbosity--simulation-options

[xvids121]

Looks like the main links are not working. I tried using with and without -g options for some of the .m3u8 files but am not sure which .m3u8 files to use.

End up getting an error. I have attached 2 files.
See SC1802H008S00_https_-_iview.abc.net.au_api_programs_catalyst_SC1802H008S00.dump for the main file.

See additional_information for output of what happens when trying grab video using .m3u8 files.

Not sure what else I can try at this point or what else I can test. Any ideas?

additional_information.txt

SC1802H008S00_https_-_iview.abc.net.au_api_programs_catalyst_SC1802H008S00.dump.txt

[ilike2burnthing]

I get the same errors using https://iview-vod.akamaized.net/prod/cla/classification_PG_2019/20191126150417/fairplay/master.m3u8

Available streams for https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00:

streams: {mpegdash: {,…}, fairplay: {,…}, hls: {,…}}
fairplay: {,…}
protected: true
sd: "https://iview-vod.akamaized.net/prod/sc1/SC1802H004S00CE1D1/20190219155946/fairplay/master.m3u8"
sd-low: "https://iview-vod.akamaized.net/prod/sc1/SC1802H004S00CE1D1/20190219155946/fairplay/master.m3u8"
hls: {,…}
protected: false
sd: "https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8?subtitle_file_pattern=identifier_langcode&subtitle_identifier=SC1802H004S00CE1D1&subtitle_location=%2Fplayback%2Fcaptions%2Fprod%2Fsc"
sd-low: "https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,650000,500000,220000,.mp4.csmil/master.m3u8?subtitle_file_pattern=identifier_langcode&subtitle_identifier=SC1802H004S00CE1D1&subtitle_location=%2Fplayback%2Fcaptions%2Fprod%2Fsc"
mpegdash: {,…}
protected: true
sd: "https://iview-vod.akamaized.net/prod/sc1/SC1802H004S00CE1D1/20190219155946/mpegdash/master.mpd"
sd-low: "https://iview-vod.akamaized.net/prod/sc1/SC1802H004S00CE1D1/20190219155946/mpegdash/master.mpd"

YTDL can only download the unprotected streams. For me, it uses https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8 but then adds a different id string each time (reusing an old one fails), e.g.:

https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8?null=0&id=AgBSDPQXAhVLHaj5ol7Orzjb1cu1pG7mZHsP7qtbKUoK+NjWQ7CBVnKOsO9VULx6phyj926EFEz+rw%3d%3d

For me this is entirely automated, so I'm not sure why it doesn't do that for you, or how to work around it.

Have you tried - https://github.com/ytdl-org/youtube-dl#developer-instructions
Download, unzip, open cmd in folder, run python -m youtube_dl https://iview.abc.net.au/show/catalyst/series/20/video/SC1802H004S00

Do you have another computer you can try it on? One that hasn't had Python or youtube-dl on it? Or can you use Windows Sandbox?

[xvids121]

Strangely I seem to get two errors and one warning, indicated in bold below:

Is the failed HEAD request causing the 405 and/or 403 errors? Is there a different mode for this?
Will try a fresh install of Windows 7 in a VM.

---

C:\TEMP\aBC-IVIEW>youtube-dl --rm-cache-dir --ignore-config --no-geo-bypass https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8
Removing cache dir C:\Users\bates/.cache\youtube-dl ..
[generic] master: Requesting header
WARNING: Could not send HEAD request to https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8: HTTP Error 405: Method Not Allowed
[generic] master: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[ilike2burnthing]

I get the same warning and errors if I just use https://iviewhlsvod-vh.akamaihd.net/i/playback/_definst_/_video/sc1/SC1802H004S00CE1D1_20190219155946_,1500000,1000000,650000,500000,220000,.mp4.csmil/master.m3u8, it needs the id string after it.

I tried --add-header but nothing I used worked. Also played around with using ffmpeg directly, but to no avail.

Fingers crossed for your VM.