[ITV] ERROR: Unable to download webpage: '' (caused by BadStatusLine("''",));

[MysticMerlin]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2021.12.17
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

PASTE VERBOSE LOG HERE

Description

WRITE DESCRIPTION HERE
I hope I'm reporting this correctly.
The error, which is not with every download, started about a day or two ago which is,
ERROR: Unable to download webpage: '' (caused by BadStatusLine("''",));
Sometimes I get the error 404 but it's really the above error. It affects only downloads from the ITV channel & it can be with a news programme from yesterday to Emmerdale from the 01st January to the 14th January to other ones since then but not all of them.
The latest youtube-dl file 17-12-2021 seemed to have more problems than the one I was using which was the 26-04-2021.
Until about 2 days ago I've never had or seen the above error before.
Firstly, the following is what came up when I first tried to download the programme.

C:\Users\User\Videos\youtube-dl Downloads>youtube-dl https://www.itv.com/hub/itv-evening-news/2a4545a1578
[ITV] 2a4545a1578: Downloading webpage
ERROR: Unable to download webpage: '' (caused by BadStatusLine("''",)); please report this issue on https://yt-dl.org/bug .
Make sure you are using the latest version; type youtube-dl -U to update. Be sure to
call youtube-dl with the --verbose flag and include its complete output.

Then the following is the verbose.

C:\Users\User\Videos\youtube-dl Downloads>youtube-dl https://www.itv.com/hub/itv-evening-news/2a4545a1578 --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.itv.com/hub/itv-evening-news/2a4545a1578', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 4.0.2, ffprobe 3.4.1
[debug] Proxy map: {}
[debug] Using fake IP 25.25.61.236 (GB) as X-Forwarded-For.
[ITV] 2a4545a1578: Downloading webpage
[ITV] 2a4545a1578: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug .
Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

As well as the above programme the following are a small selection of other programmes with the same error trying to download.
youtube-dl https://www.itv.com/hub/good-morning-britain/2a3211a3218
youtube-dl https://www.itv.com/hub/coronation-street/1a0694a10537
youtube-dl https://www.itv.com/hub/loose-women/1a3173a3902
youtube-dl https://www.itv.com/hub/this-morning/2a6292a0758
youtube-dl https://www.itv.com/hub/itv-evening-news/2a4545a1579

I've just tried a different wi-fi network & it's the same.
I've noticed in the verbose it says I've got Python version 3.4.4 (CPython) but in Control Panel it says I've got Python 3.9.5 (64-bit). The reason why I was checking to see if I've got the latest version thinking that by getting the latest version might have an effect on the error.

[dirkf]

There's a PR #30266 that is needed for ITV now (also use --user-agent "Mozilla/5.0" if the extractor hangs or times out downloading the webpage), or use yt-dlp. If you were using either of the last two release versions, I'm surprised that it worked at all.

In either case https://www.itv.com/hub/itv-evening-news/2a4545a1579 works but https://www.itv.com/hub/itv-evening-news/2a4545a1578 gives 404, "No longer available" according to ITV.

[dirkf]

WFM in UK (yt-dlp 'https://www.itv.com/hub/itv-evening-news/2a4545a1580') without any specific UA.

[MysticMerlin]

Hi,
I've had 2 family birthdays this week including on Saturday so hence my delay in catching up with the latest.
I will first say thanks to dirkf for the help/contribution etc as well as anybody else who has done so as well.
I will say dirkf that I'm intelligent but lack knowledge & experience so when you mentioned above 'There's a PR #30266 that is needed for ITV now' I didn't know what it meant precisely. Same with some other things. Then when I saw someone had put yt-dlp before a programme I thought I would have to do that with the youtube-dl file that I've got which was the latest from 17-12-2021. Well that didn't work so I thought that there must be another application file called yt-dlp so obviously I searched for that on the site & downloaded it & popped it in front of another programme to download i.e. yt-dlp https://www.itv.com/hub/itv-news-weekend-teatime/2a4546a0629 & the download worked, fantastic. I didn't know of the existence of the file yt-dlp until now. Anyway the programme downloaded but it kept on stopping & starting. I thought it might be my internet connection but I didn't have any problems with speed etc when I used get_iplayer to do a download from the BBC. Anyway it did finish downloading eventually with one time out as can be seen in the screenshots I've uploaded. As I've only done the one download I don't know if the stopping & starting will be with all downloads from now on. At least I can download again thanks to you guys.

[MysticMerlin]

Just done some more downloads & they've taken a long time to complete because of the stopping & starting. I've submitted another screenshot so you can see how the 2nd download went.

[dirkf]

yt-dlp is a forked version of yt-dl that only supports newer Python versions and has some further potential capabilities and dependencies.

As you're using yt-dlp, you should raise any problems in its tracker. Try to provide text logs (see, eg, https://www.laptopmag.com/articles/how-to-windows-10-command-prompt-copy) and use the --verbose/-v option. Having said that, yt-dlp's ITV extractor is essentially the same as the version in the yt-dl PR #30266.

Your timeout issues might be a problem with your internet connection, or with the ITV site, or possibly deliberate interference by the ITV site which might be alleviated by specifying a different user agent string, eg --user-agent "Mozilla/5.0" as suggested above.