Unable to download JSON metadata error on yt #28289

aram535 · 2021-02-27T11:49:47Z

Checklist

I'm reporting a broken site support issue
I've verified that I'm running youtube-dl version 2021.02.22 -- youtube-dl is up-to-date (2021.02.22)
I've checked that all provided URLs are alive and playable in a browser
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar bug reports including closed ones
I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: ['--download-archive', 'archive.txt', '-i', '-w', '-c', '--no-mtime', '--restrict-filenames', '--prefer-ffmpeg', '--min-sleep-interval', '2', '--max-sleep-interval', '20', '-o', '%(title)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['--download-archive', 'foodwishes.txt', '--verbose', '-i', '-q', '--no-warnings', '--restrict-filenames', '--add-metadata', '--min-sleep-interval', '1.5', '--max-sleep-interval', '7', '-o', '%(title)s.%(ext)s', 'https://www.youtube.com/user/foodwishes']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.22
[debug] Python version 3.8.3 (CPython) - Linux-4.18.0-240.10.1.el8_3.x86_64-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Proxy map: {}
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 632, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2275, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib64/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib64/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib64/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib64/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 632, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2275, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib64/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib64/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib64/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib64/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Description

Simple download of a fav playlist from yt. It has been working for years and after updating to 02.22, it seems to have broken this morning.

The text was updated successfully, but these errors were encountered:

mechalincoln · 2021-02-27T23:55:25Z

I am experiencing this issue too, for a channel with thousands of videos that worked fine until today:

Y:\non_webm>youtube-dl --cookies cookies.txt --download-archive "seelenleid_hq.log" -i --add-metadata --embed-thumbnail -o "%(playlist)s/%(title)s - %(id)s.%(ext)s" -f --merge-output-format --format (305/266/304/264/299/137/298/136/135/134/133/160/bestvideo)+(258/141/256/140/139/bestaudio) -x -k --write-description --cache-dir "Z:\cache" --sleep-interval 5 --max-sleep-interval 10 --fragment-retries 25 https://www.youtube.com/channel/UCmc-OtwOUi12C9-mzO_PW8A/videos --verbose
[debug] System config: []
[debug] User config: ['-x', '-u', 'PRIVATE', '-p', 'PRIVATE']
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--download-archive', 'seelenleid_hq.log', '-i', '--add-metadata', '--embed-thumbnail', '-o', '%(playlist)s/%(title)s - %(id)s.%(ext)s', '-f', '--merge-output-format', '--format', '(305/266/304/264/299/137/298/136/135/134/133/160/bestvideo)+(258/141/256/140/139/bestaudio)', '-x', '-k', '--write-description', '--cache-dir', 'Z:\cache', '--sleep-interval', '5', '--max-sleep-interval', '10', '--fragment-retries', '25', 'https://www.youtube.com/channel/UCmc-OtwOUi12C9-mzO_PW8A/videos', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.02.22
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 2021-02-16-git-8331eb058d-full_build-www.gyan.dev, ffprobe 2021-02-16-git-8331eb058d-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube:tab] Downloading login page
[youtube:tab] Looking up account info
WARNING: Unable to look up account info: HTTP Error 400: Bad Request
[youtube:tab] UCmc-OtwOUi12C9-mzO_PW8A: Downloading webpage
[download] Downloading playlist: Seelenleid - Videos
[youtube:tab] Downloading page 1
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Y:\non_webm>

Evaia · 2021-02-28T02:59:04Z

I also experience this. Was working fine up until a few hours ago.

F:\Media\Web Videos\WatchLater>youtube-dl --format "(bestvideo[vcodec^=av01][height>=4320][fps>30]/bestvideo[vcodec^=vp9.2][height>=4320][fps>30]/bestvideo[vcodec^=vp9][height>=4320][fps>30]/bestvideo[vcodec^=avc1][height>=4320][fps>30]/bestvideo[height>=4320][fps>30]/bestvideo[vcodec^=av01][height>=4320]/bestvideo[vcodec^=vp9.2][height>=4320]/bestvideo[vcodec^=vp9][height>=4320]/bestvideo[vcodec^=avc1][height>=4320]/bestvideo[height>=4320]/bestvideo[vcodec^=av01][height>=2880][fps>30]/bestvideo[vcodec^=vp9.2][height>=2880][fps>30]/bestvideo[vcodec^=vp9][height>=2880][fps>30]/bestvideo[vcodec^=avc1][height>=2880][fps>30]/bestvideo[height>=2880][fps>30]/bestvideo[vcodec^=av01][height>=2880]/bestvideo[vcodec^=vp9.2][height>=2880]/bestvideo[vcodec^=vp9][height>=2880]/bestvideo[vcodec^=avc1][height>=2880]/bestvideo[height>=2880]/bestvideo[vcodec^=av01][height>=2160][fps>30]/bestvideo[vcodec^=vp9.2][height>=2160][fps>30]/bestvideo[vcodec^=vp9][height>=2160][fps>30]/bestvideo[vcodec^=avc1][height>=2160][fps>30]/bestvideo[height>=2160][fps>30]/bestvideo[vcodec^=av01][height>=2160]/bestvideo[vcodec^=vp9.2][height>=2160]/bestvideo[vcodec^=vp9][height>=2160]/bestvideo[vcodec^=avc1][height>=2160]/bestvideo[height>=2160]/bestvideo[vcodec^=av01][height>=1440][fps>30]/bestvideo[vcodec^=vp9.2][height>=1440][fps>30]/bestvideo[vcodec^=vp9][height>=1440][fps>30]/bestvideo[vcodec^=avc1][height>=1440][fps>30]/bestvideo[height>=1440][fps>30]/bestvideo[vcodec^=av01][height>=1440]/bestvideo[vcodec^=vp9.2][height>=1440]/bestvideo[vcodec^=vp9][height>=1440]/bestvideo[vcodec^=avc1][height>=1440]/bestvideo[height>=1440]/bestvideo[vcodec^=av01][height>=1080][fps>30]/bestvideo[vcodec^=vp9.2][height>=1080][fps>30]/bestvideo[vcodec^=vp9][height>=1080][fps>30]/bestvideo[vcodec^=avc1][height>=1080][fps>30]/bestvideo[height>=1080][fps>30]/bestvideo[vcodec^=av01][height>=1080]/bestvideo[vcodec^=vp9.2][height>=1080]/bestvideo[vcodec^=vp9][height>=1080]/bestvideo[vcodec^=avc1][height>=1080]/bestvideo[height>=1080]/bestvideo[vcodec^=av01][height>=720][fps>30]/bestvideo[vcodec^=vp9.2][height>=720][fps>30]/bestvideo[vcodec^=vp9][height>=720][fps>30]/bestvideo[vcodec^=avc1][height>=720][fps>30]/bestvideo[height>=720][fps>30]/bestvideo[vcodec^=av01][height>=720]/bestvideo[vcodec^=vp9.2][height>=720]/bestvideo[vcodec^=vp9][height>=720]/bestvideo[vcodec^=avc1][height>=720]/bestvideo[height>=720]/bestvideo[vcodec^=av01][height>=480][fps>30]/bestvideo[vcodec^=vp9.2][height>=480][fps>30]/bestvideo[vcodec^=vp9][height>=480][fps>30]/bestvideo[vcodec^=avc1][height>=480][fps>30]/bestvideo[height>=480][fps>30]/bestvideo[vcodec^=av01][height>=480]/bestvideo[vcodec^=vp9.2][height>=480]/bestvideo[vcodec^=vp9][height>=480]/bestvideo[vcodec^=avc1][height>=480]/bestvideo[height>=480]/bestvideo[vcodec^=av01][height>=360][fps>30]/bestvideo[vcodec^=vp9.2][height>=360][fps>30]/bestvideo[vcodec^=vp9][height>=360][fps>30]/bestvideo[vcodec^=avc1][height>=360][fps>30]/bestvideo[height>=360][fps>30]/bestvideo[vcodec^=av01][height>=360]/bestvideo[vcodec^=vp9.2][height>=360]/bestvideo[vcodec^=vp9][height>=360]/bestvideo[vcodec^=avc1][height>=360]/bestvideo[height>=360]/bestvideo[vcodec^=avc1][height>=240][fps>30]/bestvideo[vcodec^=av01][height>=240][fps>30]/bestvideo[vcodec^=vp9.2][height>=240][fps>30]/bestvideo[vcodec^=vp9][height>=240][fps>30]/bestvideo[height>=240][fps>30]/bestvideo[vcodec^=avc1][height>=240]/bestvideo[vcodec^=av01][height>=240]/bestvideo[vcodec^=vp9.2][height>=240]/bestvideo[vcodec^=vp9][height>=240]/bestvideo[height>=240]/bestvideo[vcodec^=avc1][height>=144][fps>30]/bestvideo[vcodec^=av01][height>=144][fps>30]/bestvideo[vcodec^=vp9.2][height>=144][fps>30]/bestvideo[vcodec^=vp9][height>=144][fps>30]/bestvideo[height>=144][fps>30]/bestvideo[vcodec^=avc1][height>=144]/bestvideo[vcodec^=av01][height>=144]/bestvideo[vcodec^=vp9.2][height>=144]/bestvideo[vcodec^=vp9][height>=144]/bestvideo[height>=144]/bestvideo)+(bestaudio[acodec^=opus]/bestaudio)/best" --verbose --force-ipv4 --sleep-interval 5 --max-sleep-interval 30 --ignore-errors --no-continue --no-overwrites --download-archive archive.log --add-metadata --write-annotations --all-subs --embed-subs --match-filter "!is_live & !live" --output "%(upload_date)s - %(title)s [%(id)s].%(ext)s" --merge-output-format "mkv" --cookies=cookies.txt :ytwatchlater
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--format', '(bestvideo[vcodec^=av01][height>=4320][fps>30]/bestvideo[vcodec^=vp9.2][height>=4320][fps>30]/bestvideo[vcodec^=vp9][height>=4320][fps>30]/bestvideo[vcodec^=avc1][height>=4320][fps>30]/bestvideo[height>=4320][fps>30]/bestvideo[vcodec^=av01][height>=4320]/bestvideo[vcodec^=vp9.2][height>=4320]/bestvideo[vcodec^=vp9][height>=4320]/bestvideo[vcodec^=avc1][height>=4320]/bestvideo[height>=4320]/bestvideo[vcodec^=av01][height>=2880][fps>30]/bestvideo[vcodec^=vp9.2][height>=2880][fps>30]/bestvideo[vcodec^=vp9][height>=2880][fps>30]/bestvideo[vcodec^=avc1][height>=2880][fps>30]/bestvideo[height>=2880][fps>30]/bestvideo[vcodec^=av01][height>=2880]/bestvideo[vcodec^=vp9.2][height>=2880]/bestvideo[vcodec^=vp9][height>=2880]/bestvideo[vcodec^=avc1][height>=2880]/bestvideo[height>=2880]/bestvideo[vcodec^=av01][height>=2160][fps>30]/bestvideo[vcodec^=vp9.2][height>=2160][fps>30]/bestvideo[vcodec^=vp9][height>=2160][fps>30]/bestvideo[vcodec^=avc1][height>=2160][fps>30]/bestvideo[height>=2160][fps>30]/bestvideo[vcodec^=av01][height>=2160]/bestvideo[vcodec^=vp9.2][height>=2160]/bestvideo[vcodec^=vp9][height>=2160]/bestvideo[vcodec^=avc1][height>=2160]/bestvideo[height>=2160]/bestvideo[vcodec^=av01][height>=1440][fps>30]/bestvideo[vcodec^=vp9.2][height>=1440][fps>30]/bestvideo[vcodec^=vp9][height>=1440][fps>30]/bestvideo[vcodec^=avc1][height>=1440][fps>30]/bestvideo[height>=1440][fps>30]/bestvideo[vcodec^=av01][height>=1440]/bestvideo[vcodec^=vp9.2][height>=1440]/bestvideo[vcodec^=vp9][height>=1440]/bestvideo[vcodec^=avc1][height>=1440]/bestvideo[height>=1440]/bestvideo[vcodec^=av01][height>=1080][fps>30]/bestvideo[vcodec^=vp9.2][height>=1080][fps>30]/bestvideo[vcodec^=vp9][height>=1080][fps>30]/bestvideo[vcodec^=avc1][height>=1080][fps>30]/bestvideo[height>=1080][fps>30]/bestvideo[vcodec^=av01][height>=1080]/bestvideo[vcodec^=vp9.2][height>=1080]/bestvideo[vcodec^=vp9][height>=1080]/bestvideo[vcodec^=avc1][height>=1080]/bestvideo[height>=1080]/bestvideo[vcodec^=av01][height>=720][fps>30]/bestvideo[vcodec^=vp9.2][height>=720][fps>30]/bestvideo[vcodec^=vp9][height>=720][fps>30]/bestvideo[vcodec^=avc1][height>=720][fps>30]/bestvideo[height>=720][fps>30]/bestvideo[vcodec^=av01][height>=720]/bestvideo[vcodec^=vp9.2][height>=720]/bestvideo[vcodec^=vp9][height>=720]/bestvideo[vcodec^=avc1][height>=720]/bestvideo[height>=720]/bestvideo[vcodec^=av01][height>=480][fps>30]/bestvideo[vcodec^=vp9.2][height>=480][fps>30]/bestvideo[vcodec^=vp9][height>=480][fps>30]/bestvideo[vcodec^=avc1][height>=480][fps>30]/bestvideo[height>=480][fps>30]/bestvideo[vcodec^=av01][height>=480]/bestvideo[vcodec^=vp9.2][height>=480]/bestvideo[vcodec^=vp9][height>=480]/bestvideo[vcodec^=avc1][height>=480]/bestvideo[height>=480]/bestvideo[vcodec^=av01][height>=360][fps>30]/bestvideo[vcodec^=vp9.2][height>=360][fps>30]/bestvideo[vcodec^=vp9][height>=360][fps>30]/bestvideo[vcodec^=avc1][height>=360][fps>30]/bestvideo[height>=360][fps>30]/bestvideo[vcodec^=av01][height>=360]/bestvideo[vcodec^=vp9.2][height>=360]/bestvideo[vcodec^=vp9][height>=360]/bestvideo[vcodec^=avc1][height>=360]/bestvideo[height>=360]/bestvideo[vcodec^=avc1][height>=240][fps>30]/bestvideo[vcodec^=av01][height>=240][fps>30]/bestvideo[vcodec^=vp9.2][height>=240][fps>30]/bestvideo[vcodec^=vp9][height>=240][fps>30]/bestvideo[height>=240][fps>30]/bestvideo[vcodec^=avc1][height>=240]/bestvideo[vcodec^=av01][height>=240]/bestvideo[vcodec^=vp9.2][height>=240]/bestvideo[vcodec^=vp9][height>=240]/bestvideo[height>=240]/bestvideo[vcodec^=avc1][height>=144][fps>30]/bestvideo[vcodec^=av01][height>=144][fps>30]/bestvideo[vcodec^=vp9.2][height>=144][fps>30]/bestvideo[vcodec^=vp9][height>=144][fps>30]/bestvideo[height>=144][fps>30]/bestvideo[vcodec^=avc1][height>=144]/bestvideo[vcodec^=av01][height>=144]/bestvideo[vcodec^=vp9.2][height>=144]/bestvideo[vcodec^=vp9][height>=144]/bestvideo[height>=144]/bestvideo)+(bestaudio[acodec^=opus]/bestaudio)/best', '--verbose', '--force-ipv4', '--sleep-interval', '5', '--max-sleep-interval', '30', '--ignore-errors', '--no-continue', '--no-overwrites', '--download-archive', 'archive.log', '--add-metadata', '--write-annotations', '--all-subs', '--embed-subs', '--match-filter', '!is_live & !live', '--output', '%(upload_date)s - %(title)s [%(id)s].%(ext)s', '--merge-output-format', 'mkv', '--cookies=cookies.txt', ':ytwatchlater']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.02.22
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 2020-11-04-git-cfdddec0c8-full_build-www.gyan.dev, ffprobe 2020-11-04-git-cfdddec0c8-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube:tab] WL: Downloading webpage
[download] Downloading playlist: Watch later
[youtube:tab] Downloading page 1
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

coletdjnz · 2021-02-28T04:50:58Z

I'm also noticing this too on a few automated downloads but I'm having trouble reproducing this on my main with verbose logging, but it's the same behavior as the others.

YouTube is sometimes giving a 404 on a playlist page download, which doesn't seem to depend on which page.

Edit: Processing my logs, it seems the rate of 404's peaks sometime between 6am-12pm UTC over the past couple days, so I'm
thinking this is a server-side issue.

Edit2: this appears to have started happening after the YouTube channel issues the other day.

Evaia · 2021-02-28T14:02:01Z

Interesting, this morning it behaves slightly differently; it started to gather the info on the watchlater playlist, but then JSON errored out.

[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.02.22
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 2021-02-23-git-78d5e1c653-full_build-www.gyan.dev, ffprobe 2021-02-23-git-78d5e1c653-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube:tab] WL: Downloading webpage
[download] Downloading playlist: Watch later
[youtube:tab] Downloading page 1
[youtube:tab] Downloading page 2
[youtube:tab] Downloading page 3
[youtube:tab] Downloading page 4
[youtube:tab] Downloading page 5
[youtube:tab] Downloading page 6
[youtube:tab] Downloading page 7
[youtube:tab] Downloading page 8
[youtube:tab] Downloading page 9
[youtube:tab] Downloading page 10
[youtube:tab] Downloading page 11
[youtube:tab] Downloading page 12
[youtube:tab] Downloading page 13
[youtube:tab] Downloading page 14
[youtube:tab] Downloading page 15
[youtube:tab] Downloading page 16
[youtube:tab] Downloading page 17
[youtube:tab] Downloading page 18
[youtube:tab] Downloading page 19
[youtube:tab] Downloading page 20
[youtube:tab] Downloading page 21
[youtube:tab] Downloading page 22
[youtube:tab] Downloading page 23
[youtube:tab] Downloading page 24
[youtube:tab] Downloading page 25
[youtube:tab] Downloading page 26
[youtube:tab] Downloading page 27
[youtube:tab] Downloading page 28
[youtube:tab] Downloading page 29
[youtube:tab] Downloading page 30
[youtube:tab] Downloading page 31
[youtube:tab] Downloading page 32
[youtube:tab] Downloading page 33
[youtube:tab] Downloading page 34
[youtube:tab] Downloading page 35
[youtube:tab] Downloading page 36
[youtube:tab] Downloading page 37
[youtube:tab] Downloading page 38
[youtube:tab] Downloading page 39
[youtube:tab] Downloading page 40
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp6ovysl5t\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

aram535 · 2021-02-28T14:22:11Z

Interesting enough, my run this morning worked fine.
My guess is that it's either an A/B test by google or there was a sim-behaving service somewhere inline that is now working.

I'm going to leave the issue open, just in case others have not had the same resolution.

Evaia · 2021-02-28T15:03:42Z

Interesting enough, my run this morning worked fine.
My guess is that it's either an A/B test by google or there was a sim-behaving service somewhere inline that is now working.

I'm going to leave the issue open, just in case others have not had the same resolution.

Interesting, it seems to be intermittent for me. 9/10 times I got the same error but on the last time it worked. But trying to run it again immediately after and it exhibited the same error. A/B testing is a good theory, I'd be curious if it stays fixed for you.

* Currently only used by youtube Fixes ytdl-org/youtube-dl#28194 Possibly also fixes: ytdl-org/youtube-dl#28289 (can not confirm since the issue isn't reliably reproducible)

dogancelik · 2021-03-03T00:01:48Z

I'm using the latest EXE version 2021.03.02, same error with version 2021.02.22.

$ youtube-dl.exe https://www.youtube.com/c/EHPMusicChannelII/videos --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/c/EHPMusicChannelII/videos', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp65001, pref cp1252
[debug] youtube-dl version 2021.03.02
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg git-2020-01-02-81172b5, ffprobe git-2020-01-02-81172b5, phantomjs 2.5.0
[debug] Proxy map: {}
[youtube:tab] EHPMusicChannelII: Downloading webpage
[download] Downloading playlist: EHPMusicChannel - Videos
[youtube:tab] Downloading page 1
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp_qzp9er5\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp_qzp9er5\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

coletdjnz · 2021-03-03T00:25:00Z

~~If you add retries for the 404 in most causes it seems to eventually sort itself out, so I'm thinking this is probably server side.~~
Wonder if this has any similarity to #28194...
Edit: though right now there seems to be an influx in 404s, where I'm trying hundreds of retries while in the browser there's no problem.

coletdjnz · 2021-03-03T01:50:33Z

This might not be a server side issue, rather an API change (or YouTube doing A/B testing to remove the old API?)

I notice that youtube-dl is accessing https://www.youtube.com/browse_ajax?ctoken=XYZ for getting the next page, while YouTube in the browser is sending a POST request to https://www.youtube.com/youtubei/v1/browse?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 with the continuation token in the payload.

Testing with the latter in using the _call_api function in the YouTube extractor seems to work without 404s.

Unfortunately it has been hard to reproduce to debug this up until now.

ctrlcctrlv · 2021-03-03T01:51:43Z

It also works fine for me in the browser. I wrote a quick narrow script to move on in my case which isn't applicable to the project as a whole (just jQuery/bash hack)

Evaia · 2021-03-03T01:59:17Z

This might not be server side, rather an API change (or YouTube doing A/B testing to remove the old API?)

I notice that youtube-dl is accessing https://www.youtube.com/browse_ajax?ctoken=XYZ for getting the next page, while YouTube in the browser is sending a POST request to https://www.youtube.com/youtubei/v1/browse?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 with the continuation token in the payload.

Testing with the latter in using the _call_api function in the YouTube extractor seems to work without 404s.

Unfortunately it has been hard to reproduce to debug this up until now.

Confirmed that my copy was working fine a few hours ago, and now is 404 every time. Let me know if there's any debugging I can assist with.

VADemon · 2021-03-03T02:07:33Z

I can confirm, everything goes through this new front-end API endpoint now (JSON-based). Subscriptions, channel feed etc.

https://www.youtube.com/youtubei/v1/browse?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 (note the key is exactly the same as above)

Wiiplay123 · 2021-03-03T02:10:31Z

I got this issue too after updating to the latest version of youtube-dl, because the version I had been using would only download the first 30 videos of a channel.

JaKXz · 2021-03-03T07:24:04Z

Hi, sorry to comment on a closed issue, but I'm seeing a similar issue with recent updates (2021.03.02) as well:

verbose output

❯ mpv https://www.youtube.com/playlist\?list\=PLAbMhAYRuCUhawCEV2oXZGrienoKTN16X --ytdl-raw-options=verbose=true
[ytdl_hook] [debug] System config: []
[ytdl_hook] [debug] User config: []
[ytdl_hook] [debug] Custom config: []
[ytdl_hook] [debug] Command-line args: ['--no-warnings', '-J', '--flat-playlist', '--sub-format', 'ass/srt/best', '--format', 'bestvideo[height<=?1080][fps<=?60][vcodec!=?vp9]+bestaudio/best', '--verbose', 'true', '--all-subs', '--no-playlist', '--', 'https://www.youtube.com/playlist?list=PLAbMhAYRuCUhawCEV2oXZGrienoKTN16X']
[ytdl_hook] [debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[ytdl_hook] [debug] youtube-dl version 2021.03.02
[ytdl_hook] [debug] Python version 3.9.2 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[ytdl_hook] [debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[ytdl_hook] [debug] Proxy map: {}
[ytdl_hook] ERROR: 'true' is not a valid URL. Set --default-search "ytsearch" (or run  youtube-dl "ytsearch:true" ) to search YouTube
[ytdl_hook] Traceback (most recent call last):
[ytdl_hook]   File "/usr/local/Cellar/youtube-dl/2021.3.2/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
[ytdl_hook]     return func(self, *args, **kwargs)
[ytdl_hook]   File "/usr/local/Cellar/youtube-dl/2021.3.2/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 827, in __extract_info
[ytdl_hook]     ie_result = ie.extract(url)
[ytdl_hook]   File "/usr/local/Cellar/youtube-dl/2021.3.2/libexec/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 532, in extract
[ytdl_hook]     ie_result = self._real_extract(url)
[ytdl_hook]   File "/usr/local/Cellar/youtube-dl/2021.3.2/libexec/lib/python3.9/site-packages/youtube_dl/extractor/generic.py", line 2381, in _real_extract
[ytdl_hook]     raise ExtractorError(
[ytdl_hook] youtube_dl.utils.ExtractorError: 'true' is not a valid URL. Set --default-search "ytsearch" (or run  youtube-dl "ytsearch:true" ) to search YouTube
[ytdl_hook] youtube-dl failed: unexpected error occurred
Failed to recognize file format.

Exiting... (Errors when loading file)

Then I tried setting --default-search=ytsearch as suggested and got a little further but still seeing the same error:

❯ mpv https://www.youtube.com/playlist\?list\=PLAbMhAYRuCUhawCEV2oXZGrienoKTN16X --ytdl-raw-options=verbose=true,default-search=ytsearch

Playing: https://youtu.be/Al_Rm2jEOLU
[ytdl_hook] failed to parse JSON data: error
Failed to recognize file format.

Evaia · 2021-03-03T12:39:29Z

Unsure if this is an expected side-effect of the new browse API, it so that ytdl doesn't parse more than one page of a playlist; so you only get the first 100 videos of said playlist.

* https://github.com/ytdl-org/youtube-dl: release 2021.03.03 [ChangeLog] Actualize [ci skip] [youtube:tab] Switch continuation to browse API (closes ytdl-org#28289, closes ytdl-org#28327)

coletdjnz · 2021-03-03T19:54:08Z

Unsure if this is an expected side-effect of the new browse API, it so that ytdl doesn't parse more than one page of a playlist; so you only get the first 100 videos of said playlist.

Is this a private playlist? Switching to the new API broke continuation on private playlists which I don't think is fixed in youtube-dl yet (a SAPISIDHASH auth header is required). I havnt got around to reporting this yet.

Evaia · 2021-03-03T20:23:57Z

Unsure if this is an expected side-effect of the new browse API, it so that ytdl doesn't parse more than one page of a playlist; so you only get the first 100 videos of said playlist.

Is this a private playlist? Switching to the new API broke continuation on private playlists which I don't think is fixed in youtube-dl yet (a SAPISIDHASH auth header is required). I havnt got around to reporting this yet.

Affirmative, this is a private playlist (Watchlater)

closes ytdl-org#28327) Until further investigation.

* Currently only used by youtube Fixes ytdl-org/youtube-dl#28194 Possibly also fixes: ytdl-org/youtube-dl#28289 (can not confirm since the issue isn't reliably reproducible)

remitamine added the cant-reproduce label Feb 27, 2021

jdtech3 mentioned this issue Mar 3, 2021

[Bug] Visit any channel page: Unexpected token '' at line 1, column 1 iv-org/invidious#1838

Closed

coletdjnz mentioned this issue Mar 3, 2021

[YouTube] Use new browse API for continuation page extraction. yt-dlp/yt-dlp#131

Merged

9 tasks

dstftw closed this as completed in 061c030 Mar 3, 2021

XiangRongLin mentioned this issue Mar 3, 2021

Youtube channels and playlist continuations broken TeamNewPipe/NewPipeExtractor#566

Closed

leshasmlesha pushed a commit to leshasmlesha/youtube-dl that referenced this issue Mar 7, 2021

[youtube:tab] Switch continuation to browse API (closes ytdl-org#28289,

d46d4f1

closes ytdl-org#28327) Until further investigation.

leshasmlesha pushed a commit to leshasmlesha/youtube-dl that referenced this issue Mar 7, 2021

[youtube:tab] Switch continuation to browse API (closes ytdl-org#28289,

8c43984

closes ytdl-org#28327) Until further investigation.

dirkf pushed a commit to dirkf/youtube-dl that referenced this issue Mar 16, 2021

[youtube:tab] Switch continuation to browse API (closes ytdl-org#28289,

b9ef3e4

closes ytdl-org#28327) Until further investigation.

leshasmlesha pushed a commit to leshasmlesha/youtube-dl that referenced this issue Apr 3, 2021

[youtube:tab] Switch continuation to browse API (closes ytdl-org#28289,

a109dad

closes ytdl-org#28327) Until further investigation.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to download JSON metadata error on yt #28289

Unable to download JSON metadata error on yt #28289

aram535 commented Feb 27, 2021

mechalincoln commented Feb 27, 2021 •

edited

Loading

Evaia commented Feb 28, 2021 •

edited

Loading

coletdjnz commented Feb 28, 2021 •

edited

Loading

Evaia commented Feb 28, 2021

aram535 commented Feb 28, 2021

Evaia commented Feb 28, 2021

dogancelik commented Mar 3, 2021

coletdjnz commented Mar 3, 2021 •

edited

Loading

coletdjnz commented Mar 3, 2021 •

edited

Loading

ctrlcctrlv commented Mar 3, 2021

Evaia commented Mar 3, 2021

VADemon commented Mar 3, 2021

Wiiplay123 commented Mar 3, 2021

JaKXz commented Mar 3, 2021 •

edited

Loading

Evaia commented Mar 3, 2021

coletdjnz commented Mar 3, 2021 •

edited

Loading

Evaia commented Mar 3, 2021

Unable to download JSON metadata error on yt #28289

Unable to download JSON metadata error on yt #28289

Comments

aram535 commented Feb 27, 2021

Checklist

Verbose log

Description

mechalincoln commented Feb 27, 2021 • edited Loading

Evaia commented Feb 28, 2021 • edited Loading

coletdjnz commented Feb 28, 2021 • edited Loading

Evaia commented Feb 28, 2021

aram535 commented Feb 28, 2021

Evaia commented Feb 28, 2021

dogancelik commented Mar 3, 2021

coletdjnz commented Mar 3, 2021 • edited Loading

coletdjnz commented Mar 3, 2021 • edited Loading

ctrlcctrlv commented Mar 3, 2021

Evaia commented Mar 3, 2021

VADemon commented Mar 3, 2021

Wiiplay123 commented Mar 3, 2021

JaKXz commented Mar 3, 2021 • edited Loading

Evaia commented Mar 3, 2021

coletdjnz commented Mar 3, 2021 • edited Loading

Evaia commented Mar 3, 2021

mechalincoln commented Feb 27, 2021 •

edited

Loading

Evaia commented Feb 28, 2021 •

edited

Loading

coletdjnz commented Feb 28, 2021 •

edited

Loading

coletdjnz commented Mar 3, 2021 •

edited

Loading

coletdjnz commented Mar 3, 2021 •

edited

Loading

JaKXz commented Mar 3, 2021 •

edited

Loading

coletdjnz commented Mar 3, 2021 •

edited

Loading