Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[youtube] Check for incomplete data in watch initial data #6510

Merged
merged 1 commit into from Mar 13, 2023

Conversation

coletdjnz
Copy link
Member

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Currently observing YouTube giving incomplete data for this initial data occasionally, leading to missing metadata (and non-UTC upload date in some situations).

Added functionality so it falls back to the API in this case. In YoutubeTab we re-download the webpage in this case, but since the webpage in YoutubeIE is downloaded as part of the player response process it is probably easier to do it this way.

yt-dlp --print upload_date https://www.youtube.com/watch\?v\=tjjjtzRLHvA -v --proxy ''
[debug] Command-line config: ['--print', 'upload_date', 'https://www.youtube.com/watch?v=tjjjtzRLHvA', '-v', '--proxy', '']
[debug] User config "/home/coletdjnz/.config/yt-dlp.conf": ['--ffmpeg-location', '/opt/yt-dlp-ffmpeg']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: c0de21bf7
[debug] Python 3.11.0rc2 (CPython x86_64 64bit) - Linux-5.19.17-2-MANJARO-x86_64-with-glibc2.37 (OpenSSL 1.1.1t  7 Feb 2023, glibc 2.37)
[debug] exe versions: ffmpeg N-107674-gf0abd07996-20220805 (setts), ffprobe N-107674-gf0abd07996-20220805, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Plugin directories: ['/home/coletdjnz/.pyenv/versions/3.11.0rc2/envs/venv311rc2-ytdlp/lib/python3.11/site-packages/yt_dlp_plugins']
[debug] Loaded 1788 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=tjjjtzRLHvA
[youtube] tjjjtzRLHvA: Downloading webpage
[youtube] tjjjtzRLHvA: Downloading android player API JSON
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
[youtube] tjjjtzRLHvA: Downloading initial data API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] tjjjtzRLHvA: Downloading 1 format(s): 247+251
20220323

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@coletdjnz coletdjnz added the site-bug Issue with a specific website label Mar 11, 2023
@coletdjnz coletdjnz merged commit 607510b into yt-dlp:master Mar 13, 2023
11 checks passed
@coletdjnz coletdjnz deleted the fix/upload_date_utc_m11 branch March 13, 2023 01:43
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant