Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[youtube:tab] Detect looping feeds #6621

Merged
merged 2 commits into from Jul 15, 2023

Conversation

coletdjnz
Copy link
Member

@coletdjnz coletdjnz commented Mar 24, 2023

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

ADD DESCRIPTION HERE

Fixes #5555

YouTube is rolling out a change where they loop (channel?) feeds at the end. Fortunately the continuations appear to be identical, so we can detect when it loops using those.

#5555 (comment)

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

馃 Generated by Copilot at 4e53d87

Summary

馃悰馃摵馃攢

Fix infinite loop bug in YouTube feed extraction. Use a set to track seen continuation tokens in yt_dlp/extractor/youtube.py.

set stops the loop
YouTube feed pages bug fixed
autumn leaves falling

Walkthrough

  • Prevent infinite looping when fetching YouTube feed pages by using a set to store seen continuation tokens and breaking the loop if the same token is encountered again (link)

@coletdjnz coletdjnz added high-priority site-bug Issue with a specific website labels Mar 24, 2023
@coletdjnz
Copy link
Member Author

issue: first page is still repeated due to no continuation

@jmbannon
Copy link

jmbannon commented Jul 9, 2023

@coletdjnz @pukkandan 馃憢 friendly bump - Can we get this merged since it's still a valid bug with no workarounds on the the following URLs?
https://www.youtube.com/channel/UCaqOj5uQl-733nBtBpEEGBw/playlists
https://www.youtube.com/channel/UCMki-b0zfAQiMQ0nbsrIuBQ/playlists
https://www.youtube.com/channel/UCJdP0RSsGpcdI8S_aH5Y7yQ/playlists

(any auto-generated music channel that doesn't have a /releases tab)

@pukkandan
Copy link
Member

issue: first page is still repeated due to no continuation

@coletdjnz Is this the only issue? If so, we can merge it imo. It's better than nothing

@coletdjnz coletdjnz merged commit 1ba6fe9 into yt-dlp:master Jul 15, 2023
12 checks passed
@coletdjnz coletdjnz deleted the fix/yt-infinite-channel branch July 15, 2023 03:20
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Closes yt-dlp#5555

Note: the first page may still be repeated, however this is better than nothing.

Authored by: coletdjnz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

YouTube channel pages have infinite scrolling page count.
3 participants