YouTube channel pages have infinite scrolling page count. #5555
Comments
|
This is a bug on Youtube's side and will likely fix itself in a few days. In the mean time, you can use |
pukkandan
added
external issue
|
Thank you. Two things however. I don't believe Secondly, I doubt this will fix itself suddenly "in a few days". I only stumbled onto it for the first time, and this may be a very old and persistent issue that has gone unreported. Might still be worth looking at detecting infinite page dynamic loading scroll on YouTube. |
Did you try?
Even if the issue does not fix itself, the bug is still on Youtube's side. Similar issues have been observed in the past. We cannot have a simple duplicate detection since playlists may genuinely have repeated entries. Workarounds are possible (e.g. check if we are in playlist tab of a channel and encounters duplicate entry), but will be tedious to implement. Plus, the workaround will likely only solve this specific issue and similar bugs in Youtube API will require even more workarounds @coletdjnz Since similar issues have now been observed multiple times, do u have any idea if we can somehow detect this reliably? |
If the continuations are the same, we could maybe keep a cache of ones we have seen in a given context and break based on that. But I have a feeling all of them are unique for tabs. |
|
The issue still exists. We may want to try your suggestion |
coletdjnz
changed the title
|
This seems to be an intentional change rather than a backend issue, and has started rolling out to more lately. Unlike the comments loop issue, the observed behaviour is this seems to start looping at the known end of feed/channel page rather than somewhere in the middle. The above suggestion I made looks like it'll work for channel pages at least. Not sure if other feeds are impacted. |
|
Has this gone away now? Or should yt-dl be acquiring some variant of #6621? |
Have not seen or heard of any reports recently. Must have been a short-lived bug on YouTube's side. |
The infinite scrolling is still goofy on some |
|
Encountered this bug today on this channel: https://www.youtube.com/channel/UCaqOj5uQl-733nBtBpEEGBw/playlists |
|
and a ton of others: Basically any auto-generated music channel that doesn't have a /release tab has this issue form what I've seen |
|
Is there a work-around to limit the number of pages loaded in the initial scan? |
|
|
|
I think you'd also need |
|
Actually I think you'd still hit infinite scrolling unless you ctrl+c: Since it skips already downloaded playlists, it wont be able to hit the ExistingVideoReached exception |
Closes #5555 Note: the first page may still be repeated, however this is better than nothing. Authored by: coletdjnz
Closes yt-dlp#5555 Note: the first page may still be repeated, however this is better than nothing. Authored by: coletdjnz
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
Checklist
Region
United States
Provide a description that is worded well enough to be understood
YouTube has some automatically generated channels and playlists for musicians via a partner named TuneCore. The playlist page for these channels is infinite scrolling (as of today). This causes yt-dlp to grab infinite number of playlist pages. There is no option to tell yt-dlp to stop grabbing playlist pages upon repetition.
yt-dlp will need to add playlist repetition detection to abort continued grabbing of playlist pages.
Example:
https://www.youtube.com/channel/UCH0_ywht5OYCdRYWZUjBQwQ/playlists
This channel and all its playlists and videos are auto-generated by YouTube via partner TuneCore.
Provide verbose output that clearly demonstrates the problem
yt-dlp -vU <your command line>)[debug] Command-line config) and insert it belowComplete Verbose Output
The text was updated successfully, but these errors were encountered: