Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Limited to 36 videos when downloading a pornhub playlist #22824
Labels
Comments
|
This is a new issue for sure. It is only seeing the first 36 in the playlist even with --playlist-reverse. |
|
Updated debug for v. 2019.10.29 for this issue |
|
Ah, i wasnt aware people kept, up, yes, it's limited, you can page=2 in the url and then it works. Also, the reporting/caching takes awhile before the client picks it up. There was a new build today? I reported in another thread, but i get a regex error now, I did the upgrade only after it died mid job. Ah, scratch, i d/ld 11.5's and test a few ways after, the site behavior changed. |
|
Oh, iirc, Went i went a looking there were only 36's id a page, which, definitely changed, because I'd download in reverse and the 500 limit per playlist they had made me have to trim it. The regex part I didn't see until today, I had just changed things to groups under 36's, because, I noticed the user interface for playlists in places has also changed from a forever scrolling screen to the annoying main view limit. It hasnt done that view page changing to my playlists though (from my default view,) , but they way the layout changes has..same timeframe the 36 limit and if you use page=1,2,3,4,5 the view is cutup exactly that way in the playlist, 36 per page.. |
|
pornhub changed the way the pagination works for playlists. There may be a workaround. The number of videos in a playlist is stored in the PLAYLIST_VIEW JSON data. That way it should be possible to calculate the number of pages that should be downloaded. |
|
I did try setting page=?(last one) and doing reverse as normal, but it tagging the video id's seems to have a issue right before it has throw error/logging catch and where I could set and manually have it work backwards, that doesn't seem consistent again. I figure I'd wait cause it seems like a moving target. |
|
I found that you can access next videos via |
|
Thanks to @Mik-chan's findings I made a little bash script to download a playlist automatically. Made in 2 minutes and definitely not pretty but it works. |
|
I'd be very careful running parallel scrapers / downloaders against PH. ytdl itself already trips into 403 pages and other blocks pretty readily, let alone trying to run 8 at a time. If your connection is slow enough you might get lucky though... As an aside, |
|
@thisisbobsalmon nice script! Though It would be great if someone added this functionality into extractor itself. |
|
How to use the bash script to work around the limitation? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Checklist
Question
So I'm trying to download a playlist that has ~400 videos on it, however I'm only able to download 36 videos at a time. I'm not sure if it's due to the fact that Pornhub doesn't load up the full playlist, and only loads parts of the playlist as you scroll down, or if maybe I'm just entering in the wrong commands. Any way to make it scrape all the videos instead of the 36 I'm currently limited to?