Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
How doest it work? I need minimum network and cpu activity for every day incremental scraping #6227
Comments
|
The info for videos in the archive won't be extracted but currently it gets all the video ids from the playlist, which may take some time depending on its size (it use to need to download 1 webpage every 50 videos or so, I don't know if it has changed too much). Note, I'd recommend you to use |
Hi, i have a question.
For example, how does ytdl work with these arguments?
I want to scrap every day only newest( actualy i dont download videos, i just need three atributes: name, lenght and id of youtube videos) videos from specific youtube playlist.
For first time it would take a while, i now, but other day, at the incremental scrap if i have a blacklist, (ids of already downloaded/saved videos) would it scrap it faster i guess, or may i accelerate this task?
I want to do it with ~50 youtube channels/playlists every day. So it would be great if it would take minimum network and cpu activity.
Thanks