Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change auto retry time? #5652

Open
truth1ness opened this issue May 8, 2015 · 4 comments
Open

Change auto retry time? #5652

truth1ness opened this issue May 8, 2015 · 4 comments

Comments

@truth1ness
Copy link

@truth1ness truth1ness commented May 8, 2015

Is there any way to change the auto-retry wait time?

When I download very long playlists (250+, usually very short segments) I run into the "content too short" error which I'm almost certain is youtube's throttling after a certain number of videos. The auto-retry always seems to fail. I've noticed I can either wait 15 minutes or so or change my IP address via VPN. But since the retry fails I have to go back and manually figure out how far it got (when multiple playlists are queued), edit the url's, tell it what video number to start on, etc. It's very tedious.

I think this could be solved with a simple parameter for extending the retry time interval (maybe specifically for this error). It could retry once or twice, then wait x minutes between subsequent tries. This would allow queuing up of more videos without having to reset every time the throttle limit is hit. There are a number of other 'content too short' reports and I think they are all running into the same downloading limit.

PS. I have tried increasing the sleep interval (between each video) and that doesn't seem to have helped.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented May 8, 2015

If increasing sleep interval (to reasonably large value) did not help, your conclusions on throttling are obviously wrong. In any case imposing different kinds of heuristics is brittle and requires fine-grained testing.
Did not get it, why you need to edit some URLs or tell what video number to start instead of just running the same command again if you've noticed some error occured.

@truth1ness
Copy link
Author

@truth1ness truth1ness commented May 8, 2015

Ah, ok.

As for why, some courses like Udacity are broken up into many short videos and have 300+ videos per playlist. Say I put 2 of them in queue and it stops halfway through the second course. If I simply restart it will have to go through 450 videos again before getting to the first new one which takes a while (I'm not sure if it's pinging the server each time but if it is that probably doesn't help with the throttle).

I'll keep trying longer sleep times. The only problem is with that many small videos it adds a ton of time to the total download since it's getting multiplied hundreds of times.

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented May 9, 2015

For those cases you should use --download-archive some_file.txt, videos that are already downloaded will be skipped.

@PseudoPort
Copy link

@PseudoPort PseudoPort commented May 23, 2015

Any workaround to solve "ERROR: content too short" ? Need to download 1000+ videos. Still happen to me and keep manually press the command again. Retry option didn't help also.

Since youtube-dl automatically detects files already been downloaded, i finally created a bash script to auto retry the command until success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.