YouTube Download Randomly Freezes

[BullyWiiPlaza]

I'm programming in Java and call youtube-dl inside the application multiple times to download YouTube videos:

String downloadCommand = "youtube-dl.exe -u " + username + " -p " + password + " -c " + videoLink;
Process downloadProcess = Runtime.getRuntime().exec(downloadCommand);
// more code

The problem I'm having is that sometimes youtube-dl gets stuck and won't continue. Either right after the beginning of a download or at the end (usually on big files). The method Java method waitFor(long timeOut, TimeUnit unit) for process objects doesn't seem to timeout when youtube-dl gets stuck like this, just like trying to capture all console output won't bypass the deadlock.

The following are two outputs I got when youtube-dl got stuck:
ERROR: content too short (expected 28263044 bytes and served 16777836)
[download] 59.4% of 26.95MiB at 1.43MiB/s ETA 00:07
// no more output

ERROR: content too short (expected 60482506 bytes and served 39846308)
[download] 65.9% of 57.68MiB at 2.18MiB/s ETA 00:09
// no more output

I've been trying to bypass the freezing but I wasn't able to. I need youtube-dl to never ever get stuck and finish its job at all times. I do have a working and stable Internet connection and even if not, youtube-dl should never get stuck. You had ONE job, youtube-dl...

Please fix this issue. I need it to be 100% reliable at what it does otherwise I will have to drop this application for good.

[rrooij]

Can you give an example link for the videos that aren't being reliably downloaded? Also, do you run the latest version of youtube-dl?

[BullyWiiPlaza]

Yes, I am running the latest version. I did youtube-dl -U yesterday to update the application. It failed on various videos already but here's an example of a video that didn't reliably download:
https://www.youtube.com/watch?v=i5BxVDMXsiY

Note that it might not be reproducible since the failures are quite random but big videos usually are more critical. Maybe data downloads incorrectly by coincidence?

Thank you

[rrooij]

I downloaded your example 4 times without a single problem. Is your internet connection stable? What happens if you ping 8.8.8.8 for 3 minutes?

You can also try an external downloader like wget.

[BullyWiiPlaza]

I did ping for a few minutes and no packets were lost. Currently I'm downloading some more videos and no issues arose for a while but again, it shouldn't happen ever but it does. Something needs to be fixed in terms of error handling in youtube-dl though.

EDIT:
Just happened again with the "ERROR: content too short" message

[rrooij]

Hmm, then I guess it's not your network.

You can try

youtube-dl --external-downloader=[yourExternalDownloader]

In the mean time.

I've used youtube-dl a lot and never got this message. Does it also happen randomly when you're running youtube-dl directly (not via your Java application)? And what should change in the form of error handling?

There is another issue from a long time ago:
#829

youtube-dl should retry when this happens a few times in my opinion.

[dstftw]

ERROR: content too short usually means a serving problem on YouTube side. In this case youtube-dl prints this message to stderr and exits with non-zero error code immediately.
Most likely your code missbehave, however can't be sure without looking at the full listing. --verbose output would be helpful as well.

[BullyWiiPlaza]

I'm now trying the workaround with the batch file

if [%1]==[] echo URL missing & goto :EOF
:retry
youtube-dl.exe -u %1 -p %2 -c %3
if ERRORLEVEL 1 goto retry

But even if that works, it should be included in the application itself.

I assume the error comes up when one downloads a lot of videos only. It might be due to YouTube's highly-trained monkey page:
http://cdn.intertech.com/Blog/wp-content/uploads/2014/01/15_youtube-monkeys.jpg

youtube-dl should handle this properly.

[jaimeMF]

youtube-dl should handle this properly.

You should clarify what do you expect youtube-dl to do when it finds that error, just retry?

If you are passing a playlist url and want to continue with the next videos even if one of them fails, you can use the --ignore-errors option.

[BullyWiiPlaza]

Yeah, but I want all videos to be downloaded reliably and not continue until that is achieved. Kinda retrying infinitely. It seems like --ignore-errors skips videos which it shouldn't.

[jaimeMF]

I'm closing the issue as a duplicate of #829 then, thanks for the report.