Youtube-dl stops downloading halfway

[Ricksam90]

I'm currently trying to download all my soundcloud liked songs via youtube-dl, but whenever I try to do this the download stops halfway:

download] Downloading video 43 of 115
[soundcloud] withcomplimentsradio/with035: Resolving id
[soundcloud] withcomplimentsradio/with035: Downloading info JSON
[soundcloud] 227304872: Downloading track url
[soundcloud] 227304872: Checking fallback video format URL
[soundcloud] 227304872: Checking http_mp3_128_url video format URL
[download] Destination: With Compliments 035 by Franz Alice Stern-227304872.mp3
[download] 1.8% of 55.26MiB at 2.15MiB/s ETA 00:25ERROR: unable to download video data: ('The read operation timed out',)

How can I solve this issue? Thanks in advance

[dstftw]

Post the full output of youtube-dl when run with -v, i.e. add -v flag to your command line, copy the whole output and post it in the issue body wrapped in ``` for better formatting. It should look similar to this:

$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2015.12.06
[debug] Git HEAD: 135392e
[debug] Python version 2.6.6 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...

Do not post screenshots of verbose log only plain text is acceptable.

The output (including the first lines) contains important debugging information. Issues without the full output are often not reproducible and therefore do not get solved in short order, if ever.

[Arcrammer]

This also happens if you have a really bad ISP. You should be able to use the same command in the same directory and it will pick up where it left off, too. That means you won't lose 50% of the download if it's already been downloaded.

[vxbinaca]

To mitigate this, try --continue and jut keep running it over and over.

[k6ef]

I've been having it fail regularly over the past 3 days.. Here is my output..

Comcast, Bay Area, 86Mb/s/5Mb/s

mark:~/ $ youtube-dl -v https://www.youtube.com/watch\?v\=wq9Gu5N42HQ --continue [20:09:03]
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=wq9Gu5N42HQ', u'--continue']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.02.22
[debug] Python version 2.7.10 - Darwin-15.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.0, ffprobe 3.0
[debug] Proxy map: {}
[youtube] wq9Gu5N42HQ: Downloading webpage
[youtube] wq9Gu5N42HQ: Downloading video info webpage
[youtube] wq9Gu5N42HQ: Extracting video information
[youtube] wq9Gu5N42HQ: Downloading MPD manifest
[debug] Invoking downloader on u'https://r12---sn-n4v7kn7z.googlevideo.com/videoplayback?id=c2af46bb9378d874&itag=135&source=youtube&requiressl=yes&ms=au&mv=m&nh=IgpwcjAyLnBhbzAzKgkxMjcuMC4wLjE&pl=16&mn=sn-n4v7kn7z&mm=31&gcr=us&ratebypass=yes&mime=video/mp4&gir=yes&clen=222130351&lmt=1404500937148405&dur=2139.537&fexp=9416126,9420452,9422596,9423661,9423662&signature=31B0957F6DCDA1B00B7FDDFB5EE492E5E31ECF44.7CD0944C6B0D199086D57E91A715349C75DC110D&mt=1456459649&upn=BEEUNylMktQ&sver=3&key=dg_yt0&ip=67.180.239.140&ipbits=0&expire=1456481351&sparams=ip,ipbits,expire,id,itag,source,requiressl,ms,mv,nh,pl,mn,mm,gcr,ratebypass,mime,gir,clen,lmt,dur'
[download] Resuming download at byte 1570816
ERROR: unable to download video data: <urlopen error [Errno 54] Connection reset by peer>
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1618, in process_info
partial_success = dl(fname, new_info)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1566, in dl
return fd.download(name, info)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 343, in download
return self.real_download(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 58, in real_download
data = self.ydl.urlopen(request)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1905, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(_args)
File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 831, in https_open
req, *_kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 54] Connection reset by peer>

[weedy]

youtube-dl obviously has a retry mechanism.

[download]  79.2% of ~9.08GiB at  1.68MiB/s ETA 03:12[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 10 of 10)...
[download] Skipping fragment Frag6952...

It just doesn't catch everything

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 734, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 621, in read
    v = self._sslobj.read(len or 1024)
SSLError: ('The read operation timed out',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
URLError: <urlopen error timed out>

[yan12125]

@weedy Could you try this patch:

$ git diff youtube_dl/downloader/hls.py 
diff --git a/youtube_dl/downloader/hls.py b/youtube_dl/downloader/hls.py
index 7373ec0..56416c3 100644
--- a/youtube_dl/downloader/hls.py
+++ b/youtube_dl/downloader/hls.py
@@ -123,7 +123,7 @@ class HlsFD(FragmentFD):
                             frag_content = down.read()
                             down.close()
                             break
-                        except compat_urllib_error.HTTPError as err:
+                        except (compat_urllib_error.HTTPError, compat_urllib_error.URLError) as err:
                             # Unavailable (possibly temporary) fragments may be served.
                             # First we try to retry then either skip or abort.
                             # See https://github.com/rg3/youtube-dl/issues/10165,

And by the way, don't post the same message on so many issues

[ssokolow]

I have a very old DSL modem that I'm gearing up to replace and I'm getting this problem whenever the connection drops and reconnects, even when I specify a custom --retries to be sure the default of 10 isn't a lie.

Here's an example of what a typical failure looks like (dl is an alias for youtube-dl --prefer-free-formats):

ssokolow@monolith Videos % (cd ~/incoming/Videos/Gaming/;dl https://www.youtube.com/watch?v=sdtehRUfzWc)
[youtube] sdtehRUfzWc: Downloading webpage
[youtube] sdtehRUfzWc: Downloading video info webpage
[youtube] sdtehRUfzWc: Extracting video information
[youtube] sdtehRUfzWc: Downloading MPD manifest
WARNING: Your copy of avconv is outdated and unable to properly mux separate video and audio files, youtube-dl will download single file media. Update avconv to version 10-0 or newer to fix this.
[download] Destination: cat planet-sdtehRUfzWc.webm
[download]  74.5% of 24.17MiB at 229.91KiB/s ETA 00:27ERROR: unable to download video data: ('The read operation timed out',)
ssokolow@monolith Videos %

According to youtube-dl --version, I'm currently on 2017.05.01 but the problem hasn't visibly changed over the last year or so. (The avconv error is because I'm on *buntu 14.04 LTS)

I've asked youtube-dl to update to 2017.05.23 so that, next time my connection hiccups while i'm downloading, I can give a more up-to-date report.

My workaround (when I remember to use it) is the same kind of shell script wrapper "retry until youtube-dl exits cleanly" solution given in issue #4240.