Retry does not seem active on read timeout

[masklinn]

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.12.23. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2017.12.23

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

Report

I am currently on holiday in a place with debatable and variable internet quality: nominally 1024k (~120kB/s effective) with ~50ms ping best case

However the ping can randomly shoot up to the second range, and packet loss can reach 10% in these situations (when the connection doesn't go down entirely) with effective bandwidth being similarly affected.

A more typical scenario is ~80kB/s effective with lag spikes into the 300ms range (connection is currently as perfect as I usually see it and running ping for 150 packets yields min 45.604, avg 55.589, max 330.341, stdev 37.513)

As a result watching videos online is an impossibility and I rely on youtube-dl to get the videos offline and watch uninterrupted.

However the "retry" option does not seem to affect Read Timeout errors, which happen any time the ping shoots up a bit too high or the connection falls down entirely (even if it reconnects by the time the timeout actually triggers, the original read will never complete).

Verbose Log

These are two runs (back to back):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-R', u'infinite', u'https://www.youtube.com/watch?v=bAVS6IlujZE']
[debug] Encodings: locale utf-8, fs utf-8, out utf-8, pref utf-8
[debug] youtube-dl version 2017.12.23
[debug] Python version 2.7.14 - Darwin-15.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.4.1, ffprobe 3.4.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] bAVS6IlujZE: Downloading webpage
[youtube] bAVS6IlujZE: Downloading video info webpage
[youtube] bAVS6IlujZE: Extracting video information
[youtube] bAVS6IlujZE: Downloading MPD manifest
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on u'https://r4---sn-4g5ednsk.googlevideo.com/videoplayback?id=6c0552e8896e8d91&itag=298&source=youtube&requiressl=yes&pl=16&mm=31&ei=3OBCWvinHYS9VImqvpgP&initcwndbps=392500&mn=sn-4g5ednsk&ms=au&mv=m&ratebypass=yes&mime=video/mp4&gir=yes&clen=6156947069&lmt=1513885734203450&dur=32046.048&mt=1514332278&key=dg_yt0&signature=7CC8C5B6640B3BB3BE9DE36255423EA1D7B224AD.6DC7F6A546407547833CB9F1E11FE3E34391197C&ip=109.221.254.140&ipbits=0&expire=1514353980&sparams=ip,ipbits,expire,id,itag,source,requiressl,pl,mm,ei,initcwndbps,mn,ms,mv,ratebypass,mime,gir,clen,lmt,dur'
[download] Resuming download at byte 143191710
[download] Destination: Tastosis, iNcontroL, & Day[9] host Day 1 of the Twitch StarCraft Holiday Bash-bAVS6IlujZE.f298.mp4
[download]   2.8% of 5.73GiB at 97.32KiB/s ETA 16:41:18ERROR: unable to download video data: ('The read operation timed out',)
Traceback (most recent call last):
  File "/opt/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1889, in process_info
    partial_success = dl(fname, new_info)
  File "/opt/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1834, in dl
    return fd.download(name, info)
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 361, in download
    return self.real_download(filename, info_dict)
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 287, in real_download
    return download()
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 180, in download
    data_block = ctx.data.read(block_size if not is_test else min(block_size, data_len - byte_counter))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 597, in read
    s = self.fp.read(amt)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 766, in recv
    return self.read(buflen)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 653, in read
    v = self._sslobj.read(len)
SSLError: ('The read operation timed out',)

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-R', u'infinite', u'https://www.youtube.com/watch?v=bAVS6IlujZE']
[debug] Encodings: locale utf-8, fs utf-8, out utf-8, pref utf-8
[debug] youtube-dl version 2017.12.23
[debug] Python version 2.7.14 - Darwin-15.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.4.1, ffprobe 3.4.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] bAVS6IlujZE: Downloading webpage
[youtube] bAVS6IlujZE: Downloading video info webpage
[youtube] bAVS6IlujZE: Extracting video information
[youtube] bAVS6IlujZE: Downloading MPD manifest
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on u'https://r4---sn-4g5ednsk.googlevideo.com/videoplayback?id=6c0552e8896e8d91&itag=298&source=youtube&requiressl=yes&mv=m&mn=sn-4g5ednsk&mm=31&initcwndbps=398750&ms=au&ei=UuRCWsyUAoiwctPhuugD&pl=16&ratebypass=yes&mime=video/mp4&gir=yes&clen=6156947069&lmt=1513885734203450&dur=32046.048&key=dg_yt0&signature=1314D2559FCF006A2C124D277BAC2B067ED8A643.51E83F8C277A34C4237096EE6D9F3163E89F029E&mt=1514333185&ip=109.221.254.140&ipbits=0&expire=1514354866&sparams=ip,ipbits,expire,id,itag,source,requiressl,mv,mn,mm,initcwndbps,ms,ei,pl,ratebypass,mime,gir,clen,lmt,dur'
[download] Resuming download at byte 169922912
[download] Destination: Tastosis, iNcontroL, & Day[9] host Day 1 of the Twitch StarCraft Holiday Bash-bAVS6IlujZE.f298.mp4
[download]   3.6% of 5.73GiB at 39.71KiB/s ETA 40:34:00ERROR: unable to download video data: ('The read operation timed out',)
Traceback (most recent call last):
  File "/opt/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1889, in process_info
    partial_success = dl(fname, new_info)
  File "/opt/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1834, in dl
    return fd.download(name, info)
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 361, in download
    return self.real_download(filename, info_dict)
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 287, in real_download
    return download()
  File "/opt/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 180, in download
    data_block = ctx.data.read(block_size if not is_test else min(block_size, data_len - byte_counter))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 597, in read
    s = self.fp.read(amt)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 766, in recv
    return self.read(buflen)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 653, in read
    v = self._sslobj.read(len)
SSLError: ('The read operation timed out',)

Mitigation

I currently have a small script just running ydl in a loop until it succeeds, but a working Retry option would definitely be superior.

[Saxarok]

I'm facing the same issue. It's insanely annoying.