Split Retry to Retry to Access, Retry After Partial Download and Retury when Data received.

[sirinath]

The retry option can be split to 3.

1. retry to access page and start download. (Keep current retry option for this)
2. retry a part download until completed. (Introduce a new option. This can default to inf retries as if some data is received.)
3. retries if no data received. (# retries if no data received in the previous try. Perhaps this also can default to 10.)

[phihag]

I think I do understand your suggestion, but I don't see any rationale. Why should we split the option in three? I'm all for making sure that retry always works (there were some reports of an error there), but as a user, I'd certainly be confused by three different retry options.

[sirinath]

If some data was received I do not want it to give up when the retry number is reached.

[sirinath]

Also would be helpful to find a way to overcome time out errors if you had finer control on what cases to re try on:

[youtube] PW1ylyeX364: Downloading video webpage
ERROR: Unable to download video webpage: <urlopen error [Errno 10054] An existin
g connection was forcibly closed by the remote host>
Traceback (most recent call last):
File "youtube_dl\extractor\youtube.pyo", line 1265, in _real_extract
File "urllib2.pyo", line 127, in urlopen
File "urllib2.pyo", line 404, in open
File "urllib2.pyo", line 422, in _open
File "urllib2.pyo", line 382, in _call_chain
File "youtube_dl\utils.pyo", line 561, in https_open
File "urllib2.pyo", line 1184, in do_open
URLError: <urlopen error [Errno 10054] An existing connection was forcibly close
d by the remote host>
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 428, in extract_info
File "youtube_dl\extractor\common.pyo", line 131, in extract
File "youtube_dl\extractor\youtube.pyo", line 1267, in _real_extract
ExtractorError: Unable to download video webpage: <urlopen error [Errno 10054] A
n existing connection was forcibly closed by the remote host>

[download] Downloading video #70 of 142
[youtube] LSJnX7Fi1T4: Downloading video webpage
[youtube] LSJnX7Fi1T4: Downloading video info webpage
ERROR: The read operation timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 428, in extract_info
File "youtube_dl\extractor\common.pyo", line 131, in extract
File "youtube_dl\extractor\youtube.pyo", line 1305, in _real_extract
File "youtube_dl\extractor\common.pyo", line 211, in _download_webpage
File "youtube_dl\extractor\common.pyo", line 176, in _download_webpage_handle
File "socket.pyo", line 351, in read
File "httplib.pyo", line 543, in read
File "httplib.pyo", line 612, in _read_chunked
File "httplib.pyo", line 658, in _safe_read
File "socket.pyo", line 380, in read
File "ssl.pyo", line 241, in recv
File "ssl.pyo", line 160, in read
SSLError: The read operation timed out

[download] Downloading video #71 of 142
[youtube] NcobAzffgvg: Downloading video webpage
[youtube] NcobAzffgvg: Downloading video info webpage
ERROR: The read operation timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 428, in extract_info
File "youtube_dl\extractor\common.pyo", line 131, in extract
File "youtube_dl\extractor\youtube.pyo", line 1305, in _real_extract
File "youtube_dl\extractor\common.pyo", line 211, in _download_webpage
File "youtube_dl\extractor\common.pyo", line 176, in _download_webpage_handle
File "socket.pyo", line 351, in read
File "httplib.pyo", line 543, in read
File "httplib.pyo", line 603, in _read_chunked
File "httplib.pyo", line 658, in _safe_read
File "socket.pyo", line 380, in read
File "ssl.pyo", line 241, in recv
File "ssl.pyo", line 160, in read
SSLError: The read operation timed out

....
....
....

[download] 2.9% of 25.02MiB at 4.92KiB/s ETA 01:24:13ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[sehrgut]

#4240 is specific to the SSLError part of this. While I agree they shouldn't be split out, possible exceptions should be treated specifically enough to retry after all transient errors. Because SSLError subclasses socket.error, they are being treated as socket errors (i.e. only connection resets are treated as recoverable), while there are many recoverable SSLError.