Overheads - Protocol, Request, JSON, Page Download, total, percentages, etc.

[sirinath]

Is it possible to have more data traster when downloading.

[phihag]

Sorry, I have no idea what you mean here. Was is a traster? While we are thankful for every issue, if you are requesting a feature, please elaborate a little bit on

• In what scenario is it important?
• What precisely does the feature do?
• Why does it solve the problem?

[sirinath]

If you are on a connection which is billed for download MB after a threshold you have to be very sure how much data is been transferred before you see the bill. Also just to get a idea how much is useful transfers and now much is not.

Downloaded file size may not be representative of the internet usage. Also if you can stop and download the next month in your fee quota. In case you want do download a video on metered quota overheads / stats can help you idea of the costs involved.

[sirinath]

BTW, what is traster?

[sirinath]

Also this is very important when your ISP throttles your connection on average 5kbs once you have downloaded more than your quota. In this senario also the % of useful data vs overhead is important as information.

[download] Destination: Immutable Singly Linked List-A0HK9pFhvdY.mp4
[download] 26.7% of 113.93MiB at 1.61KiB/s ETA 14:42:26ERROR: unable to downlo
ad video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #78 of 142
[youtube] F9Ah27DVv88: Downloading video webpage
[youtube] F9Ah27DVv88: Downloading video info webpage
[youtube] F9Ah27DVv88: Extracting video information
[download] Downloading video #79 of 142
[youtube] CPuWwjBKPDI: Downloading video webpage
[youtube] CPuWwjBKPDI: Downloading video info webpage
[youtube] CPuWwjBKPDI: Extracting video information
[download] Resuming download at byte 44637243
[download] Destination: Linked List Based Queue (using Scala)-CPuWwjBKPDI.mp4
[download] 80.7% of 52.85MiB at 1.96KiB/s ETA 01:28:47ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #80 of 142
[youtube] _5ernqymnfs: Downloading video webpage
[youtube] _5ernqymnfs: Downloading video info webpage
[youtube] _5ernqymnfs: Extracting video information
[download] Resuming download at byte 1536
[download] Destination: Priority Queues (using Scala)-_5ernqymnfs.mp4
[download] 2.9% of 25.02MiB at 4.92KiB/s ETA 01:24:13ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #81 of 142
[youtube] GD1sh4MYArE: Downloading video webpage
[youtube] GD1sh4MYArE: Downloading video info webpage
[youtube] GD1sh4MYArE: Extracting video information
[download] Resuming download at byte 730447
[download] Destination: Sorted Linked List Priority Queue (using Scala)-GD1sh4MY
ArE.mp4
[download] 4.7% of 45.34MiB at 4.03KiB/s ETA 03:02:48ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #82 of 142
[youtube] ubxYowsfqsw: Downloading video webpage
[youtube] ubxYowsfqsw: Downloading video info webpage
[youtube] ubxYowsfqsw: Extracting video information
[download] Resuming download at byte 7087
[download] Destination: Testing the Sorted Linked List Priority Queue (using Sca
la)-ubxYowsfqsw.mp4
[download] 3.4% of 40.90MiB at 4.98KiB/s ETA 02:15:24ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #83 of 142
[youtube] DHPEjpZDNoQ: Downloading video webpage
[youtube] DHPEjpZDNoQ: Downloading video info webpage
[youtube] DHPEjpZDNoQ: Extracting video information
[download] Resuming download at byte 9216
[download] Destination: Refactoring - Smells (using Scala)-DHPEjpZDNoQ.mp4
[download] 0.0% of 71.22MiB at 404.00B/s ETA 51:20:11ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[download] Downloading video #84 of 142
[youtube] B8565qOmnlg: Downloading video webpage
[youtube] B8565qOmnlg: Downloading video info webpage
[youtube] B8565qOmnlg: Extracting video information
[download] Destination: Refactoring - Smells 2 (using Scala)-B8565qOmnlg.mp4
[download] 2.6% of 50.98MiB at 4.73KiB/s ETA 02:59:23ERROR: unable to downloa
d video data: timed out
Traceback (most recent call last):
File "youtube_dl\YoutubeDL.pyo", line 776, in process_info
File "youtube_dl\FileDownloader.pyo", line 593, in _do_download
File "socket.pyo", line 380, in read
File "httplib.pyo", line 567, in read
File "socket.pyo", line 380, in read
timeout: timed out

[phihag]

I have no idea what a traster is, you mentioned it in your first post, that's why I asked.

Unfortunately, we cannot feasibly determine in advance how large the associated data transfers are. However, they pale in comparison to any high-quality video.

Since youtube-dl does not run with superuser rights we can also not determine the overhead of the TCP/IP headers. I'd estimate that these add less than 20% overhead.

I do not know how the logs you posted relate to the overhead. They are evident of a very poor connection. In youtube-dl 2013.12.1 (which will be released today), we increase the timeout so that should not happen.

[sirinath]

I ment transfer.

[sirinath]

Will check the new release out.