Segmented downloads with multiple connections per file causes transfers to fail and exceed file size on server

[lpagani91]

Running Version 8.3.3 (37544), I tried to download a 39GB file on my computer. With a ~15 MB/s transfer rate, it indicated about 55 min. to download the file (accelarating, slowing down, etc...). I returned 4 hours later and it was still running with a file being 4.5 times the size of the original file

I killed it and tried again from home tonight. Same result. It reached 100% and went on downloading. I stopped it at 101%.
I am downloading it via good old ftp now but it is 3 times slower so only at 57% at the moment.
Is it a problem with cyberduck or with the file or both ?

[lpagani91]

The download with old ftp file was fine. It was only much longer. So the problem is with Cyberduck only :
150 Opening BINARY mode data connection for m9501_109.tar (39285760000 bytes).
100% |***********************************| 37465 MiB 5.46 MiB/s 00:00 ETA
226 Transfer complete.
39285760000 bytes received in 1:54:14 (5.46 MiB/s)

[arex388]

I can also confirm this on 8.3.3 when downloading from an S3 bucket. I was trying to download a 134.7GB file, and the download continued even though the estimated time had long since passed. After waiting for two hours just in case it was real, I stopped the download and it said that it had downloaded 184.2GB. It would appear that Cyberduck has an issue with large files. Perhaps its not "realizing" the file has finished and is repeating the ending segment(s)?

[networkneil]

I was having this issue too. I was trying to upload a folder of size 130GB and the copy speed starts of good and drops to almost zero and its failing constantly. Then I had to resort to AWS CLI.

[networkneil]

I was having this issue too. I was trying to upload a folder of size 130GB and the copy speed starts of good and drops to almost zero and its failing constantly. Then I had to resort to AWS CLI.

Edit:
sorry forgot to mention, I was doing an upload to s3 bucket.

[demodit]

Downloads of big files from FTP server by FTP-SSL do not work since months (earlier it did work). Now I encounter also infinite downloading.

[lpagani91]

You mean this is independent of Cyberduck and rooted in FTP-SSL ?

Le 22 mai 2022 à 12:23, demodit ***@***.*** ***@***.***>> a écrit : Downloads of big files from FTP server by FTP-SSL do not work since months (earlier it did work). Now I encounter also infinite downloading. — Reply to this email directly, view it on GitHub <#13374 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADJVTNFGSFTOEQST23QLLDLVLIDIVANCNFSM5VSZPZWA>. You are receiving this because you authored the thread.

"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" (devise Shadok)

[demodit]

I mean downloads by Cyberduck.

[spacepluk]

This is still happening

[krilupo]

Hi guys, issue still present - also on my end.
Trying to download a 4.9GB file which results in an endless loop ...
Please fix this.
Thanks

[LucasJanin]

Same issue on my side with version 8.4.4.
Switching temporarily on Filezilla

[dkocher]

@lpagani91 Try to uncheck Preferencers → Transfers → Downloads → Segmented downloads with multiple connections per file to reduce the load on the server.

[LucasJanin]

@dkocher Yes this fixes the infinite downloading!!
Thanks for the tips!
Hoping the problem gets resolved soon.

[krilupo]

Hi David, thanks a lot for the feedback and suggestion. I tried this. Unfortunately it does not work completely: it stops the endless download into one file, yes. However, now the behavior is that once the file is downloaded it wants to download it again and finds a file with the same filename and a dialog box pops up what to do with the file, e.g. OVERWRITE/SKIP/etc. I select SKIP (since it is already downloaded) and then I get in the transfers window the following: My settings are: Attached is additionally the Cyberduck log file from that download session (download of a Backup file yesterday 2022-10-10) Thanks, Christian

…

On 10. Oct 2022, at 17:48, David Kocher ***@***.***> wrote: @lpagani91 <https://github.com/lpagani91> Try to uncheck Preferencers → Transfers → Downloads → Segmented downloads with multiple connections per file to reduce the load on the server. — Reply to this email directly, view it on GitHub <#13374 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2B2VOYHZBAD5HA7LOISPWDWCQ3GVANCNFSM5VSZPZWA>. You are receiving this because you commented.

[LucasJanin]

FYI, the problem isn't present over SFTP

[HKdAlex]

I experience the same problem when downloading a folder from an S3 server.

[dkocher]

I experience the same problem when downloading a folder from an S3 server.

This should be resolved in the current snapshot build.