Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCP transfer - attempts to resume, even though it technically can't - results in file with duplicate portions of resumed data #1513

Closed
cyberduck opened this issue Dec 22, 2007 · 1 comment

Comments

@cyberduck
Copy link
Collaborator

@cyberduck cyberduck commented Dec 22, 2007

4b7ae70 created the issue

Transferring 3 large files with "scp" selected as the SFTP transfer method, I stopped each transfer around 30 megs so I could change my network connection to a more reliable wired connection.

When the files go to the point they were supposed to be done, the transfer continued past the maximum file size, and the ETA time started going negative.

Opening the files in a binary file editor, it was clear that what happened was instead of "resuming" the file, the entire file was appended to the existing file that was downloaded.

I don't believe SCP is capable of doing resumes, so I was surprised when CyberDuck gave me that option - and I thought "hey, cool, CyberDuck has some magic way of resuming scp transfers... might as well try it"

With a hex editor, it was trivial to remove the portion that was duplicated (search for the first 1024 or 2048 bits which in the split bzip2 file are pretty much guaranteed to be unique and find the offset they are repeated at - then delete everything prior to that offset and save it...)

When using the SCP transfer method, it should either warn you that you wont be able to resume (put this in the preferences) - and when you try to download an existing file, it should not present you with the option to resume it.

If SOME scp transfers are capable of being resumed - you could implement a method to check it - where it will download the first few Kb into a buffer and you could compare that with the first few Kb of the portion to resume to check that it is, in fact, resuming - then continue on - or delete the existing portion and redownload the whole file.

I'm not an expert on the details of SCP, so I don't know if it is ever capable of being resumed or not.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Dec 23, 2007

@dkocher commented

I suppose you selected Resume in the overwrite prompt dialog. This is now disabled for SCP transfers as of 8dde79e. Before only the Resume option in the transfer window toolbar was disabled.

Loading

@cyberduck cyberduck closed this Dec 23, 2007
@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants