Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Files with same name are concatenated, not over-written #10861

Closed
cyberduck opened this issue Nov 1, 2019 · 4 comments
Closed

Files with same name are concatenated, not over-written #10861

cyberduck opened this issue Nov 1, 2019 · 4 comments

Comments

@cyberduck
Copy link
Collaborator

@cyberduck cyberduck commented Nov 1, 2019

d2f945c created the issue

On MacOS 10.13.6, using Cyberduck Version 6.7.0 (28613), when I upload a file with the same name via SFTP to our SFTP server, the uploaded file is concatenated onto the existing file, it does not overwrite it.

The 'Existing Files' setting on the 'General' tab of the 'Transfers' preferences page is set to 'Overwrite'.

For some files types of course, this breaks the file. I have tested this with Excel (files get corrupted), txt (files are concatenated and legible with the content of the existing and new file both visible), gzip archives (the archive is concatenated and this survives gzip decompression so that the resulting decompressed file is also concatenated - pretty magical!).

Attached video shows this - you can see that each time I drag and drop a file into the Cyberduck browser window, the size of the remote file increases by the size of the uploaded file.


Attachments

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Nov 3, 2019

@dkocher commented

I cannot reproduce this against OpenSSH_7.4. What server software are you running?

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Nov 4, 2019

d2f945c commented

Okay - so if login via command-line sftp from the same computer I'm running Cyberduck on, and use verbose, I get:

"OpenSSH_7.8p1, LibreSSL 2.6.2"

Once logged in to the SFTP server in question, if I type 'version' I get:

"SFTP protocol version 3"

I've emailed the operators of the SFTP server to find out more details of the software they're running. I could create a test account on that server for you if that would help? If so - please tell me how to securely send you account details.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Nov 4, 2019

d2f945c commented

Further to this - logging in with a command-line client and issuing a 'put' to the server in question over-writes an existing file. Using Cyberduck, concatenates the file. I've spoken now with the server provider, and they've checked the logs and said that Cyberduck is opening the file for writing without deleting an existing file with the same name in the directory first. Whereas, when we use the command-line client, the existing file is deleted before opening the new one for writing.

I have also changed the preferences in Cyberduck to 'Prompt' instead of 'Overwrite' and in this case CyberDuck does indeed prompt what to do - so it is checking for existing files and finding them. But, when I then click 'Overwrite' - it then still appends data instead of deleting first.

Finally - I turned on 'Upload with a temporary filename' - and that fixes the issue - in that of course CyberDuck now uploads to a temp file name, then issues a REMOVE on the existing file, before renaming the temporary file. The CyberDuck log shows this. Lines 160-185 below are with 'Overwrite' selected, and no temporary filename. Then lines 186+ are for with 'Use temporary filename' switched on.

160 OPENDIR
161 READDIR
162 READDIR
163 CLOSE
164 OPEN
165 WRITE
166 WRITE
167 WRITE
168 WRITE
169 WRITE
170 WRITE
171 WRITE
172 WRITE
173 WRITE
174 WRITE
175 WRITE
176 WRITE
177 WRITE
178 WRITE
179 WRITE
180 CLOSE
181 SETSTAT
182 OPENDIR
183 READDIR
184 READDIR
185 CLOSE
186 OPENDIR
187 READDIR
188 READDIR
189 CLOSE
190 OPEN
191 WRITE
192 WRITE
193 WRITE
194 WRITE
195 WRITE
196 WRITE
197 WRITE
198 WRITE
199 WRITE
200 WRITE
201 WRITE
202 WRITE
203 WRITE
204 WRITE
205 WRITE
206 CLOSE
207 SETSTAT
208 REMOVE
209 RENAME
210 OPENDIR
211 READDIR
212 READDIR
213 CLOSE

So 'Use temporary filename' at least provides us with a workaround at the moment, but I still feel the 'Overwrite' preference (whether prompted or set prejudicially) is not working correctly.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Nov 8, 2019

@dkocher commented

When a file already exists, we open the file in truncate mode specified in the SFTP protocol. This forces an existing file with the same name to be truncated to zero length.

Loading

@cyberduck cyberduck closed this Nov 21, 2019
@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants