Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[S3 compatible - OpenStack Swift] Multipart upload chunks are not deleted after overwrite operation #11690

Closed
cyberduck opened this issue May 30, 2021 · 4 comments

Comments

@cyberduck
Copy link
Collaborator

@cyberduck cyberduck commented May 30, 2021

28592ff created the issue

I have noticed that when we overwrite a file with s3 protocol, the data chunks are still present for both the upload id in +segments container.

Storage: Openstack Swift - S3 Compatible.
Steps to reproduce the bug.

  1. Upload a file - SLO/DLO
  2. Reupload the file with overwrite option.
  3. Check the +segments containers in openstack swift. It should have both the upload ID with data chunks.

Expected behavior: Data chunks in older upload ID folder should have been deleted.

Cyberduck Version: 7.9.2
Platform: Mac, Windows

Screenshots:
Two folders with different upload ID after successful overwrite: https://imgur.com/S4hY9zY
Screenshot from Cyberduck showing successful overwrite: https://imgur.com/b1zTVlZ


Attachments

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jun 9, 2021

@dkocher commented

We do delete segments of large file uploads when connected to OpenStack Swift. My understanding is that when connecting with S3 interoperability this would be the responsibility of the server as S3 does not have such a concept.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Aug 20, 2021

28592ff commented

Here is another aspect I observed: When I issue a normal delete, the chunks in +segments folder is deleted properly. It's only with overwrites that I hit the issue.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Aug 20, 2021

28592ff commented

Closing this. Seems to be bug in openstack swift.

Future Reference: https://bugs.launchpad.net/swift/+bug/1813202

Loading

@cyberduck cyberduck closed this Aug 20, 2021
@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Aug 20, 2021

@dkocher commented

Replying to [comment:2 highlow]:

Here is another aspect I observed: When I issue a normal delete, the chunks in +segments folder is deleted properly. It's only with overwrites that I hit the issue.

Overwriting files from a transfer will not explicitly delete files and thus we do not handle this use case.

Loading

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant