Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use dedicated upload session struct #4402

Merged
merged 25 commits into from
Dec 21, 2023

Conversation

butonic
Copy link
Contributor

@butonic butonic commented Dec 12, 2023

The current upload session handling is rundundant on several levels. This PR starts moving parts to a dedicated upload session we introduced in #4148. Actually, we did learn a few things while working on that.

This PR uses the same struct to

  • implement the tusd Upload interface
  • implement the storage UplvoadSession interface
  • persist the upload without changing the on disk format, so it is 100% backward compatible (I'll wait for the real chinges until after mergin this)
  • reduce some overhead by doing less stat requests and tree walking, is should improve performance under load
  • prepare reusing some upload session code for tus upload that is eg also required by Implement OCM TUS uploads #4410

Copy link

update-docs bot commented Dec 12, 2023

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

@butonic butonic self-assigned this Dec 12, 2023
@butonic butonic force-pushed the dedicated-upload-session-struct branch 6 times, most recently from a92800b to 6309bec Compare December 14, 2023 11:21
@butonic butonic marked this pull request as ready for review December 14, 2023 16:30
@butonic butonic requested review from a team, labkode and glpatcern as code owners December 14, 2023 16:30
aduffeck and others added 12 commits December 15, 2023 16:50
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

fix upload binPath

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic force-pushed the dedicated-upload-session-struct branch from 69e9fdb to 11d5bda Compare December 15, 2023 15:52
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
pkg/storage/utils/decomposedfs/node/node.go Outdated Show resolved Hide resolved
butonic and others added 3 commits December 18, 2023 10:30
Co-authored-by: Andre Duffeck <aduffeck@users.noreply.github.com>
Signed-off-by: jkoberg <jkoberg@owncloud.com>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic marked this pull request as draft December 18, 2023 13:13
@butonic
Copy link
Contributor Author

butonic commented Dec 18, 2023

keeping draft until full ci in ocis has run

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic marked this pull request as ready for review December 20, 2023 11:19
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
pkg/storage/utils/decomposedfs/node/node.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/upload/session.go Outdated Show resolved Hide resolved
pkg/storage/utils/decomposedfs/upload/store.go Outdated Show resolved Hide resolved
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
@butonic butonic requested a review from kobergj December 21, 2023 12:44
@butonic butonic merged commit 6fe91cb into cs3org:edge Dec 21, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants