Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

several postprocessing cornercases are broken #8848

Open
butonic opened this issue Apr 12, 2024 · 0 comments
Open

several postprocessing cornercases are broken #8848

butonic opened this issue Apr 12, 2024 · 0 comments
Labels

Comments

@butonic
Copy link
Member

butonic commented Apr 12, 2024

In cs3org/reva#4625 we added test cases for the possible async postprocessing outcomes when twe uploads are being processed in parallel. They were discovered while implementing cs3org/reva#4615 which fixes #8606. But only on the happy path.

There are still outcomes where file metadata is not cleaned up propery. I'll reproduce the cases here

  • the first can succeed before the second succeeds - Works!
  • the first can succeed after the second succeeds - Works!
  • the first can succeed before the second fails - Works!
  • the first can succeed after the second fails - the file should still be in "processing" after second upload is deleted
  • the first can fail before the second succeeds - list of revisions should be 0
  • the first can fail after the second succeeds - list of revisions should be 0
  • the first can fail before the second fails - file should be deleted, size should be correctly reverted
  • the first can fail after the second fails - file should be deleted, size should be correctly reverted

The root cause is the lack of consistent tracking of currently ongoing upload sessions.

Currently, we create the node metadata only when an upload finishes successfully. In order to tie together concurrently running upload sessions we have to create the node as part of the InitiateFileUpload call. Clients can already deal with an unaccessible node that has status 425, so this should not be a problem.

Then we can add all upload sessions to the node metadata with a list of key value pairs:
user.ocis.upload.{timestamp} -> {size};{sessionid}

The size can be used to calculate the sizediff when comparing it with the current node size. When stating a file the latest ongoing upload will override the node size. If an upload is successful, it will replace the size in the node. hm, then the second youngest upload size will overrule the latest size ...

🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Prio 3 or less
Development

No branches or pull requests

1 participant