Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Metadata writes atomic #60

Closed
3 tasks done
fako1024 opened this issue Mar 4, 2023 · 0 comments · Fixed by #62
Closed
3 tasks done

Make Metadata writes atomic #60

fako1024 opened this issue Mar 4, 2023 · 0 comments · Fixed by #62
Assignees
Labels
feature New feature or request
Milestone

Comments

@fako1024
Copy link
Collaborator

fako1024 commented Mar 4, 2023

In order to minimize the likelihood of broken blocks or even full GPDir / GPFile instances we should make the atomic (or at least as atomic as possible). That way we can not only minimize the possibility of corruption but also "recover" even if e.g. all or some of the GPFiles in a GPDir have been written when an issue (e.g. system halt or similar) occurs by simply picking up the file offsets from the last successful write, losing only the block during which the error occurred.

DoD

  • Make metadata write atomic (use temporary file, then move after successful serialization)
  • Ensure that GPFile instances are opened at their last known offset (not the end) and overwriting is allowed
  • Add test(s) to reproduce a failure / recovery scenario
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant