Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Transaction Log v2 #42

Open
ecton opened this issue Mar 3, 2022 · 0 comments
Open

Implement Transaction Log v2 #42

ecton opened this issue Mar 3, 2022 · 0 comments
Labels
enhancement New feature or request
Projects
Khonsu Labs Roadmap

Comments

@ecton
Copy link
Member

ecton commented Mar 3, 2022

  • Add CRCs to the transaction log #35
  • Add "Session ID" to transaction log entries #40
  • Consider allowing larger transaction payloads
  • Add atomic upgrade
    • TransactionLog should expose a new error when the log format is the old format.
    • TransactionLog::repair should be added to rewrite transaction log to new file using new format, atomic swap to overwrite.
    • Add unit test upgrading an existing log file to the new format. Ensure testing one that contains out-of-order log entries.
@ecton ecton added the enhancement New feature or request label Mar 3, 2022
@ecton ecton added this to To do in Khonsu Labs Roadmap via automation Mar 3, 2022
ecton added a commit that referenced this issue Jul 10, 2022
@ecton
Roots now supports Sediment
da39002 
This is still a little ugly, but all unit tests are passing. Compaction
is currently not implemented, and currently Sediment acts as an
append-only file because Roots doesn't ever checkpoint it.

This commit replaces the append-only transaction format with a
commit-log powered transaction log using Sediment. When committing
multiple trees, the transaction log thread will send the trees to the
thread pool to be committed, while it commits the transaction log
itself. Once the transaction log is fully synced, the log manager thread
waits for all other trees to finish syncing, then continues to work like
the previous implementation.

The benefit of using Sediment is that this format now supports
checkpointing the transaction log, and it also implements several other
todos in the process.

Closes #42. Closes #35. Closes #40.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
Beta Roadmap
Status: Todo
Khonsu Labs Roadmap
To do
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ecton