Closed
Conversation
Changes the bootstrap sequence, such that the initial system table
schemas and contents are stored in the commitlog.
Opening a database proceeds as follows:
- If a snapshot exists, restore it and replay the history suffix
- Otherwise:
- Create the "seed tables" `st_table` and `st_columns` in-memory
- Replay the history
- Whenever a table in the reserved system table range is found,
reset the system table schema and layout information in the
page manager, so as to repair incomplete schema information.
- If the history was empty:
- Create the rest of the system tables (i.e. sans seed tables) in a
transaction.
- Commit and persist that transaction.
gefjon
reviewed
Jul 25, 2024
Contributor
gefjon
left a comment
gefjon
left a comment
There was a problem hiding this comment.
I do not love this. Having reviewed it, I am now much more convinced by your inline snapshots proposal and am inclined to push that forward rather than this.
| ignore_duplicate_insert_error(st_tables.insert(blob_store, &row))?; | ||
| } | ||
|
|
||
| // |
cloutiertyler
added
release-1.0
and removed
release-any
|
|
cloutiertyler
added
the
close-stale-pr-create-issue
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes the bootstrap sequence, such that the initial system table
schemas and contents are stored in the commitlog.
Opening a database proceeds as follows:
st_tableandst_columnsin-memoryreset the system table schema and layout information in the
page manager, so as to repair incomplete schema information.
transaction.
Alternatives Considered
The already brittle bootrapping process is now even more brittle. In particular
the various incantations of repair / rebuild routines is hard to follow and easy
to break.
It is also unfortunate that the commitlog will be non-empty in case the
initialization from the user-supplied module fails.
As an alternative, a snapshot could be taken right after the database was fully
initialized, incl. the module. Instead of hashes, this snapshot could contain
their preimages and be inlined into the commitlog.
Restoring a database from such a snapshot requires only the statically known
schemas of the seed tables and a single repair pass.
API and ABI breaking changes
It will not be possible to open a database from a commitlog created by an older
version, because bootstrapping the system tables will only occur if the history
is empty.
Expected complexity level and risk
5 - database bootstrap is highly fragile