docs: Add commitlog reference document#4668
Merged
clockwork-labs-bot merged 7 commits intomasterfrom Apr 15, 2026
Merged
Conversation
TypeScript modules use ctx.random (a deterministic RNG on the context), not Math.random(). The old note incorrectly stated that Math.random() was automatically seeded by SpacetimeDB. Changes: - Remove Math.random() from the warning about non-deterministic RNGs - Replace the incorrect note with a proper table row documenting ctx.random in the TypeScript context properties reference - Apply same fixes to versioned docs (v1.12.0)
Consolidates proposals 0003 (commitlog format), 0016 (offset index), and 0027 (commitlog epoch) into a single reference document covering the on-disk format, integrity model, transaction record structure, offset indexing, wire format, and versioning. Written in textbook style with no emdashes or contractions, per the docs style guide.
Per Tyler: this is a reference manual, not something users need to know to use the database. Moved from core-concepts/databases/ to resources/reference/internals/ alongside the ABI reference and BSATN docs.
Corrections after auditing against crates/commitlog source: - Flags use high bits (7, 6, 5), not low bits (0, 1, 2) - Inputs have a u32 length prefix before the reducer name - Varint encoding is standard variable-length, not protobuf zig-zag - Outputs use a 1-byte length prefix (max 255 bytes) - V0 commits have no epoch field (8 bytes shorter) - Format version 1 is the current default - Version 1 can read version 0 segments (backward compatible) - Mutations: table IDs are u32, counts are varint
cloutiertyler
approved these changes
Mar 19, 2026
kim
reviewed
Mar 20, 2026
Contributor
kim
left a comment
kim
left a comment
There was a problem hiding this comment.
It may be worth mentioning that commitlog segments may be compressed. The only supported compression format is zstd seekable, which allows the offset indexes to remain valid after compression.
- Remove paragraph about old sled-based design (not relevant to reference doc) - Simplify commitlog terminology entry - Clarify that records payload is defined by the datastore, not the commitlog - Add note about zstd seekable compression for segments
kim
reviewed
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per Tyler's request. Consolidates the three commitlog proposals from SpacetimeDBPrivate into a single public reference document:
The document is written in textbook style with no emdashes or contractions, following the docs style guide. All proposal-specific language (motivation sections, alternatives considered, implementation stages, etc.) has been removed in favor of a direct reference format.
Placed at
docs/docs/00200-core-concepts/00100-databases/00400-commitlog.mdalongside the existing database concept docs.No private repo details are referenced.