Storage Format en

Storage Format

What a .srcs file actually looks like on disk, and what happens byte-by-byte when you commit. This is the code-level follow-up to the "reverse delta" idea introduced in System Overview.

Header

A v2 file starts with a header line:

# SimpleRCS v2.0; hash_algo=sha256; encoding=utf-8;

v1 is the older, headerless format (see "How v1 differs" below).

What a block looks like

A block is a series of key @value@; lines, terminated by a blank line before the next block. The @ character gets escaped as @@ inside a value — binary payloads included, with no exception. See "Escaping and parity" below for why.

ver @1.2@;
date @2026-08-04T12:00:00@;
author @wkpark@;
log @fix typo@;
delta @0a1,1
> new line
@;
prev_hash @9f86d0...@;
hash @a3f5c2...@;

Field breakdown:

• ver, date, author, log — always present
• text — this block holds full text (a text HEAD)
• delta — this block holds a delta (the instructions to go from the next newer version back to this one)
• binary — used instead of text/delta when the content is binary. Format is <length>;<encoding>,<payload>. The tag is one of raw, base64 or base85, and the length counts the bytes actually on disk, after escaping, so a forward parser can skip the payload in one seek
• prev_hash, hash — v2 only. Previous block's hash and this block's hash
• signature — repeated once per GPG signature, if any are attached

Escaping and parity

Escaping is a property of the container, not of the encoding. Every value doubles its @, including one whose alphabet cannot contain an @ — the rule applies uniformly even where it is a no-op, as it is for base64. The encoding tag says only how the bytes were encoded; it says nothing about escaping.

That buys one thing: every @ inside a value appears an even number of times, so an odd run of @ is a delimiter. A parser can decide a field boundary from either direction, which is what the backward scan for HEAD relies on.

Without the rule a payload could imitate the block structure, since base85 has @ in its alphabet and raw carries arbitrary bytes verbatim.

What the hash actually covers

This part is a bit subtle. The hash is computed over the version's logical full text, not whatever is physically stored (see codec.calculate_block_hash). The payload is ver|date|author|log|<full text>|<previous hash>, concatenated in that order and hashed.

The reason: when a block that used to be HEAD gets converted into a delta on the next commit, its hash value must not change. The stored representation changed, but the actual content that version held didn't — so the hash has to stay the same, or the chain breaks.

What happens on disk during a commit

flowchart TD
    C["commit()"] --> B1["old HEAD as a delta block<br/>_format_block(is_delta=True)"]
    B1 --> B2["new content as a full-text block<br/>_format_block(is_delta=False)"]
    B2 --> D{"_rewrite_head<br/>is there a file_path,<br/>and do we own the handle?"}

    D -- "yes: file path" --> A1["_copy_prefix<br/>copy [0, HEAD start) into a temp file"]
    A1 --> A2["append both blocks to the temp file"]
    A2 --> A3["fsync(temp)<br/>only when durable"]
    A3 --> A4["os.replace(temp, original)<br/>atomic swap"]
    A4 --> A5["reopen the stream, drop the HEAD cache"]
    A5 --> A6["fsync(directory)<br/>only when durable"]

    D -- "no: BytesIO / caller stream" --> I1["seek(HEAD start)"]
    I1 --> I2["write(both blocks)"]
    I2 --> I3["truncate()"]

    classDef safe fill:#e8f5e9,stroke:#2e7d32
    classDef risky fill:#ffebee,stroke:#c62828
    class A4 safe
    class I2 risky

Loading

commit() builds two blocks:

1. Re-encode the current HEAD as a delta block (_format_block with is_delta=True)
2. Encode the new content as a full-text block (is_delta=False)

How those get onto disk depends on the backend (_rewrite_head).

File-path backend — temp file plus os.replace

1. Copy everything up to where the HEAD block starts — the unchanged prefix — into a temp file beside the original
2. Append "the re-encoded old HEAD" + "the new HEAD"
3. fsync the temp file, when durable is set
4. os.replace() it over the original, atomically
5. fsync the directory, when durable is set

Nothing live is ever overwritten. An interrupted commit costs the commit, not the history — which matters here more than usual: every older version is a reverse delta anchored on HEAD, so half-writing HEAD makes all of them unrecoverable at once.

RCS uses the same temp-and-rename shape but re-serialises the whole ,v per commit, because it stores HEAD first and a commit shifts everything after it. We store HEAD last, so the unchanged part is a prefix — copied as raw bytes, with no parsing and no re-serialisation. The reasoning and the measurements are in Plan Atomic Commit.

Memory and caller-owned stream backends

A BytesIO, or a file object the caller opened, has nothing to rename over, so it keeps the in-place rewrite:

self.stream.seek(self.head_info["start"])
self.stream.write(payload)
self.stream.truncate()

A BytesIO disappears with the process anyway, so durability is not a question being asked of it, and a caller-supplied stream has its lifetime owned by the caller.

durable

SimpleRCS(path, durable=True) is the default and fsyncs the temp file and the directory. The cost is dominated by those fsyncs rather than by the copy, so bulk work (benchmarks, migrations) can turn them off.

Net effect: every commit changes exactly one block — the last one — and everything before it stays byte-identical. On the file-path backend that "stays" is achieved by leaving the original alone and building a copy.

Snapshots

Passing commit(..., snapshot=True) changes step 3: instead of a delta, the old HEAD gets stored as full text. That breaks the delta chain at that point — reading that version means reading full text directly, no deltas to apply. If you know a section of history is going to get long and frequently read, taking a snapshot there buys back lookup performance for that stretch. It's opt-in per commit, not automatic.

Reading an old version

checkout(ver) starts at HEAD and walks backward one block at a time, applying each delta, until it hits the target version (_get_prev_block). If it hits a snapshot block along the way, it jumps straight to the full text there. It never loads the whole file into memory — it only reads the blocks it needs, in order.

How v1 differs (read-only support, historical reference)

v1 was an early format that was never actually shipped — no header, and delta blocks used the same text key as full-text blocks (no separate delta key). No hash chain, no GPG signatures. simple_rcs.py still has self._version < 2 branches for reading it. The v1-to-v2 migration tool still exists in repository history (added and reverted as commits 0c7783e/90f2706) but isn't in the current tree.

Text and binary delta formats

• Text: an RCS-style diff -n script — human-readable add/delete instructions.

• Binary: a BSDIFF40-compatible patch, encoded as raw, base64 or base85 and embedded in the block. Since the patch format itself is standard, it also interoperates with the native bsdiff/bspatch tools if you have them. raw is the RCS approach — keep the bytes and let the escaping carry them — which costs ~0.4% against base64's 33%.

  This is our storage format and is not compatible with git's binary patches (git layers pack-delta, zlib and base85). When git has to read the result, srcs_diff --binary emits a GIT binary patch block instead, via simple_rcs/gitpatch.py — see CLI Tools.

What this means in practice

Because revisions aren't full snapshots, total file size ends up closer to "sum of delta sizes + one full text" rather than "revision count × page size." A one-line edit typically only adds a few hundred bytes. Wiki Backend Design (Korean only) has real measurements of what this means when you're storing .srcs content in a database.