diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index fe79ca5..0e9160f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -195,9 +195,10 @@ sequenceDiagram U->>L: open(path, options) L->>L: validate options.superblock_count ∈ 2..=16 - L->>L: file exists & non-empty? - alt file exists - L->>IO: PageIo::open (acquires flock) + L->>L: file_exists? (pre-lock stat;
gates create_if_missing only) + L->>IO: PageIo::open (acquires flock) + L->>IO: page_count() — post-lock length + alt existed (post-lock page_count > 0) L->>TM: open_existing(cache) TM->>IO: read pages 0..MAX_SUPERBLOCKS TM->>SB: deserialize each candidate @@ -208,7 +209,6 @@ sequenceDiagram TM->>TM: re-derive handle-table + membership
outer depth from their roots (I99 / C1) TM-->>L: TransactionManager else fresh - L->>IO: PageIo::open L->>TM: create_new(cache, superblock_count) Note over TM: I2: write all N slots with
staggered counters so a torn
first commit has a fallback end @@ -217,6 +217,8 @@ sequenceDiagram The format-version gate after `select` is what makes the README's "sacred within a major version" promise concrete: a file written by a future, incompatible MAJOR is rejected with `UnsupportedFormatVersion`. Same-major files (any minor) open cleanly. (See I29 for the packed-MAJOR/MINOR scheme; I31 for the per-page version byte that supports lazy upgrade within a major.) +**Landmine:** the create-vs-open decision must come from `page_count()` observed *after* `PageIo::open` holds the flock, never from the pre-lock `file_exists` stat — a pre-lock decision races a concurrent creator (process B stats an empty file, process A creates + commits + closes, B's stale decision then runs `create_new` over A's committed data). `file_exists` still gates `create_if_missing`, since a refused open must never materialize an empty file — but it must not also decide create-vs-open (I143). + --- ## On-disk format @@ -610,6 +612,8 @@ The cap parameter (`DefragOptions::max_pages`) bounds the number of *values* rel On any fatal error — an `IoError` from `fsync`, a `ChecksumMismatch` on a page load, a `CorruptSuperblock` on open, a `DecryptionFailed`, any error raised after the commit protocol has begun — the `TransactionManager` becomes **poisoned**. Every subsequent call returns `ChiselError::Poisoned`, **including reads**. The only legal recovery is to drop the `Chisel` handle and call `Chisel::open` again; the shadow-paging recovery path then returns the database to its last-durable state. (See I1; rationale — fsyncgate, the Mutex analogy — in THEORY.md.) +**Landmine:** every `TransactionManager` method with a fatal-error path must route its `Result` through `self.poison_on_fatal(result)` before returning. A new entry point that instead propagates the error directly (a bare `?`) bypasses this — the manager stays unpoisoned while a subsequent partial mutation may already have reached `current_roots`, so a later `commit()` can durably persist an indeterminate state (I145: `reclaim_freemap_orphans` missed this wrap). + ### Engine-activity counters `Chisel::counters()` returns a `ChiselCounters` snapshot of four cumulative-from-open counters: `cache_hits`, `cache_misses`, `pages_allocated`, and `fsync_calls`. Each counter is a `Cell` living at the site that increments it (`PageCache` for the first three, `PageIo` for fsync), and `PageCache::counters()` aggregates them into a single struct read via `PageIo::fsync_count()`. @@ -639,6 +643,8 @@ Chisel supports optional authenticated encryption of database files (shipped; MA **On-disk stride.** Encrypted databases use a uniform 8232-byte stride for every page including the superblock slots. Plaintext databases use the 8192-byte stride; the two are mutually exclusive and the stride is recorded in the superblock's plaintext crypto-header (bytes 325..329) so the engine reads the correct number of bytes before any operation. The `page_io` layer is stride-agnostic: callers set the stride once (via `PageIo::set_stride`) and all subsequent raw reads/writes use it. +**Landmine:** `stride` is guarded only by the page's forgeable XXH3 checksum, not the AEAD tag — it must be readable before any DEK is available, so it cannot live inside the sealed body. `open_existing` validates `stride == ENC_PAGE_SIZE` before calling `set_stride`; skip that check and a forged stride of 0 divides-by-zero panics `set_stride`, while a huge forged stride drives multi-GiB read allocations — both on `open` alone, no key required (I144). + **Envelope (key hierarchy).** A random 256-bit per-database DEK encrypts all page content. The DEK is never stored in plaintext: it is wrapped under a key-encryption key (KEK) and the wrapped form is held in a plaintext key-slot table inside the superblock's reserved region. The crypto-header + key-slot table occupy bytes **324..1356**: ```text diff --git a/python/README.md b/python/README.md index 1789f96..e4fffe8 100644 --- a/python/README.md +++ b/python/README.md @@ -349,6 +349,11 @@ Catch and continue. | `InvalidSuperblockCountError` | `superblock_count` outside `2..=16` | | `CacheFullError` | Page cache hit its strict `cache_max_bytes` cap with every cached page dirty (no clean page available for eviction) AND the spillway is disabled (`spillway_max_bytes=0`); commit or roll back to drain. When the spillway is enabled (default), the cache overflows into it instead and you'll see `SpillwayFullError` only if the spillway also fills. | | `SpillwayFullError` | Spillway sidecar's `spillway_max_bytes` cap was reached during a transaction; commit or roll back to drain the spillway. Database is intact. | +| `NoEncryptionKeyError` | Opened an encrypted database without supplying `encryption_key` | +| `InvalidEncryptionKeyError` | `encryption_key` was supplied but unwraps no key slot (wrong passphrase or wrong raw bytes) | +| `EncryptionNotSupportedError` | `encryption_key` was supplied but the database is plaintext | +| `NoFreeKeySlotError` | `add_key` / `rotate_key` attempted but all 8 key-slot table entries are in use | +| `LastKeySlotError` | `remove_key` would clear the last active key slot, leaving the database permanently unopenable | | `TagMismatchError` | `delete_tagged(handle, tag)` was passed a `tag` that doesn't match the handle's stored tag; the chunk and membership index are left untouched | | `ClosedError` | Any call on a `Chisel`, `Transaction`, or `Savepoint` after `db.close()` | | `AlreadyFinishedError` | Second explicit drive on a transaction or savepoint | @@ -360,6 +365,7 @@ Drop the handle and reopen. | Class | When it fires | |---|---| | `IoError` | Underlying filesystem I/O error. Also subclasses the builtin `OSError`, so it is catchable as `except OSError` and `.errno` is `OSError`'s native attribute (with `.strerror` set when an errno exists). Carries `.errno` (the raw OS error code, or `None` if unavailable) and `.kind` (the Rust `io::ErrorKind` name, e.g. `"PermissionDenied"`), so callers can branch on the cause without parsing the message. | +| `DecryptionFailedError` | A page or the superblock body failed AEAD authentication (wrong key, or the ciphertext was tampered with) | | `ChecksumMismatchError` | A page's XXH3 checksum did not validate on load | | `CorruptSuperblockError` | No readable superblock slot found | | `FileSizeMismatchError` | File size inconsistent with the superblock's claim |