Skip to content

Logbook: the append-only build trace — Supabase live store, hash-chained git archive, full backfill (#628) - #632

Merged
MaxGhenis merged 20 commits into
mainfrom
chronicle-628
Aug 7, 2026
Merged

Logbook: the append-only build trace — Supabase live store, hash-chained git archive, full backfill (#628)#632
MaxGhenis merged 20 commits into
mainfrom
chronicle-628

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Closes #628. Every build attempt at every rung gets one row in one system of record — Pavel's "where can I see the trace of all sparses, all denses" answered structurally.

  • Supabase schema logbook (migration committed): builds + predictions tables, disposition enum, FKs, forced RLS with an insert-only writer role (no UPDATE/DELETE), read-only exporter, break-glass admin, a trigger-enforced hash chain (SHA-256 of canonical row JSON + prev digest; advisory locking kills concurrent forks; wrong predecessors fail closed), and the redacted builds_public view for populace.dev.
  • Client (populace.build.logbook): validates fail-closed, spools locally first (fsync + atomic rename), then REST-delivers; absent credentials = spool-only mode by design (builds never block on the SaaS); reconciliation uses ordered duplicate-ignore inserts.
  • CLI (tools/logbook.py): fail-closed export (chain-continuity-verified append to logbook.jsonl), validate (a CI guard), and markdown render.
  • Backfill: 26 chained rows — all 23 HF releases reconciled against tags/manifests/pins/gates, plus the run 5/6/7 era as first-class rows (dispositions: 22 published, 1 certified, 2 failed, 1 superseded; costs $122/$65/$47.50; prediction links p001–p008 seeded, 21 prediction events).

Operator steps after merge (not in this PR): apply the migration to the Supabase project, load the backfill spool, wire populace.dev to builds_public. The pool tool's row emission ships in #631. Anthony's #576 staging-telemetry opt-out reconciles against this as the tracking surface.

🤖 Generated with Claude Code

MaxGhenis added a commit that referenced this pull request Aug 7, 2026
…ation (#632 review O1/O3)

Rows export to a public archive, so artifact_location and gate receipt
references now render through _local_artifact_reference: anchored to the
owning checkout, then home (~/), never a host-absolute path that leaks
local usernames and layout. Tests reconstruct receipt files from the
reference's declared anchor instead of parsing absolute paths, and a
direct unit test pins all three anchor branches.

to_mapping now re-authenticates through from_mapping at the serialization
boundary, converging the fallback writer on the #632 module's hardening
(nested-container mutation after construction cannot persist).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Aug 7, 2026
PostgREST's idempotent replay is the client's only INSERT shape:
?on_conflict=build_id with resolution=ignore-duplicates plans as
INSERT ... ON CONFLICT (build_id) DO NOTHING. PostgreSQL requires
plan-time SELECT privilege on the conflict-target column even on an
empty table, and FORCE ROW LEVEL SECURITY additionally requires a
SELECT policy for the conflict check — without both, every writer POST
fails with 42501 and the live store receives zero rows while the spool
absorbs everything. Verified end-to-end against a live PostgreSQL 14 in
review: plain INSERT succeeded, the replay shape failed, the column
grant advanced it to the RLS refusal, and the policy completed it, with
cost_usd still unreadable and UPDATE/DELETE still denied afterward.

The schema round-trip test now pins both lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis changed the title Chronicle: the append-only build trace — Supabase live store, hash-chained git archive, full backfill (#628) Logbook: the append-only build trace — Supabase live store, hash-chained git archive, full backfill (#628) Aug 7, 2026
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Round 1 verdict applied: F1 fixed, tool renamed Chronicle → Logbook

F1 (blocking) — applied exactly as the reviewer verified on live PostgreSQL 14: GRANT SELECT (build_id) to the writer plus the FOR SELECT policy so PostgREST's ON CONFLICT (build_id) DO NOTHING replay can plan under FORCE RLS. Least-privilege boundary unchanged — every other column (including cost_usd) stays unreadable, UPDATE/DELETE stay denied. The schema round-trip test now pins both lines (c6319e21).

Rename (5373dd9f) — Max adjudicated the name collision today: the data platform's Ledger repo becomes PolicyEngine/chronicle, so this tool takes Logbook. Schema, roles, policies, functions, module, CLI, tests, migration filenames, spool dir (logbook-spool/), and archive (logbook.jsonl) all move together. Persisted row bytes are untouched — the digest contract and <row_digest>.json grammar carry no tool name, and all 26 backfill rows + the 21 seeded prediction records remain byte-identical (the p018 claim note keeps its historical "chronicle rows" wording; the digest test caught my sweep touching it and I restored the bytes). Focused battery: 72/72, matching the review's count.

O1 — fixed on #631 (bcd62faf): rows now render artifact_location and gate receipt references through a local:// normalizer (checkout-anchored, then ~/, never host-absolute).

O3 — converged on #631 in the same commit: the fallback's to_mapping now re-authenticates through from_mapping at the serialization boundary.

O2 — owner ruling posted on #628.

Merging on green CI per the review's conditional approve.

🤖 Generated with Claude Code

MaxGhenis added a commit that referenced this pull request Aug 7, 2026
#578) (#631)

* Add uniform stacked survey sampling

* Add durable Chronicle spool rows

* Support tail transfer on partial clone attachments

* Bind stacked doctrines to PUF QRF checkpoints

* Expand stacked battery to production surface

* Integrate the PUF capital-gains tail into stacked builds

* Adopt the stacked spine in the production pool tool

* Exercise stacked checkpoint resumes through the tool

* Document stacked pool production adoption

* Authenticate stacked terminal gate aliases

* Resume stacked builds before source assembly

* Make stacked Chronicle receipts attempt durable

* Keep stacked QRF paths out of canonical checkpoints

* Pin legacy pool entrypoint publication bytes

* Match stacked Chronicle rows to the rung contract

* Prove one-target battery registry omissions fail by name

* Document the production stacked pool doctrine

* Harden stacked bank and partial-tail identity tests

* Prove the stacked tool uses the production orchestrator

* Pair stacked tail clones by assembly identity

* Add the stacked adoption changelog fragment

* Complete Chronicle durability on exact retry

* Reject unreconcilable Chronicle NUL text

* Normalize Chronicle receipt JSON

* Bind stacked failures to realized identity

* Bind and preserve the stacked capital gains tail

* Reject nonfinite stacked terminal values

* Keep stacked tail provenance owner-scoped

* Recognize extended stacked authority receipts

* Invalidate pre-tail stacked checkpoints

* Pin Chronicle rows before stacked preflight

* ci: retrigger checks (workflow did not fire on prior push)

* Skip PyTables-dependent Chronicle and resume tests in the PyTables-free wheel gate

The five new stacked-entrypoint tests write pool H5 output through pandas
HDFStore, which requires the tables package. The wheels job installs the
base wheels without the pytables extra, so these tests must skip there —
the same importorskip guard the file's other seven H5 tests already use.
Verified both directions: 6 passed with tables importable, 6 skipped with
tables blocked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Normalize exported row locations and converge to_mapping re-authentication (#632 review O1/O3)

Rows export to a public archive, so artifact_location and gate receipt
references now render through _local_artifact_reference: anchored to the
owning checkout, then home (~/), never a host-absolute path that leaks
local usernames and layout. Tests reconstruct receipt files from the
reference's declared anchor instead of parsing absolute paths, and a
direct unit test pins all three anchor branches.

to_mapping now re-authenticates through from_mapping at the serialization
boundary, converging the fallback writer on the #632 module's hardening
(nested-container mutation after construction cannot persist).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Rename the build-trace tool from Chronicle to Logbook

The data platform's Ledger repo is becoming PolicyEngine/chronicle, so the
build-trace tool takes Logbook to keep the two systems' names disjoint
(populace#628). Module, tests, CLI flag (--logbook-prev-row-digest),
environment override (POPULACE_LOGBOOK_PREV_ROW_DIGEST), and the spool and
receipt directory conventions (logbook-spool/, logbook-receipts/) all move
together; the 17-field row schema, digest contract, and spool filename
grammar (<row_digest>.json) are unchanged, so existing spool rows and
digests are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
MaxGhenis and others added 20 commits August 7, 2026 08:03
PostgREST's idempotent replay is the client's only INSERT shape:
?on_conflict=build_id with resolution=ignore-duplicates plans as
INSERT ... ON CONFLICT (build_id) DO NOTHING. PostgreSQL requires
plan-time SELECT privilege on the conflict-target column even on an
empty table, and FORCE ROW LEVEL SECURITY additionally requires a
SELECT policy for the conflict check — without both, every writer POST
fails with 42501 and the live store receives zero rows while the spool
absorbs everything. Verified end-to-end against a live PostgreSQL 14 in
review: plain INSERT succeeded, the replay shape failed, the column
grant advanced it to the RLS refusal, and the policy completed it, with
cost_usd still unreadable and UPDATE/DELETE still denied afterward.

The schema round-trip test now pins both lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The data platform's Ledger repo is becoming PolicyEngine/chronicle, so
the build-trace tool takes Logbook to keep the two systems disjoint
(populace#628, Max 2026-08-07). Schema, roles, policies, trigger and
function names, module, CLI, tests, migration filenames, and the spool
directory and archive filename conventions (logbook-spool/,
logbook.jsonl) all move together. Persisted row bytes are untouched:
the 17-field schema, digest contract, and <row_digest>.json spool
grammar carry no tool name, and the seeded prediction records keep
their historical claim text byte-identical to the raw ledger (the p018
note legitimately says 'chronicle rows' — history stays history).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 8d7ac67 into main Aug 7, 2026
4 checks passed
@MaxGhenis
MaxGhenis deleted the chronicle-628 branch August 7, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chronicle: one append-only trace for every build attempt at every rung

1 participant