Per-checkpoint git-ref checkpoint store (git-refs backend)

[Soph]

https://entire.io/gh/entireio/cli/trails/693

Implements the per-checkpoint git-ref checkpoint store (#1471): each checkpoint is stored under its own ref refs/entire/checkpoints/<shard>/<id> (pointing at a commit whose tree root is that checkpoint's contents) instead of the single append-only entire/checkpoints/v1 branch. Selectable via config, off by default — the git-branch store remains the default and is unchanged.

Targets main. #1576 (path.Join cleanup) is stacked on top of this branch.

What's in it

1. Ref resolver — CheckpointID.ShardFor shards on the last two characters for both formats (legacy 12-hex and ULID): one positional rule, no format detection, so the shard can't be computed inconsistently between callers, and the random suffix spreads evenly while IDs stay sortable. RefName builds the ref and rejects invalid/KindUnknown IDs (returns an error rather than a malformed ref); ParseRef is the inverse. This is the git-refs namespace only — the v1 branch tree keeps its independent first-two Path() layout.
2. git-refs store — gitRefsStore implementing the full api/checkpoint contract over per-checkpoint refs, sharing the treeWriter core (so dupl stays quiet); registered as a git-backed git-refs backend. Orphan-then-parented per-checkpoint history; stamps checkpoint_version = refs-v1. Plus the flock-protected JSONL push-discovery queue. Read paths distinguish a genuinely-absent checkpoint (→ not-found) from real IO/fetch errors, which now propagate instead of being masked as "not found."
3. Config-aware pre-push — when the primary is git-refs, PrePush drains the queue and pushes the per-checkpoint refs fast-forward-only (never force). There is no server-side ref protection, so a force push could let a buggy/racing client clobber good remote history; instead a diverged ref is recovered by fetch + replay of the local-only commit onto the remote tip, then retried. It also honors the checkpoint policy (skips the push on a diverged/unsupported-format policy, leaving refs queued) exactly like the v1 path.
4. On-demand ref fetch — a missing checkpoint ref is fetched on read (resume/explain/attribution/tokens), so checkpoints written on another machine resolve after clone.
5. refs-v1 read/write policy + explain-on-clone fetches the specific ref by full ID.
6. e2e/CI — backend-aware test harness, E2E_CHECKPOINT_STORE knob → ENTIRE_CHECKPOINTS_PRIMARY env override, the git-refs leg added to the PR-CI canary matrix, and an e2e-checkpoint-store workflow to run the full suite against either backend.

Rollout

Selected via the checkpoints.{primary, mirrors} taxonomy (or the ENTIRE_CHECKPOINTS_PRIMARY env override). primary: git-refs + mirror: git-branch is the parallel-rollout shape (refs authoritative; v1 still written locally). ID generation stays 12-hex for now — emitting ULIDs is a separate, deliberately deferred change; the store shards/reads both formats today.

Explicitly out of scope (follow-ups)

• ULID emission (generation switch).
• Backfill/migration of existing v1 checkpoints into per-checkpoint refs.
• v1-mirror push for full downgrade safety (mirrors are write-only locally for now).
• OPF for the git-refs push path.

Testing

mise run lint 0 issues (dupl quiet — treeWriter core is shared); full unit suite passes; mise run test:integration 379 pass; go vet -tags e2e ./e2e/... clean. Canary green in both modes: git-branch 59/59 + roger 4/4 (unchanged), and E2E_CHECKPOINT_STORE=git-refs 58/59 + roger 4/4 (the one skip is TestAlternates, a v1-branch-specific object-alternate rebase-sync test with no analog for the git-refs store's independent per-checkpoint refs).

Refs: #1471

🤖 Generated with Claude Code

---

Note

High Risk
Large change to checkpoint persistence, git push/sync, and remote fetch paths; wrong behavior could lose or fail to sync checkpoint metadata across machines, though git-branch remains the default.

Overview
Adds an optional git-refs checkpoint backend: each checkpoint lives on its own ref refs/entire/checkpoints/<shard>/<id> (commit tree = checkpoint contents) instead of only the shared entire/checkpoints/v1 branch. Default remains git-branch; selection is via checkpoints.primary or ENTIRE_CHECKPOINTS_PRIMARY / ENTIRE_CHECKPOINTS_MIRRORS.

The new gitRefsStore implements the same read/write contract as the branch store (shared tree helpers), stamps refs-v1, and enqueues touched refs in a flock-protected JSONL push queue. Pre-push drains that queue and batch force-pushes those refs when the primary is git-refs (no v1 OPF path yet). On-demand ref fetch (RefFetcher / FetchCheckpointRef) lets resume, explain, attribution, and tokens resolve checkpoints written on another machine after clone.

refs-v1 is now read/write-supported in checkpoint policy; tests that mocked “unsupported refs” move to refs-v2. E2E gains backend-aware helpers, E2E_CHECKPOINT_STORE, and a e2e-checkpoint-store workflow to run the suite against either backend.

^{Reviewed by Cursor Bugbot for commit 220c6b1. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 220c6b1. Configure here.}

[Copilot]

Pull request overview

Adds an optional per-checkpoint git-ref persistent checkpoint store (git-refs) where each checkpoint is stored under refs/entire/checkpoints/<shard>/<id> (commit tree root = checkpoint contents), alongside backend-aware push/fetch behavior and E2E coverage. This fits into the checkpoint persistence layer as an alternative to the existing append-only entire/checkpoints/v1 branch backend, selectable via settings/env overrides.

Changes:

• Introduces git-refs as a new git-backed persistent checkpoint backend, with ref naming/parsing and a flock-protected JSONL push-discovery queue.
• Updates CLI read paths (resume/explain/attribution/tokens) to support on-demand fetching of missing checkpoint refs.
• Extends E2E harness to run against either backend and adds a workflow to run E2E by checkpoint store.

Reviewed changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
e2e/testutil/repo.go	Switches repo setup + checkpoint artifact capture helpers to be backend-aware.
e2e/testutil/backend.go	Adds backend selection + checkpoint state digest and ref/path helpers for git-branch vs git-refs.
e2e/testutil/assertions.go	Updates checkpoint advance/existence assertions to work under both topologies.
e2e/testutil/artifacts.go	Captures per-backend checkpoint trees/metadata for artifacts.
e2e/tests/main_test.go	Adds E2E_CHECKPOINT_STORE → ENTIRE_CHECKPOINTS_PRIMARY suite-wide wiring.
e2e/tests/session_lifecycle_test.go	Uses backend-aware checkpoint state for “no checkpoint advance” assertions.
e2e/tests/rewind_test.go	Uses backend-aware checkpoint state for multi-checkpoint rewind tests.
e2e/tests/resume_remote_test.go	Replaces v1-branch existence checks with backend-aware “checkpoints present” checks.
e2e/tests/explain_test.go	Replaces v1-branch existence checks with backend-aware “checkpoints present” checks.
e2e/tests/edge_cases_test.go	Uses backend-aware checkpoint state in edge-case tests.
e2e/tests/attach_test.go	Uses backend-aware “checkpoints present” guard before reading checkpoint baseline.
e2e/tests/alternates_test.go	Skips alternates-specific test under git-refs (branch-topology-specific).
e2e/README.md	Documents E2E_CHECKPOINT_STORE.
cmd/entire/cli/tokens_profile.go	Wires RefFetcher into checkpoint.Open for tokens profiling reads.
cmd/entire/cli/resume.go	Wires RefFetcher into checkpoint.Open for resume reads.
cmd/entire/cli/explain.go	Wires RefFetcher into checkpoint.Open for explain reads.
cmd/entire/cli/explain_export.go	Adds git-refs-aware remote fallback fetch when exporting by full checkpoint ID.
cmd/entire/cli/attribution.go	Wires RefFetcher into checkpoint.Open for attribution reads.
cmd/entire/cli/git_operations.go	Adds FetchCheckpointRef (single-ref fetch) used by git-refs read paths.
cmd/entire/cli/settings/checkpoints.go	Adds env overrides ENTIRE_CHECKPOINTS_PRIMARY / ENTIRE_CHECKPOINTS_MIRRORS.
cmd/entire/cli/settings/checkpoints_test.go	Tests env override precedence + mirror parsing.
cmd/entire/cli/strategy/push_common.go	Adds helpers for batching force-push of per-checkpoint refs.
cmd/entire/cli/strategy/manual_commit_push.go	Adds git-refs primary pre-push path draining queue and batch force-pushing refs.
cmd/entire/cli/strategy/refs_push_test.go	Tests force-push batching and stale-ref partitioning behavior.
cmd/entire/cli/strategy/checkpoint_policy_test.go	Updates “unsupported refs” stand-in version to refs-v2.
cmd/entire/cli/checkpointpolicy/format.go	Marks refs-v1 as read/write supported in policy format table.
cmd/entire/cli/checkpointpolicy/format_test.go	Updates tests to expect refs-v1 read/write supported.
cmd/entire/cli/checkpointpolicy/*.go (tests)	Adjusts tests to use refs-v2 as the unsupported version.
cmd/entire/cli/checkpoint/registry.go	Registers git-refs backend type and adds RefFetcher to open env.
cmd/entire/cli/checkpoint/open.go	Adds OpenOptions.RefFetcher + PrimaryIsRefs helper and plumbs env.
cmd/entire/cli/checkpoint/fetching_tree.go	Defines RefFetchFunc used by git-refs backend.
cmd/entire/cli/checkpoint/aliases.go	Exposes CheckpointVersionRefsV1 alias.
cmd/entire/cli/checkpoint/persistent.go	Extracts shared tree-reading helpers usable by both git backends.
cmd/entire/cli/checkpoint/id/id.go	Adds Kind() and ShardFor() (ULID last-2 shard vs legacy first-2).
cmd/entire/cli/checkpoint/id/id_test.go	Adds ShardFor() coverage.
cmd/entire/cli/checkpoint/refs_naming.go	Implements RefName + ParseRef for per-checkpoint refs.
cmd/entire/cli/checkpoint/refs_naming_test.go	Tests ref naming/parsing and shard validation.
cmd/entire/cli/checkpoint/refs_store.go	Implements gitRefsStore (persistent store over per-checkpoint refs) incl. on-demand ref fetch + list.
cmd/entire/cli/checkpoint/refs_store_test.go	Unit tests for write/read variants, sharding, history, on-demand fetch, queue enqueue.
cmd/entire/cli/checkpoint/refs_store_seam_test.go	Seam test for git-refs primary + git-branch mirror topology.
cmd/entire/cli/checkpoint/pushqueue.go	Adds flock-protected JSONL push queue stored in git common dir.
cmd/entire/cli/checkpoint/pushqueue_test.go	Tests queue enqueue/drain/remove semantics and malformed line skipping.
api/checkpoint/errors.go	Adds CheckpointVersionRefsV1 constant (refs-v1).
.github/workflows/e2e-checkpoint-store.yml	Adds workflow to run E2E against either checkpoint backend.