Map: rearchitect Codex rollout token accounting around lineages

[iam-brain]

Destination

Land a sequence of isolated, reviewable PRs on the up-to-date main branch in iam-brain/CodexBar, replacing physical-file accounting with a validated lineage-ledger scanner, then prepare one cohesive upstream PR against steipete/CodexBar with materially closer token totals and bounded residual uncertainty.

Notes

This map carries execution, not planning alone. Each implementation task is one isolated stacked PR on the fork; a locally validated PR is sufficient to resolve its ticket without waiting for CI or merge. Never resolve more than one ticket per agent session. Fork main is synchronized to current steipete/CodexBar main; its former divergent tip is preserved on main-pre-upstream-sync-2026-07-13. Preserve production behavior until a dedicated promotion decision. Use the existing fast JSONL parser, active and archived rollout data, sanitized issue steipete#2037 fixtures, finalized OpenAI UTC data from July 9-12, make check, make test, and the repository review gate. Never include private rollout contents in issues or PRs.

Correctness comes before performance budgets. Discovery and accounting windows are separate. A lineage is scoped to one normalized Codex home; active and archived directories beneath it share scope, while separate homes never merge. Cache parsed documents separately from atomic, family-fingerprinted accounting results. Family-level conservative containment may remain indefinitely for malformed or irrecoverable evidence, but ordinary and known fork goldens must use primary ledger accounting. Diagnostics remain debug-only. Promotion uses evidence-based material improvement and classified residuals rather than a hard aggregate error rate.

Decisions so far

• Land the shadow lineage ledger foundation — The first isolated PR establishes transitive family accounting and UTC/local projections without changing production totals.

• Feed the fast parser into typed rollout documents — One fast JSONL pass now feeds both legacy snapshots and complete typed lineage observations while preserving cancellation and identity evidence.

• Discover complete lineages with bounded historical exceptions — Bounded discovery now follows only referenced ancestors across active and archived roots, diagnoses missing parents, and canonicalizes UUID identity casing.

• Preserve pricing and accounting dimensions through the ledger — Deduplicated observations now retain model and token dimensions, explicit pricing status, and UTC/local daily rows with per-observation long-context pricing.

• Run the lineage pipeline beside the production scanner — Refreshes now emit bounded, privacy-safe shadow differences and coverage diagnostics while legacy reports remain authoritative and failures stay contained.

Not yet specified

• Whether the final upstream submission is best presented as one squashed implementation PR or a small ordered stack; decide after fork integration establishes the final diff shape.

Out of scope

• Perfect equality with OpenAI usage statistics; the goal is a defensible material reduction in the gap.
• Changes to OpenAI prompt-cache behavior or heuristics that infer billing solely from cache ratios.
• Implicitly merging histories from separate Codex homes, even when provider-account identity appears equal.
• Token accounting for providers other than Codex.
• UI work unrelated to exposing existing daily cost and token reports.