fix(codex): re-scan sessions resumed in older date partitions on a warm cache

[OfficialAbhinavSingh]

Closes #2558.

Problem

A Codex session whose rollout file lives in a date partition outside the scan window is unreachable by every candidate source once the cache is warm:

1. listCodexSessionFilesByDatePartition walks only YYYY/MM/DD directories inside the window (CostUsageScanner.swift:1462).
2. the flat root listing does not cover partitioned files.
3. cachedCodexSessionFiles requires the path to already be in cache.files — and it never got there.

The mtime lookback that exists precisely to catch active sessions was gated behind:

let shouldRunColdCacheLookback = cache.files.isEmpty || plan.rootsChanged

i.e. cold cache only. So resuming such a session appends usage that is never scanned again. Recovery needs forceRescan, a roots change, or deleting the cache — pricingChanged / needsProjectMetadataMigration set forceFullScan but do not add the file to the candidate list, so they do not recover it either.

Silent: no error, no diagnostic, just Codex cost and token totals that read low across codexbar cost, serve /cost, /dashboard/v1/snapshot, Usage & Spend, and model analytics.

Fix

Run the lookback on every refresh rather than only cold ones.

The important part is the cost. listCodexRecentlyModifiedFiles does two things: a bounded partition lookback (codexActiveSessionLookbackDays extra day directories) and an unbounded recursive walk of the whole sessions root. Only the bounded half now runs when the cache is warm — the recursive walk stays cold-cache only, so a warm refresh does not gain a cost that scales with corpus size. That matters here given #2538.

17 lines in one function plus the call site.

Proof (Linux, swift:6.3.3)

Pure filesystem — no network, no credentials. Write an in-window session so the cache is warm, write a second rollout into an older partition, scan, append a turn to the old rollout as resuming does, scan again, compare against forceRescan.

Before (stock 78523f4a):

✘ warm refresh reported 110 tokens but a forced rescan reported 4455;
  the resumed session in an older partition was never re-scanned

After:

✔ Test "a session resumed in an older partition keeps being counted on a warm cache" passed

I reverted the fix and re-ran to confirm the test genuinely fails without it, rather than trusting a green run.

Full Linux suite 318/318, swiftformat --lint clean, swiftlint --strict 0 violations / 1668 files.

Scope note

The fix covers resumes within codexActiveSessionLookbackDays of the window start, which is what the bounded lookback is designed for and matches the test. A session resumed after a gap longer than that still needs the recursive walk, and I deliberately did not enable that on warm refreshes because of its cost profile. Happy to extend if you'd prefer broader coverage over the cheaper refresh.

This file is currently busy (#2520, #2525, #2538) — I kept the change as small as I could to minimise conflicts, and I'm happy to rebase whenever.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed August 2, 2026, 8:59 PM ET / August 3, 2026, 00:59 UTC.

ClawSweeper review

What this changes

The PR makes Codex warm-cache refreshes progressively discover and rescan recently modified rollout files in older date partitions without enabling corpus-wide recursive discovery on every refresh.

Merge readiness

⚠️ Ready for maintainer review - 2 items remain

This PR remains necessary: current main still runs the older-partition modified-file lookback only for a cold cache or changed roots, which leaves the reported warm-cache undercount unresolved. The proposed branch replaces that gap with persisted, shared-budget discovery; the prior starvation concern is addressed on the current head, and no discrete correctness defect surfaced in this review.

Priority: P2
Reviewed head: 186029684f602d15368db60233fd7d1af0a551e6

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	Strong real filesystem/CLI proof and focused regression coverage support a clean bounded repair; exact-head platform checks remain the final merge gate.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (terminal): The PR body and maintainer follow-up provide after-fix terminal evidence: a warm post-resume scan reaches 4,455 tokens and matches an independent cold/forced scan, with focused and full-suite results reported for the rebased head.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (terminal): The PR body and maintainer follow-up provide after-fix terminal evidence: a warm post-resume scan reaches 4,455 tokens and matches an independent cold/forced scan, with focused and full-suite results reported for the rebased head.
Evidence reviewed	5 items	Current-main gap remains: Current main constructs the recent-file lookback with both partition and recursive discovery, but invokes it only under the cold-cache/roots-changed condition; a warmed cache cannot discover a newly resumed rollout in an older partition. Bounded persistent repair: The PR creates a per-root cursor and pending-file state, charges date-partition checks to the existing shared budget, and carries discovered files into the ordinary scan before finalizing persisted progress. Regression proof covers the earlier starvation finding: The Linux regression uses two deliberately undersized budget passes to prove the saved cursor advances into the old partition, then verifies the caught-up warm total equals a forced rescan.
Findings	None	None.
Security	None	None.

How this fits together

CodexBar’s Codex cost scanner enumerates local rollout files, parses their token usage, and persists results in a cache used by the CLI and Usage & Spend UI. This change affects the bounded discovery stage that feeds files into the existing parser and catch-up flow.

flowchart LR
  A[Codex rollout folders] --> B[Session discovery]
  B --> C[Older-partition lookback]
  C --> D[Shared scan budget]
  D --> E[Persistent cache state]
  E --> F[Usage parser]
  F --> G[CLI and Spend Dashboard totals]

Loading

Before merge

• Resolve merge risk (P1) - The branch changes persisted cache writes and consumes bounded warm-refresh discovery budget; queued Linux and macOS checks should complete successfully on the exact head before merge.
• Complete next step (P2) - No repair lane is needed: the review found no actionable patch defect, and the remaining step is ordinary exact-head CI completion plus maintainer merge review.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch surface	530 added, 51 removed across 6 files	The repair changes discovery and persisted cache state, with focused Linux and scanner-regression coverage.
Budget progression proof	2 constrained passes plus 1 convergence pass	The regression confirms deferred older partitions progress instead of repeatedly restarting at the oldest day.

Merge-risk options

Maintainer options:

1. Require an exact-head platform pass (recommended)
   Wait for the queued Linux and macOS checks to validate the portable cache write and bounded discovery behavior on the current head before merging.

Technical review

Best possible solution:

Land the bounded, resumable older-partition discovery path after the exact-head platform checks pass, keeping whole-corpus recursive discovery restricted to cold-start cycles.

Do we have a high-confidence way to reproduce the issue?

Yes. The added filesystem-only regression creates a warm cache, resumes a rollout in an older partition, and compares bounded warm catch-up with a forced rescan; current main’s cold-only call site explains why the path fails there.

Is this the best way to solve the issue?

Yes. Persisting bounded partition progress and discovered paths is the narrow maintainable repair: it restores discovery without making every warm refresh perform an unbounded recursive corpus walk.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4f8e3049a86c.

Labels

Label justifications:

• P2: The PR repairs silent Codex token and cost undercounting with a bounded but user-visible impact.
• merge-risk: 🚨 compatibility: The branch extends the persistent cost-cache schema and changes its atomic write implementation.
• merge-risk: 🚨 availability: Warm refreshes gain bounded discovery work, so preserving shared-budget limits matters for large local corpora.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body and maintainer follow-up provide after-fix terminal evidence: a warm post-resume scan reaches 4,455 tokens and matches an independent cold/forced scan, with focused and full-suite results reported for the rebased head.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR body and maintainer follow-up provide after-fix terminal evidence: a warm post-resume scan reaches 4,455 tokens and matches an independent cold/forced scan, with focused and full-suite results reported for the rebased head.

Evidence

What I checked:

• Current-main gap remains: Current main constructs the recent-file lookback with both partition and recursive discovery, but invokes it only under the cold-cache/roots-changed condition; a warmed cache cannot discover a newly resumed rollout in an older partition. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:2081, 4f8e3049a86c)
• Bounded persistent repair: The PR creates a per-root cursor and pending-file state, charges date-partition checks to the existing shared budget, and carries discovered files into the ordinary scan before finalizing persisted progress. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:2268, 186029684f60)
• Regression proof covers the earlier starvation finding: The Linux regression uses two deliberately undersized budget passes to prove the saved cursor advances into the old partition, then verifies the caught-up warm total equals a forced rescan. (TestsLinux/CodexWarmCacheResumeLinuxTests.swift:195, 186029684f60)
• Cache compatibility is additive: The cache schema adds an optional Codable active-lookback state, so existing cache files without this property decode through the existing defaulted structure. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift:163, 186029684f60)
• Feature-history provenance: The branch is based on the merged bounded Codex catch-up work, then received maintainer-authored budget, persistent-cursor, test-portability, and cache-write reconciliation commits; the proposed head is not contained in current main or a release tag. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:4506, 186029684f60)

Likely related people:

• steipete: Authored the branch’s budget integration, resumable-discovery repair, portable regression follow-up, and final cache-write change after reconciling the feature with current main. (role: recent area contributor; confidence: high; commits: d35be70f363a, 44274374dfc8, 186029684f60; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift)
• xx205: Authored the merged bounded Codex catch-up baseline that supplies the shared scan budget and persistent catch-up model this PR extends. (role: bounded-scanner feature contributor; confidence: high; commits: 2920019bc16d; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (8 earlier review cycles)

• reviewed 2026-08-01T20:06:42.080Z sha 405acce :: needs maintainer review before merge. :: none
• reviewed 2026-08-01T22:11:33.976Z sha 405acce :: needs maintainer review before merge. :: none
• reviewed 2026-08-02T07:06:56.655Z sha 31cf29b :: needs maintainer review before merge. :: none
• reviewed 2026-08-02T23:22:10.086Z sha d35be70 :: needs changes before merge. :: [P2] Advance deferred lookback partitions across refreshes
• reviewed 2026-08-02T23:27:26.229Z sha 542ed75 :: needs changes before merge. :: [P2] Advance deferred lookback partitions across refreshes
• reviewed 2026-08-03T00:19:07.933Z sha 542ed75 :: needs changes before merge. :: [P2] Advance deferred lookback partitions across refreshes
• reviewed 2026-08-03T00:26:14.109Z sha 4427437 :: needs maintainer review before merge. :: none
• reviewed 2026-08-03T00:36:49.060Z sha 6fc7a3a :: needs maintainer review before merge. :: none

[steipete]

Rebased onto 2920019bc1 (post-#2525) and force-pushed the maintainer-editable contributor branch. I did not merge.

Reconciliation with #2525:

• The warm path now checks only the extra 30 older date partitions; the corpus-wide recursive fallback remains cold-cache-only.
• Each older-partition directory check is charged to fix: preserve fork accuracy during bounded Codex cost catch-up #2525's shared CodexScanBudget, including its byte and wall-clock limits.
• The persisted missing-parent discovery state is unchanged. The quiescence regression still proves that an unchanged missing parent performs zero repeated head parses, while a newly created parent is published once.
• The Codex parser producer hash was regenerated to 5b3a8f9c68c15741.

The earlier CI failure was not a product defect or the #2573 spend-dashboard clock fixture. The substantive failure was codex project metadata migration drops unscanned legacy files: its synthetic “old, inactive” rollout had a brand-new filesystem mtime, so the new active-session lookback correctly classified it as recently modified. The fixture now pins that rollout's mtime to its old session date. The aggregate lint-build-test failure was downstream of that macOS shard failure.

Proof on the rebased head:

• CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 swift test --filter CodexWarmCacheResumeLinuxTests — 2/2 passed, including warm-total-equals-cold/forced reference and explicit shared-budget deferral/catch-up.
• CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 swift test --filter 'CostUsage(ScannerBreakdown|PerformanceGate)Tests' — 119/119 passed.
• make check — parser hash current; SwiftFormat clean; SwiftLint 0 violations.
• CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 make test — 782/782 selections; 66/66 groups passed first attempt; zero retries, recoveries, timeouts, or Kiro waiver.
• Source-blind public CLI fixture with isolated CODEX_HOME and cache homes — pre-resume 110 tokens; warm post-resume 4,455; independent cold scan 4,455.
• Autoreview, local reconciliation and complete branch diff against origin/main — clean, no accepted/actionable findings.

Commits:

• 24ac2b9c27 — contributor implementation rebased onto post-fix: preserve fork accuracy during bounded Codex cost catch-up #2525 main.
• d35be70f36 — shared-budget integration, regression reconciliation, and regenerated parser hash.
• 542ed75d48 — make the shared-budget proof portable across Linux and macOS.
• 44274374df — persist bounded lookback progress/candidates and reconcile file-path aliases.
• 6fc7a3ab21 — make persisted-cursor assertions use the exact cache artifact on Linux.

CI/review follow-up: the first rebased run exposed one Linux-only assertion in the new budget test. A later ClawSweeper review correctly found that a budget smaller than the partition offset would restart at the oldest day and could starve later partitions. The final implementation persists per-root day cursors and discovered candidate paths, keeps cold recursive discovery confined to its original cycle, and reconciles /var versus /private/var cache aliases by file identity. Two repeated 10-unit passes now prove progress into the old partition before a larger pass converges to the cold reference. Linux assertions decode the exact persisted artifact rather than the migration-aware current-cache view; the production second pass still proves reload and advancement. The final focused suites, 782-selection full suite, make check, source-blind CLI contract, and complete-branch autoreview are green on 6fc7a3ab21.

[steipete]

Exact-head follow-up for 186029684f (not merged):

• CI run 30775618181 is fully green: lint, Linux x64, Linux ARM64, Linux musl, both macOS shards, and the aggregate gate all passed.
• The preceding run's x64 and ARM64 jobs both failed at the same persisted-artifact assertion because Foundation replaceItemAt removed the existing cache destination without publishing its replacement on Linux. 186029684f replaces that double-staging path with the repository's cross-platform Data.write(..., .atomic) pattern.
• The Codex parser hash is current at dcab8c067490a89e.
• Post-fix local proof: warm-cache regressions 2/2; broader CostUsage suites 347/347; make check clean; make test 782/782 selections and 66/66 groups on the first pass with zero retries, timeouts, or Kiro waiver.
• Final local autoreview was clean with no accepted/actionable findings. ClawSweeper also re-reviewed exact head 186029684f and found no actionable defect; exact-head CI was its only remaining merge gate.