Fix burn hotspots silent hang on multi-session ledgers

[willwashburn]

Summary

• burn hotspots --since=7d (and equivalents) was hanging silently for minutes after ✔ read N turns. Root cause: runHotspotsAttribution issued queryUserTurns({sessionId}) once per eligible session, and the file adapter implements that as streamLines(ledger.jsonl) + in-memory filter — i.e., one full ledger scan per session. On a 7-day window with 169 sessions and a 190MB ledger, that's hundreds of full-file scans. There was also no withProgress around the loop, so the spinner disappeared.
• Fixed by replacing the per-session calls with a single queryUserTurns() pass plus an in-memory bucket-by-sessionId (new bulkUserTurnsBySession helper). Same shape pattern that loadToolResultEventsForTurns already uses, with the same comment explaining why.
• Added a withProgress task around the content-sidecar loop so it reports (i/N) progress instead of going silent.
• The --patterns user-turn loader had the same per-session pathology and now shares the same bulk helper.
• The HotspotsAttributionDeps test contract changed from per-session callbacks to bulk Set<string> → Map<string, T[]> loaders — the natural shape for the refactor. Updated EMPTY_DEPS and added a coverage test.

End-to-end on the reporter's live ledger: 6,614 turns / 169 sessions / 190MB ledger now completes in ~8.6s instead of hanging.

Test plan

• pnpm run test:ts — all 896 tests pass
• burn hotspots --since=7d against a real 190MB / 169-session ledger completes in ~8.6s with no silent gap after read N turns
• New test asserts loadUserTurnsBySession and loadContentBySession are each invoked exactly once with the eligible session-id set (regression guard for the per-session loop)

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d31e9c02f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid materializing all user turns before session filtering

bulkUserTurnsBySession now calls queryUserTurns() with no query filter and only applies the sessionIds filter afterward, which means both burn hotspots attribution and --patterns first load every user-turn record in the ledger into memory. Because queryUserTurns() eagerly collects to an array, a large historical ledger (especially when the command is scoped to a small recent window) can see major memory growth or heap OOMs that were not possible with the previous per-session filtered reads.

Useful? React with 👍 / 👎.

[willwashburn]

Good catch — valid concern, fixed in 2dc000f.

The risk: bulkUserTurnsBySession was calling queryUserTurns() with no filter and applying the sessionIds set only after collection, which on a small recent window against a long historical ledger would buffer every historical user-turn record into memory before bucketing. That turns the previous bounded per-session peak into an unbounded one.

The fix: forward q.since and q.source into queryUserTurns(q) so the streaming filter narrows during the pass.

What's not forwarded and why:

• q.until — the user turn that carries tool_results for the last in-window assistant turn can lag a few seconds past a hard until cutoff (userTurn.ts >= turn.ts), so filtering by until could lose attribution at the right edge.
• q.project — userTurnPasses ignores it by design (per the comment in file-adapter.ts), and the sessionIds set upstream is already project-scoped via queryAll(q).
• q.sessionId — would defeat the bulk call.

Also tightened the no-deps fallback in runHotspotsAttribution to the original per-session form, so any future caller that skips deps doesn't trip the same memory hazard.

Added hotspots-bulk-load.test.ts with an integration test that seeds a temp ledger with old + new user turns across multiple sessions and asserts since is honored end-to-end.