fix: --project/--exclude are ignored by the durable headline totals - #864
Conversation
2bc7f21 to
6d071d6
Compare
The durable headline unions the carry-forward daily cache with today's live parse. Cached days were sliced to the requested provider (sliceDayToProvider) but never to the requested project, so a --project/--exclude query counted every historical day WHOLE while the detail panels — By Project / By Activity / By Model, all built from the name-filtered live parse — left the filtered projects out. The Overview total and the panels below it could not be reconciled; on a real one-week corpus the headline ran $2,428.85 over the By Project sum, which is exactly the excluded projects' spend. Add sliceDayToProject, the project-level counterpart of sliceDayToProvider, and apply it to the cache-sourced days. Cost, calls, sessions and savings come out of the per-project day stats the cache has carried since v15, so they stay exact and still include days whose session files have expired. Provider slices carry their own project split, so --provider on top of a project filter stays consistent. Fields the cache has no per-project split for are handled explicitly rather than reported as the filtered projects' own: - tokens/models/categories come from the (project-filtered) live parse when a project filter is active, which is exact for every surviving session and matches the panels that read the same parse; - days, or provider slices, carried from before v15 have no project split at all and cannot be attributed, so they contribute nothing and their cost is returned as unattributedCostUSD and footnoted by the terminal overview instead of being folded into a filtered total.
6d071d6 to
4bfcc02
Compare
…s date-deterministic Two issues on top of the --project/--exclude durable-headline fix: - sanitizeProjects dropped any project whose key is an Object.prototype member name (constructor, valueOf, __proto__, ...). A project key is a directory basename, so such a name is legitimate, and dropping it left the day's per-project split summing to less than the day cost — so the sliced, project-filtered headline silently lost that project's spend with no footnote. The keys are written via setOwn (defineProperty), so keeping them is pollution-safe; only the redundant `name in Object.prototype` guard is removed. Regression test added (mutation-checked: fails without the guard removed). - The new project-filter tests seeded a carried day 10 days ago but ranged over the calendar month, so within the first 10 days of a month that day fell out of range and the tests went red. Replaced with a fixed 20-day window that always spans the seeded day.
iamtoruk
left a comment
There was a problem hiding this comment.
Reviewed on 3432a07. This is a correct and valuable fix — it closes the ~$2,400 headline error where --project/--exclude were ignored by the durable Overview total while By Project / By Activity / By Model honored them.
Verified end to end. Driving the real buildDurablePeriod against isolated caches with an in-range carried day: --exclude → keeps the other project's carried cost, --project → keeps only the named one, path-pattern matching works, exclude-all → 0, unfiltered control unchanged, and the pre-v15 unattributedCostUSD path correctly footnotes an un-splittable day instead of folding it in. An adversarial pass confirmed the money math holds across combined --provider+--project, savings, heatmap day-selection, substring over-match, and the no-double-count sum-invariant (matched + excluded + unattributed == whole).
Two things I fixed and pushed (a769b50):
- Silent dropped cost for prototype-named projects. A project directory named exactly a JS prototype member (
constructor,valueOf,__proto__, ...) was stripped bysanitizeProjectson cache load, so the day's per-project split no longer summed to the day cost and the sliced headline silently lost that project's spend with no footnote (repro: a $15 day read as $5). The keys are written viasetOwn/defineProperty, so keeping them is pollution-safe; I removed only the redundantname in Object.prototypeguard and added a mutation-checked regression test. - Time-flaky tests. The new tests seeded a carried day 10 days ago but ranged over the calendar month, so within the first 10 days of a month that day fell out of range and they went red (they were failing on today's date). Replaced the range with a fixed 20-day window that always spans the seeded day; all 12 now pass deterministically.
Also merged main (17 commits, clean) to resolve the lag. tsc clean; full suite green apart from the two pre-existing parser.test.ts failures that reproduce on main. Finding 2 (a v14-era expired day is conservatively over-attributed to the footnote — disclosed, never silent or double-counted) is acceptable as-is. Good to merge.
…day proof The tests added in #864 seeded today's live session at a fixed wall-clock hour (12:00 local). The periods they build end at `new Date()`, and the suite runs under TZ=UTC, so for any run before 12:00 UTC that timestamp is in the FUTURE and the range filter correctly drops it. The live half of the cache/live union then contributes nothing, and the one assertion that needs a non-zero headline — the unattributed-cost footnote — fell into renderOverview's "No usage found" early return and went red. Half of every day was a failing window; a769b50 fixed the start-of-month flake but this one survived it. Verified by bisecting the fixture on the unpatched test: moving the seeded hour from 12:00 to 01:00 (past, at a 03:38 UTC run) turns the same 12 tests green, so the timestamp's position relative to `now` is the whole cause. - Seed the session a few minutes BEFORE now, clamped to today's midnight, so it is always both inside today and already in the past. - Stop the footnote test depending on the live parse at all: seed a second, attributable cached day so the headline is non-zero from the cache alone. The test now exercises the footnote instead of the fixture's timing.
…ed days The provider list rebuilt its own day set straight from the daily cache, unioning unfiltered historical days with an already-filtered today. Per-provider costs therefore counted every carried day whole while today honoured the name filters, so the list could not be reconciled with the headline or the By Project panel. getagentseal#864 fixed the headline and left this deliberately untouched. Reuse durable.days, which is the same union the headline is built from, already narrowed by range, day selection and project filter. That is what the comment above the buildDurablePeriod call already promised this section would do. Providers whose entire spend is excluded do not vanish from the list: the installed-but-zero backfill below still adds them at cost 0.
Summary
--project/--excludewere ignored by the durable headline:buildDurablePeriodslices the cache-sourced days to the requested provider but never to the requested project, so the Overview total counted excluded projects while By Project / By Activity / By Model (built from the name-filtered live parse) left them out.sliceDayToProject, the project-level counterpart ofsliceDayToProvider, so cost / calls / sessions / savings are sliced out of the per-project day stats the cache has carried since v15 — exact, and still including days whose session files have expired.unattributedCostUSDand footnoted, never silently folded in.Testing
npm testpassesnpm run buildsucceedsnpm test: 2,492 passed, 5 failed — all 5 fail identically on a clean checkout of this PR's base commit, so none are mine:cache-refresh-lock(fence/heartbeat),parser(copilot file-purge, orphan fingerprint),sharing/connect-timeout, andcli-durable-totals"resolves provider filters identically". That last one fails onmainbecauseparseAllSessionsreturns nothing for a today-only range once the on-disk session cache is warm — unrelated to this change, but you may want a separate look.npx tsc --noEmitis clean.Repro
Overview reported $7,962.65 while every By Project row summed to ~$5,503.70 — a ~$2,458.95 gap, exactly the excluded projects' spend. By Activity and By Model both summed into the same $5,500–5,600 band, so the detail panels agreed with each other and only the headline was off.
Measured on the same corpus with
--format json(the-p weekwindow had moved on by then, so absolute figures differ slightly):Calls were off by the same mechanism: 84,458 → 56,492. The small residual is not from this change — the unfiltered control shows a larger pre-existing divergence, because the durable path counts whole local days out of the cache while the live parse filters turns by exact timestamp, so the two differ at a rolling range's boundary.
Footnote behaviour on the same real cache (15 of 93 days predate the per-project split):
Root cause
buildDurablePeriod()unions two sources: today, viafp(await parseAllSessions(...))wherefp = filterProjectsByName(...)— filtered — and history, viagetDaysInRange(cache, ...)merged byunionDaysForPeriod()— never project-filtered. Provider filtering did slice the historical days (pf === 'all' ? allDays : allDays.map(d => sliceDayToProvider(d, pf))); there was simply no project-level counterpart.buildJsonReport()readstotalCostUSDstraight fromdurable.data, which is why Overview overstated, while By Project reads the correctly-filtereddurable.liveProjects.Filter matching mirrors
filterProjectsByNameexactly — case-insensitive substring against the project name or its path, include then exclude — so a filter selects the same projects whether it resolves against a fresh parse or the day cache. Provider slices carry their ownprojectssplit, so they are sliced too and--provider X --exclude Ystays consistent with the day-level slice.Known limitation — your call
ProjectDayStatsis{ cost, calls, savingsUSD, sessions, path? }: there is no per-project split for tokens, models or categories. I picked the option that never reports another project's numbers as the filtered projects', but the alternative is defensible and I'm happy to switch:unattributedCostUSDand the overview footnotes it. Folding them in would keep--project Xclaiming spend that provably isn't X's, so I chose a stated shortfall over a silent overstatement — but that trades against the never-lose-history invariant, so say the word if you'd rather keep them.A refinement I deliberately left out: on the 13 days in my cache where the day has a split but one provider slice doesn't, the missing slice is derivable as
day.projectsminus the other slices' projects. That assumes the merged day is internally consistent and a clamped subtraction would fail silently when it isn't, so it seemed better as a follow-up than as part of a correctness fix.Adjacent, not fixed here
The menubar /
statusprovider list has the same root cause:usage-aggregator.ts:632-635buildsunfilteredProviderDaysfromgetDaysInRange(cache, ...)(unfiltered) plusgetTodayAllDays()(filtered), so per-provider costs still ignore--project/--excludefor historical days. I left it out because narrowing it also changes which providers appear in the list at all, which felt like your call rather than mine. Happy to do it here or in a follow-up.Tests
tests/project-filter-durable-totals.test.ts, 11 cases, the failing-first ones written before the fix: exclude and include filters against a two-project carried day, path matching, a fully-excluded day, project-underneath-provider, and the two pre-v15 fallbacks plus the overview footnote. Two are invariant guards that hold before and after the change rather than red-first tests — the unfiltered headline being byte-for-byte what it was, and the menubar payload staying in step with the report under a project filter.Filed as #865, which also carries the two policy questions above in case you'd rather settle them there.
🤖 Generated with Claude Code