Skip to content

fix: --project/--exclude are ignored by the durable headline totals - #864

Merged
iamtoruk merged 3 commits into
getagentseal:mainfrom
GodCC6:fix/project-filter-durable-headline
Aug 1, 2026
Merged

fix: --project/--exclude are ignored by the durable headline totals#864
iamtoruk merged 3 commits into
getagentseal:mainfrom
GodCC6:fix/project-filter-durable-headline

Conversation

@GodCC6

@GodCC6 GodCC6 commented Jul 31, 2026

Copy link
Copy Markdown

Summary

  • --project / --exclude were ignored by the durable headline: buildDurablePeriod slices the cache-sourced days to the requested provider but never to the requested project, so the Overview total counted excluded projects while By Project / By Activity / By Model (built from the name-filtered live parse) left them out.
  • Adds sliceDayToProject, the project-level counterpart of sliceDayToProvider, so cost / calls / sessions / savings are sliced out of the per-project day stats the cache has carried since v15 — exact, and still including days whose session files have expired.
  • Fields the cache has no per-project split for are handled explicitly rather than attributed to the filtered projects; a day that cannot be attributed at all is reported as unattributedCostUSD and footnoted, never silently folded in.

Testing

  • I have tested this locally against real data (not just unit tests)
  • npm test passes
  • npm run build succeeds

npm test: 2,492 passed, 5 failed — all 5 fail identically on a clean checkout of this PR's base commit, so none are mine: cache-refresh-lock (fence/heartbeat), parser (copilot file-purge, orphan fingerprint), sharing/connect-timeout, and cli-durable-totals "resolves provider filters identically". That last one fails on main because parseAllSessions returns nothing for a today-only range once the on-disk session cache is warm — unrelated to this change, but you may want a separate look. npx tsc --noEmit is clean.

Repro

npx codeburn report -p week \
  --exclude Projects/ai-invest-copilot \
  --exclude writing/publishing/automation \
  --exclude Projects/novel-writing-publish \
  --exclude ai/invest/copilot

Overview reported $7,962.65 while every By Project row summed to ~$5,503.70 — a ~$2,458.95 gap, exactly the excluded projects' spend. By Activity and By Model both summed into the same $5,500–5,600 band, so the detail panels agreed with each other and only the headline was off.

Measured on the same corpus with --format json (the -p week window had moved on by then, so absolute figures differ slightly):

Overview total By Project sum gap
before $8,004.30 $5,575.45 +$2,428.85
after $5,535.12 $5,574.74 −$39.62
unfiltered control (after) $8,095.19 $8,166.12 −$70.93

Calls were off by the same mechanism: 84,458 → 56,492. The small residual is not from this change — the unfiltered control shows a larger pre-existing divergence, because the durable path counts whole local days out of the cache while the live parse filters turns by exact timestamp, so the two differ at a rolling range's boundary.

Footnote behaviour on the same real cache (15 of 93 days predate the per-project split):

Bottom line: Lifetime totals $15,148.50 across 18,512,251,593 tokens, mostly claude / gpt-5.6-sol.
  includes $27.26 preserved from expired session logs
  excludes $2,214.63 from days with no per-project history

Root cause

buildDurablePeriod() unions two sources: today, via fp(await parseAllSessions(...)) where fp = filterProjectsByName(...) — filtered — and history, via getDaysInRange(cache, ...) merged by unionDaysForPeriod() — never project-filtered. Provider filtering did slice the historical days (pf === 'all' ? allDays : allDays.map(d => sliceDayToProvider(d, pf))); there was simply no project-level counterpart. buildJsonReport() reads totalCostUSD straight from durable.data, which is why Overview overstated, while By Project reads the correctly-filtered durable.liveProjects.

Filter matching mirrors filterProjectsByName exactly — case-insensitive substring against the project name or its path, include then exclude — so a filter selects the same projects whether it resolves against a fresh parse or the day cache. Provider slices carry their own projects split, so they are sliced too and --provider X --exclude Y stays consistent with the day-level slice.

Known limitation — your call

ProjectDayStats is { cost, calls, savingsUSD, sessions, path? }: there is no per-project split for tokens, models or categories. I picked the option that never reports another project's numbers as the filtered projects', but the alternative is defensible and I'm happy to switch:

  • tokens / models / categories now come from the (already project-filtered) live parse when a project filter is active. Exact for every surviving session and consistent with the By Model / By Activity panels that read the same parse — but bounded by source retention, so a project-filtered lifetime query no longer counts tokens from days whose logs are gone. Keeping the day-level values instead would leave those fields overstating by exactly the excluded projects.
  • days (or provider slices) carried from before v15 have no project split at all, so they cannot be attributed and contribute nothing; their cost comes back as unattributedCostUSD and the overview footnotes it. Folding them in would keep --project X claiming spend that provably isn't X's, so I chose a stated shortfall over a silent overstatement — but that trades against the never-lose-history invariant, so say the word if you'd rather keep them.

A refinement I deliberately left out: on the 13 days in my cache where the day has a split but one provider slice doesn't, the missing slice is derivable as day.projects minus the other slices' projects. That assumes the merged day is internally consistent and a clamped subtraction would fail silently when it isn't, so it seemed better as a follow-up than as part of a correctness fix.

Adjacent, not fixed here

The menubar / status provider list has the same root cause: usage-aggregator.ts:632-635 builds unfilteredProviderDays from getDaysInRange(cache, ...) (unfiltered) plus getTodayAllDays() (filtered), so per-provider costs still ignore --project/--exclude for historical days. I left it out because narrowing it also changes which providers appear in the list at all, which felt like your call rather than mine. Happy to do it here or in a follow-up.

Tests

tests/project-filter-durable-totals.test.ts, 11 cases, the failing-first ones written before the fix: exclude and include filters against a two-project carried day, path matching, a fully-excluded day, project-underneath-provider, and the two pre-v15 fallbacks plus the overview footnote. Two are invariant guards that hold before and after the change rather than red-first tests — the unfiltered headline being byte-for-byte what it was, and the menubar payload staying in step with the report under a project filter.

Filed as #865, which also carries the two policy questions above in case you'd rather settle them there.

🤖 Generated with Claude Code

@GodCC6
GodCC6 force-pushed the fix/project-filter-durable-headline branch from 2bc7f21 to 6d071d6 Compare July 31, 2026 11:29
The durable headline unions the carry-forward daily cache with today's
live parse. Cached days were sliced to the requested provider
(sliceDayToProvider) but never to the requested project, so a
--project/--exclude query counted every historical day WHOLE while the
detail panels — By Project / By Activity / By Model, all built from the
name-filtered live parse — left the filtered projects out. The Overview
total and the panels below it could not be reconciled; on a real
one-week corpus the headline ran $2,428.85 over the By Project sum,
which is exactly the excluded projects' spend.

Add sliceDayToProject, the project-level counterpart of
sliceDayToProvider, and apply it to the cache-sourced days. Cost, calls,
sessions and savings come out of the per-project day stats the cache has
carried since v15, so they stay exact and still include days whose
session files have expired. Provider slices carry their own project
split, so --provider on top of a project filter stays consistent.

Fields the cache has no per-project split for are handled explicitly
rather than reported as the filtered projects' own:

- tokens/models/categories come from the (project-filtered) live parse
  when a project filter is active, which is exact for every surviving
  session and matches the panels that read the same parse;
- days, or provider slices, carried from before v15 have no project
  split at all and cannot be attributed, so they contribute nothing and
  their cost is returned as unattributedCostUSD and footnoted by the
  terminal overview instead of being folded into a filtered total.
…s date-deterministic

Two issues on top of the --project/--exclude durable-headline fix:

- sanitizeProjects dropped any project whose key is an Object.prototype member
  name (constructor, valueOf, __proto__, ...). A project key is a directory
  basename, so such a name is legitimate, and dropping it left the day's
  per-project split summing to less than the day cost — so the sliced,
  project-filtered headline silently lost that project's spend with no footnote.
  The keys are written via setOwn (defineProperty), so keeping them is
  pollution-safe; only the redundant `name in Object.prototype` guard is removed.
  Regression test added (mutation-checked: fails without the guard removed).

- The new project-filter tests seeded a carried day 10 days ago but ranged over
  the calendar month, so within the first 10 days of a month that day fell out
  of range and the tests went red. Replaced with a fixed 20-day window that
  always spans the seeded day.

@iamtoruk iamtoruk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on 3432a07. This is a correct and valuable fix — it closes the ~$2,400 headline error where --project/--exclude were ignored by the durable Overview total while By Project / By Activity / By Model honored them.

Verified end to end. Driving the real buildDurablePeriod against isolated caches with an in-range carried day: --exclude → keeps the other project's carried cost, --project → keeps only the named one, path-pattern matching works, exclude-all → 0, unfiltered control unchanged, and the pre-v15 unattributedCostUSD path correctly footnotes an un-splittable day instead of folding it in. An adversarial pass confirmed the money math holds across combined --provider+--project, savings, heatmap day-selection, substring over-match, and the no-double-count sum-invariant (matched + excluded + unattributed == whole).

Two things I fixed and pushed (a769b50):

  1. Silent dropped cost for prototype-named projects. A project directory named exactly a JS prototype member (constructor, valueOf, __proto__, ...) was stripped by sanitizeProjects on cache load, so the day's per-project split no longer summed to the day cost and the sliced headline silently lost that project's spend with no footnote (repro: a $15 day read as $5). The keys are written via setOwn/defineProperty, so keeping them is pollution-safe; I removed only the redundant name in Object.prototype guard and added a mutation-checked regression test.
  2. Time-flaky tests. The new tests seeded a carried day 10 days ago but ranged over the calendar month, so within the first 10 days of a month that day fell out of range and they went red (they were failing on today's date). Replaced the range with a fixed 20-day window that always spans the seeded day; all 12 now pass deterministically.

Also merged main (17 commits, clean) to resolve the lag. tsc clean; full suite green apart from the two pre-existing parser.test.ts failures that reproduce on main. Finding 2 (a v14-era expired day is conservatively over-attributed to the footnote — disclosed, never silent or double-counted) is acceptable as-is. Good to merge.

@iamtoruk
iamtoruk merged commit 2de4d10 into getagentseal:main Aug 1, 2026
iamtoruk pushed a commit that referenced this pull request Aug 3, 2026
…day proof

The tests added in #864 seeded today's live session at a fixed wall-clock
hour (12:00 local). The periods they build end at `new Date()`, and the
suite runs under TZ=UTC, so for any run before 12:00 UTC that timestamp
is in the FUTURE and the range filter correctly drops it. The live half
of the cache/live union then contributes nothing, and the one assertion
that needs a non-zero headline — the unattributed-cost footnote — fell
into renderOverview's "No usage found" early return and went red. Half
of every day was a failing window; a769b50 fixed the start-of-month
flake but this one survived it.

Verified by bisecting the fixture on the unpatched test: moving the
seeded hour from 12:00 to 01:00 (past, at a 03:38 UTC run) turns the
same 12 tests green, so the timestamp's position relative to `now` is
the whole cause.

- Seed the session a few minutes BEFORE now, clamped to today's
  midnight, so it is always both inside today and already in the past.
- Stop the footnote test depending on the live parse at all: seed a
  second, attributable cached day so the headline is non-zero from the
  cache alone. The test now exercises the footnote instead of the
  fixture's timing.
ozymandiashh added a commit to ozymandiashh/codeburn that referenced this pull request Aug 3, 2026
…ed days

The provider list rebuilt its own day set straight from the daily cache,
unioning unfiltered historical days with an already-filtered today. Per-provider
costs therefore counted every carried day whole while today honoured the name
filters, so the list could not be reconciled with the headline or the By Project
panel. getagentseal#864 fixed the headline and left this deliberately untouched.

Reuse durable.days, which is the same union the headline is built from, already
narrowed by range, day selection and project filter. That is what the comment
above the buildDurablePeriod call already promised this section would do.

Providers whose entire spend is excluded do not vanish from the list: the
installed-but-zero backfill below still adds them at cost 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants