Skip to content

Track cumulative annual model spend - #2469

Open
Chipagosfinest wants to merge 14 commits into
steipete:mainfrom
Chipagosfinest:agent/cumulative-spend
Open

Track cumulative annual model spend#2469
Chipagosfinest wants to merge 14 commits into
steipete:mainfrom
Chipagosfinest:agent/cumulative-spend

Conversation

@Chipagosfinest

@Chipagosfinest Chipagosfinest commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes Usage & Spend useful for a genuinely multi-provider setup while keeping incomplete data honest:

  • adds 7-, 30-, and 365-day dashboard windows
  • keeps every enabled/configured provider visible, including API-backed services such as OpenRouter plus Grok and Gemini
  • separates roster visibility from cost eligibility, so unsupported or unavailable spend never becomes fake $0
  • distinguishes connected, configured, unavailable, and no-usage-yet states
  • shows an estimated tracked subtotal (~$…) with reporting coverage when only some services expose cost history
  • adds a 310-point Overview hero for combined spend/tokens, with Codex prominent and compact rows for the remaining providers
  • removes the six-provider truncation; long Overview lists use native AppKit scrolling
  • adds a local-only share/copy card that preserves the full connected roster, explicit missing spend, +N overflow, currency separation, and reporting coverage
  • keeps refresh/error state ahead of stale metrics and lets accessibility-sized rows expand

Root cause

Tracked Access used enabledFirstPartyProvidersForDisplay() as its roster. That helper is intentionally filtered by refresh-time availability, so a provider disappeared exactly when auth, CLI, or provider availability needed explanation.

The dashboard now derives visibility from provider enablement and continues to derive aggregation from token-cost capability plus active-account state. Overview display selection remains a presentation preference; its combined total and share payload are derived from all enabled providers.

Refs #2695, #2290, and #1674. Model canonicalization remains scoped to #2257, and month-to-date semantics remain scoped to #2087.

Native runtime proof

All values and labels below are synthetic. No real account data, keys, or usage values are included.

All-provider Overview at the production 310-point width

CodexBar Overview with combined estimated spend, token total, and six providers

Local share card opened from Overview

CodexBar local share card with six connected services and three-of-six spend coverage

Settings proof:

Current-head evidence

Exact head: 66dcfdc8694c07b713331828639cd751a058101a.

  • all active providers are resolved by default, including a 20-provider regression
  • Overview totals/share use the all-enabled roster even when detail rows are user-filtered or error-only
  • compact rows have separate height-cache identity, explicit unavailable-spend disclosure, and accessibility growth
  • 20-row coarse-wheel input returns to native AppKit scrolling instead of synthetic highlight movement
  • share-card tests cover 6- and 20-service rosters, partial tokens/spend, overflow-safe aggregation, native currencies, and missing provider data
  • independent final code review: no blockers; ready to commit
  • Computer Use verified the production 310-point Overview and the share flow end to end

Validation

  • git diff --check
  • ./Scripts/lint.sh lint-macos (22 locale catalogs and SwiftFormat green)
  • Swift parser checks for every changed Swift file
  • production swift build --product CodexBar green with a temporary local compatibility shim for this Mac's missing SwiftUIMacros.EntryMacro; the shim and proof launcher were removed before commit
  • focused swift test remains toolchain-indeterminate locally because the active CLI toolchain lacks the repository's Testing module
  • exact-head hosted CI is running: https://github.com/steipete/CodexBar/actions/runs/31240622976

Scope

This PR does not claim to solve cross-provider model canonicalization (#2257) or calendar month-to-date reporting (#2087). It establishes the truthful provider roster, longer history, compact Overview, and shareable aggregate needed for those follow-ups.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: needs changes before merge. Reviewed August 8, 2026, 1:55 AM ET / 05:55 UTC.

ClawSweeper review

What this changes

Adds 7-, 30-, and 365-day Usage & Spend windows, fuller provider coverage disclosure, compact Overview/share summaries, and OpenRouter activity history.

Merge readiness

⚠️ Needs maintainer review before merge - 6 items remain

Keep open: strong native UI proof is present, but three prior correctness blockers remain on the current head.

Priority: P2
Reviewed head: 253b48102e19d0e20f4de7fe33b0077e233185f8

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Strong native UI proof and focused tests are offset by three concrete data-semantics blockers.
Proof confidence 🦞 diamond lobster (5/6) ✨ media proof bonus Sufficient (screenshot): Four prepared local native UI captures directly show the 310-point Overview, local share card, and tracked-access behavior at wide and narrow layouts using synthetic data.
Patch quality 🦐 gold shrimp (3/6) 3 actionable review findings remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (screenshot): Four prepared local native UI captures directly show the 310-point Overview, local share card, and tracked-access behavior at wide and narrow layouts using synthetic data.
Evidence reviewed 5 items Shared 30-day consumer: The menu-bar cost token reads the current token snapshot’s field named for the last 30 days directly.
Current-head Overview anchor: The PR head builds Overview with the newest snapshot update time rather than the current date, retaining stale periods as current summaries.
Current-head Settings share call: The Settings share call still supplies only the dashboard model and subscription names, while the new tracked roster is held separately.
Findings 3 actionable findings [P1] Isolate the 365-day dashboard publication
[P2] Use the current date for Overview aggregation
[P2] Include tracked sources in the Settings share payload
Security None None.

How this fits together

Provider usage snapshots feed CodexBar’s Usage & Spend aggregation, then Settings, menu-bar Overview, widgets, and share cards. This PR expands the history data flowing through those shared consumers.

flowchart LR
A[Provider snapshots] --> B[Shared cost snapshots]
B --> C[Usage and Spend dashboard]
B --> D[30-day menu and widget]
B --> E[Overview summary]
C --> F[Settings share card]
E --> G[Overview share card]
Loading

Before merge

  • Isolate the 365-day dashboard publication (P1) - A 365-day dashboard refresh is retained as valid after the override clears, while the menu-bar .cost30d path reads the same aggregate field as a 30-day total. Keep extended dashboard results separate or derive a fixed 30-day value before publication.
  • Use the current date for Overview aggregation (P2) - Using the latest snapshot update time anchors an offline Overview in the past, so stale data is presented as the current reporting window. Build the model with the current capture time instead.
  • Include tracked sources in the Settings share payload (P2) - This call omits the tracked roster, so configured or unavailable providers with no dashboard row disappear from Settings sharing despite appearing in Tracked Access. Pass that roster into the builder.
  • Resolve merge risk (P1) - A 365-day dashboard refresh can be displayed by existing 30-day menu or widget UI as a 30-day total.
  • Resolve merge risk (P1) - Offline Overview can present an old reporting period as current.
  • Resolve merge risk (P1) - Settings sharing can omit sources that the same screen shows as tracked.

Findings

  • [P1] Isolate the 365-day dashboard publication — Sources/CodexBar/UsageStore+TokenCost.swift:112-115
  • [P2] Use the current date for Overview aggregation — Sources/CodexBar/StatusItemController+OverviewSpend.swift:16
  • [P2] Include tracked sources in the Settings share payload — Sources/CodexBar/PreferencesSpendDashboardPane.swift:385-388
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch surface 74 files; +3,038 / -250 lines The change spans provider fetches, shared snapshots, Settings, menu UI, localization, docs, and tests.
Runtime proof 4 native UI screenshots They directly demonstrate the claimed Overview, share-card, and responsive tracked-access layouts.

Merge-risk options

Maintainer options:

  1. Isolate dashboard history (recommended)
    Keep extended dashboard history out of fixed 30-day consumers and add regression coverage for pane closure and stale cache behavior.
  2. Pause the feature
    Defer the long-history and share additions until their shared-snapshot compatibility boundary is redesigned.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Preserve fixed 30-day menu and widget semantics after a 365-day Usage & Spend refresh; use dashboard-local history or derive fixed-window values, anchor Overview to the current date, include tracked sources in Settings sharing, add focused regressions, and do not edit release-owned CHANGELOG.md.

Technical review

Best possible solution:

Use dashboard-local extended-history results, anchor Overview to the current time, and pass the tracked roster into the Settings share payload.

Do we have a high-confidence way to reproduce the issue?

Yes: select 365 days with a configured 30-day history, close the pane, and render a 30-day menu token; the stale Overview and Settings-share omissions also follow directly from current-head source.

Is this the best way to solve the issue?

No: the shared extended-history publication is unsafe for fixed-window consumers; dashboard-local history plus current-time anchoring and roster propagation is narrower.

Full review comments:

  • [P1] Isolate the 365-day dashboard publication — Sources/CodexBar/UsageStore+TokenCost.swift:112-115
    A 365-day dashboard refresh is retained as valid after the override clears, while the menu-bar .cost30d path reads the same aggregate field as a 30-day total. Keep extended dashboard results separate or derive a fixed 30-day value before publication.
    Confidence: 0.99
  • [P2] Use the current date for Overview aggregation — Sources/CodexBar/StatusItemController+OverviewSpend.swift:16
    Using the latest snapshot update time anchors an offline Overview in the past, so stale data is presented as the current reporting window. Build the model with the current capture time instead.
    Confidence: 0.99
  • [P2] Include tracked sources in the Settings share payload — Sources/CodexBar/PreferencesSpendDashboardPane.swift:385-388
    This call omits the tracked roster, so configured or unavailable providers with no dashboard row disappear from Settings sharing despite appearing in Tracked Access. Pass that roster into the builder.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 8fc67d7f04ad.

Labels

Label justifications:

  • P2: Incorrect cost-period reporting misleads normal users but does not block access or availability.
  • merge-risk: 🚨 compatibility: Existing fixed 30-day UI consumes the shared snapshot that the PR refreshes using a longer window.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (screenshot): Four prepared local native UI captures directly show the 310-point Overview, local share card, and tracked-access behavior at wide and narrow layouts using synthetic data.
  • proof: sufficient: Contributor real behavior proof is sufficient. Four prepared local native UI captures directly show the 310-point Overview, local share card, and tracked-access behavior at wide and narrow layouts using synthetic data.
  • proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. Four prepared local native UI captures directly show the 310-point Overview, local share card, and tracked-access behavior at wide and narrow layouts using synthetic data.

Evidence

Acceptance criteria:

  • [P1] make test.
  • [P1] make check.
  • [P1] focused UsageStore, Overview aggregation, and ShareStats payload tests.

What I checked:

Likely related people:

  • steipete: Feature history concentrates on Peter Steinberger across the shared token-publication and dashboard surfaces; the repository owner also requested the rebase and continued review. (role: recent area contributor; confidence: high; commits: d96e1822f856; files: Sources/CodexBar/UsageStore+TokenCost.swift, Sources/CodexBar/PreferencesSpendDashboardPane.swift, Sources/CodexBar/StatusItemController+MenuBarLayout.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Isolate 365-day dashboard results from fixed 30-day consumers.
  • Anchor Overview to the current time and preserve the tracked roster in Settings sharing.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (27 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-05T16:39:39.652Z sha 926f1d1 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-05T16:56:48.831Z sha 1320f07 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-05T17:23:35.603Z sha 1320f07 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-08T03:08:27.350Z sha 016c1f9 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-08T03:31:12.317Z sha 5896570 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-08T03:51:02.128Z sha 5896570 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-08T04:51:42.972Z sha dbd547f :: needs maintainer review before merge. :: none
  • reviewed 2026-08-08T05:03:29.972Z sha 66dcfdc :: needs changes before merge. :: [P1] Keep dashboard history out of 30-day snapshots | [P2] Anchor Overview windows to the current date | [P2] Preserve tracked access in the Settings share payload

@Chipagosfinest
Chipagosfinest force-pushed the agent/cumulative-spend branch from 1e35ae0 to 4b5b985 Compare July 28, 2026 04:02
@Chipagosfinest
Chipagosfinest force-pushed the agent/cumulative-spend branch from aad4afc to b85cd49 Compare July 28, 2026 04:08
@Chipagosfinest
Chipagosfinest marked this pull request as ready for review July 28, 2026 04:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b85cd4997c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/CodexBar/SpendDashboardController.swift Outdated
@Chipagosfinest
Chipagosfinest force-pushed the agent/cumulative-spend branch 3 times, most recently from b1ba0c7 to cad600b Compare July 28, 2026 04:46
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. label Jul 28, 2026
@Chipagosfinest
Chipagosfinest force-pushed the agent/cumulative-spend branch 4 times, most recently from bb9d74e to 27185a6 Compare July 28, 2026 07:31
@clawsweeper clawsweeper Bot removed the merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. label Jul 28, 2026
@Chipagosfinest
Chipagosfinest force-pushed the agent/cumulative-spend branch 4 times, most recently from 2c8715e to 8a5b6f4 Compare July 29, 2026 03:02
@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 29, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 29, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jul 29, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Aug 5, 2026
@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 5, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Aug 8, 2026
@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

@codex review

@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dbd547ff0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +112 to +116
let requiredHistoryDays = self.settings.effectiveCostUsageHistoryDays
guard publication.historyDays >= requiredHistoryDays,
publication.scopeSignature == self.tokenSnapshotScopeSignature(
for: provider,
historyDays: publication.historyDays)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep dashboard history out of 30-day snapshots

When a user selects 365 days in Usage & Spend while their configured cost history is 30 days, the shared publication becomes a 365-day snapshot; after the pane closes, this >= check continues treating it as valid for 30-day consumers. menuBarLayoutCostStrings then passes last30DaysCostUSD to the .cost30d layout token, and the widget has similar 30-day presentation paths, so the annual total can be displayed as a 30-day cost until another refresh replaces it. Either isolate the dashboard's extended history or derive fixed-window values before feeding those consumers.

Useful? React with 👍 / 👎.

snapshot: snapshot)
}
let requestedDays = self.settings.effectiveCostUsageHistoryDays
let now = inputs.map(\.snapshot.updatedAt).max() ?? Date()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Anchor Overview windows to the current date

When cached publications are stale, using the latest snapshot's updatedAt anchors the Overview's “last N days” window in the past rather than today. For example, an offline user with a month-old cached snapshot can still see that old period's spend presented as the current 30-day summary; the main dashboard correctly anchors retained data to the request time. Build this Overview model with the current capture time so expired entries fall outside the displayed window.

Useful? React with 👍 / 👎.

model: SpendDashboardModel,
subscriptionNames: [String: ShareStatsSubscriptionName] = [:]) -> ShareStatsPayload?
subscriptionNames: [String: ShareStatsSubscriptionName] = [:],
providerRoster: [ShareStatsProviderRosterEntry] = []) -> ShareStatsPayload?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include tracked access in the Settings share payload

When Settings → Usage & Spend shows a configured or enabled provider that has no dashboard model row—for example Gemini, Grok, or an unavailable key—the pane's ShareStatsBuilder.make call omits providerRoster, so this empty default silently drops that source from the resulting card even though the adjacent Tracked Access panel keeps it visible. Pass the pane's tracked roster into the builder so its Share Stats action preserves the full roster just as the new Overview share action does.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. labels Aug 8, 2026
@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. labels Aug 8, 2026
@Chipagosfinest

Copy link
Copy Markdown
Contributor Author

Conflict-free successor opened as #2759 after #2758 moved OpenRouter onto the portable QuickJS/JavaScriptCore plugin architecture. I preserved this PR's history instead of force-rewriting the branch; active review and exact-head CI are continuing on #2759.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants