Skip to content

fix(hermes): attribute mixed-session usage by model - #170

Merged
pitimon merged 1 commit into
mainfrom
fix/169-hermes-mixed-model-usage
Aug 6, 2026
Merged

fix(hermes): attribute mixed-session usage by model#170
pitimon merged 1 commit into
mainfrom
fix/169-hermes-mixed-model-usage

Conversation

@pitimon

@pitimon pitimon commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • read Hermes session_model_usage when available and aggregate usage by (session, model) across billing/task dimensions
  • track incremental snapshots independently per session and model
  • use each model's authoritative last_seen timestamp for bucket placement
  • preserve the legacy aggregate-session parser for older Hermes databases
  • baseline existing legacy cursors once to avoid replaying lifetime totals during upgrade
  • add mixed-model Sol/Terra regression coverage with later Terra-only growth
  • bump release metadata to 0.39.48

Root cause

Hermes records mixed-model sessions in session_model_usage, while TokenTracker only read aggregate counters plus the single sessions.model value. Tokens produced after an in-session model switch were therefore counted but attributed to the original/default model.

Verification

  • regression failed before implementation (eventsAggregated 1 instead of 2)
  • mixed-model initial totals split correctly
  • later Terra growth emits only under Terra
  • older Hermes schema fallback tests pass
  • named profile isolation tests pass
  • live legacy-cursor dry run establishes per-model snapshots with zero replay/queue writes
  • all Hermes-focused tests pass
  • npm run ci:local passes
  • version lockstep, OpenWiki, pack/publish dry run, diff checks pass

Migration note

Existing aggregate cursors are baselined once to avoid double-counting. Previously misattributed historical tokens remain in their old model bucket; new per-model growth is attributed correctly after adoption.

Closes #169

@pitimon
pitimon force-pushed the fix/169-hermes-mixed-model-usage branch 2 times, most recently from 185ac4f to 9e7e976 Compare August 6, 2026 00:17
Read session_model_usage when available, track cumulative snapshots per session and model, and preserve legacy cursors without replaying lifetime totals.
@pitimon
pitimon force-pushed the fix/169-hermes-mixed-model-usage branch from 9e7e976 to 72511ef Compare August 6, 2026 00:25
@pitimon
pitimon merged commit f62ccc0 into main Aug 6, 2026
1 check passed
@pitimon
pitimon deleted the fix/169-hermes-mixed-model-usage branch August 6, 2026 00:29
@pitimon

pitimon commented Aug 6, 2026

Copy link
Copy Markdown
Owner Author

Release and live verification complete:

  • published npm @ipv9/tokentracker-cli@0.39.48 (sha1 a1438e9e2d545382d86947263eb086624f2188db)
  • GitHub release v0.39.48 with macOS/Windows assets
  • local dashboard and sync LaunchAgents pinned to 0.39.48; plist lint clean; processes healthy; sync exit 0; HTTP 200
  • live source session_model_usage -> per-model cursor -> queue -> API -> Day UI verified
  • Day UI now shows HERMES 2 models: gpt-5.6-sol 99.4% and gpt-5.6-terra 0.6%
  • live API showed Terra 1,125,246 tokens and $0.767189 for 2026-08-06 at verification time

Follow-up publish snapshot PR #171 merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]: Hermes mixed-model sessions are attributed entirely to sessions.model

1 participant