External power-cost config for local/subscription models

[Pauliehedron]

TokenTelemetry's pricing table in pricing.py maps model names to per-1M-token USD rates. That works for API calls. But not every session fits that model:

• Local models (ollama, llama.cpp, vLLM) have no API cost — the cost is electricity
• Subscription access (Ollama Cloud Pro at $20/mo) is flat-rate, not per-token
• Proxied endpoints (nginx -> ollama.com) are the same subscription, just routed differently

A fixed lookup table can't express any of these. An external config file can.

The Pkl schema:

module tokentelemetry.PowerConfig

class PowerConfig {
  /// Power draw at full inference load (watts) — GPU, total system, or both
  loadWatts: Int = 80

  /// Electricity cost per kWh in your currency
  costPerKwh: Float = 0.15

  /// Endpoints covered by a flat subscription (monthly/yearly, not per-token).
  /// Sessions hitting these get cost=0 — the subscription cost is tracked separately.
  subscriptionEndpoints: Listing<String> = new Listing {
    "https://ollama.com/v1"
  }
}

config: PowerConfig = new PowerConfig {
  loadWatts = 80
  costPerKwh = 0.15
  subscriptionEndpoints = new Listing {
    "https://ollama.com/v1"
    "http://127.0.0.1:3000/v1"
  }
}

Evaluated to JSON for TT:

{
  "loadWatts": 80,
  "costPerKwh": 0.15,
  "subscriptionEndpoints": ["https://ollama.com/v1", "http://127.0.0.1:3000/v1"]
}

Per-session math:

For local sessions: (output_tokens / real_tokens_per_second) * loadWatts / 3_600_000 * costPerKwh

Token rate is computed from the agent's own logs (out=NNN latency=N.Ns per API call), not a hardcoded estimate. Sessions hitting a subscriptionEndpoint get cost=0.

How it plugs in:

Two implementation options, same config shape:

1. In-TT (upstream change): Load power.json alongside existing configs, branch in calculate_cost() for models with no API rate. ~10 lines of Python.

2. Companion tool (no TT change): A standalone script that reads the power config, queries the agent's session database, computes per-session cost, and writes it back. TT picks it up on next refresh. Works if the agent stores session cost in a database TT already reads.

Why external config beats a fixed table:

Scenario	Fixed table	External config
Local model inference	Fallthrough to $0 or wrong API rate	True electricity cost
Flat subscription (Ollama Pro, etc.)	Priced at per-token rates	Correctly $0 — cost tracked separately
Multiple proxy paths to same service	Each path needs a table entry	Single subscriptionEndpoints list
User upgrades hardware (80W→150W GPU)	Requires source code change	Edit one config value
Electricity rate changes	Requires source code change	Edit one config value

Three-way coverage:

User Type	What They Interact With	Tooling Needed
Newcomer	Shipped power.json with defaults	None
Power user	power.pkl with types, docs, defaults	Pkl CLI
Everyone else	power.json with a text editor	None

The upstream would ship both config/power.pkl (authored source, typed, documented) and config/power.json (pre-rendered, ready to use). Zero new dependencies for TT — it already reads JSON configs. Pkl stays optional for users who want it.

No code submission — just the design artifact and the pattern.

---

Variant: Isolated Cost Cache (DuckDB intermediate)

If writing to the agent's internal database is too invasive for upstream adoption, an intermediate cache decouples the cost computation from the source of truth entirely.

Architecture:

agent.log ──► tt-power-cost ──► power-cost.duckdb    ← standalone, ACID, no schema coupling
                                       ↕ (TT reads both)
agent.state.db ────► TokenTelemetry

Instead of tt-power-cost writing estimated_cost_usd directly into Hermes' state.db, it writes to its own DuckDB file (~/.tokentelemetry/power-cost.duckdb). TokenTelemetry reads the cost from DuckDB in addition to state.db — a two-source join unified in the dashboard.

Why DuckDB fits the requirement:

Requirement	Property
Safe concurrent writes	ACID — cron crash leaves prior state intact, not corrupted
Decoupled schema	Companion tool owns its columns; no coupling to the agent's internal schema
Trivial upstream read	TT already reads a database (state.db). A second DuckDB file at a known path is the same pattern — open, query, close
Optional, graceful	If the file is missing, TT falls through to existing API-rate cost — zero regression

The job for the companion tool: Owns the entire computation — reads the power config, parses agent logs for real token rates, writes per-session cost to the DuckDB file. The upstream doesn't compute anything; it just enriches a session view by checking a second source.

The job for the upstream: Optionally read a second database file. If present, prefer its estimated_cost_usd over the API-rate fallback. That's it — ~2 lines of conditional logic, no new dependencies (DuckDB Python module is already common in the ecosystem, but you could also ship a CLI wrapper if you prefer zero dep overhead).

How this compares to direct state.db write:

Aspect	Direct write to state.db	DuckDB intermediate
Safety	External tool mutates Hermes internals	Hermes state.db untouched
Schema coupling	Must match state.db exactly	Independent schema
Durability	Crash mid-write could corrupt	ACID — atomic commits
Upstream ask	"Modify agent's DB schema"	"Read sidecar DuckDB file"
TT adoption cost	High — coordination with Hermes team	Low — 2 lines of optional Python
Queryability	Only cost field	DuckDB can cache rates, profiles, aggregates

---

Idea drafted with assistance from Ensemble (ENSEMBLE Framework v3.3.3 protocol on Hermes Agent). Accountable human: @Pauliehedron, Conductor

[Pauliehedron]

And as I sit here looking at this some more:

Bonus — same DuckDB file could replace pricing.py entirely. Ship an auto-updater script that fetches the latest per-model API rates (from OpenRouter, provider docs, or a canonical source) and writes them into the same DuckDB under a pricing table or namespace. The hardcoded lookup table in pricing.py goes away — the DuckDB is the single source for all costs: API rates from the updater, power costs from the companion tool, subscription costs from the config. The maintainer maintains one update script instead of chasing model price changes by hand.

And the power config itself (loadWatts, costPerKwh, subscriptionEndpoints) could live in TT's web settings panel — same pattern as your OpenAI-compatible backend config. No external file needed for the primary path. The Pkl schema still works as a source-of-truth definition for the shape, and power users who want file-based config can use the companion tool instead of the web UI.

No DB, you have your current defaults.

Comment drafted with assistance from Ensemble (ENSEMBLE Framework v3.3.3 protocol on Hermes Agent). Accountable human: @Pauliehedron, Conductor

[Pauliehedron]

Broader implication — TT owns its own data. Projects, session groupings, cost enrichment, deduplication — anything that today depends on Hermes' internal state.db schema and its quirks lives in TT's own DuckDB instead. If Hermes changes its DB schema or a session has weird metadata, TT doesn't care — it reads from its own sidecar, populated by its own tools on its own schedule. The Hermes source of truth is a raw feed; the DuckDB is TT's curated view.

Comment drafted with assistance from Ensemble (ENSEMBLE Framework v3.3.3 protocol on Hermes Agent). Accountable human: @Pauliehedron, Conductor

[Pauliehedron]

To give an example of my implementation, where I am writing directly to my hermes sqlite, I get artifacts like a "free" model showing in the stats with a cost. So I'll have to go back and adjust that and update the local database. (note: I can just set all the cost/usage data back to zero, then re-parse the agent.log files to undo things but doing it in a separate duckDB sounds more appealing the more I think about it)

Models
moonshotai/kimi-k2.6:free
1 · $1.04
deepseek-v4-flash:cloud
185 · $0.46
deepseek/deepseek-v4-flash
1 · $0.18
unsloth/Qwen3.6-35B-A3B-MTP-GGUF:IQ4_NL
88 · $0.07
deepseek-v4-flash
104 · $0.06
unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL
172 · $0.06
qwen3.6-35b-mtp
20 · $0.03
unsloth/Qwen3.6-27B-GGUF:IQ4_NL
1 · $0.0082

I have ollama-cloud and openrouter.ai as cloud model handlers as well as llama-swap which can do proxy to the cloud as well, and I do, because within hermes agent the aux handlers have no failover config today, and I have tested specific models to do particular tasks where they excel, (gpt-oss-20b for some and gpt-oss-120b for others, they are not interchangable, they each to a particular job better than the other for the same task).

I know in my installation, models that follow the author/model:quant format are always local because I use huggingface cache for my model handling, not individual files, and I standardized on naming conventions that follow the cache as it is unique; as it feeds other things like my benchmarking tooling and llama-swap config. I have added complexity, so I rely on process and standards and common configuration language to keep things orderly.

[VasiHemanth]

Update — what shipped, and the roadmap to fully close this

Shipped (PR #74): the core of this proposal is now implemented in-TT:

• ~/.tokentelemetry/power.json with loadWatts / costPerKwh / subscriptionEndpoints (the JSON schema from this thread)
• Local models priced by the electricity formula (output_tokens / tok_per_sec) × loadWatts / 3_600_000 × costPerKwh
• Flat-subscription endpoints → $0/call
• GET/PUT /config/power

Validated end-to-end on a local Ollama nemotron-3-nano:4b session via OpenCode.

Being upfront about what is not done yet — tracked for the next PR (feat/local-power-insights, Phase 1):

1. Measured throughput. tok_per_sec is currently a fixed 30. This thread rightly asked for it to come from each agent's own logs (out=NNN latency=N.Ns). Hermes already records per-call latency, so we'll compute real tok/s there, derive it from message timestamps where available elsewhere, and fall back to a per-model default (a 4B ≠ a 30B) before the global default.
2. Real power instead of a guessed wattage. nvidia-smi gives real draw with no root on NVIDIA. On Apple Silicon there is no no-root path — powermetrics is root-only and battery readings are ~0 on AC — so there it'll be an opt-in, user-consented powermetrics/battery-discharge/calibration flow. Every figure will carry its source + confidence ("measured via nvidia-smi" vs "estimated").
3. Local detection by endpoint/provider, not "model isn't in the price table" — fixes name collisions (a local llama-3.3-70b must not get cloud pricing) and wires into the new per-agent local billing mode.
4. Settings UI for the power config (your web-panel suggestion, @Pauliehedron).

Insights layer (Phase 2, planned): energy (Wh/kWh) per session/day, savings-vs-cloud ("locally $0.0002; ~$0.05 on Claude Sonnet"), a per-model tok/s efficiency leaderboard, and an opt-in CO₂ estimate (kWh × grid intensity) — on a dedicated Local & Power view.

Design principle we're adopting (Phase 3): as TT grows these insights, it must stay observably cheap on your machine — heavy features opt-in, low-duty sampling only during active sessions, on-demand not continuous. We plan to dogfood the power feature on TT itself ("TokenTelemetry used ~X Wh today").

On Pkl & the DuckDB sidecar: we took the in-TT path. Pkl we'll ship as an optional typed-source artifact; the DuckDB intermediate we're consciously not adopting (the in-TT read is simpler for users) — flagging that as a decision, not an oversight, happy to revisit if there's appetite for the companion-tool model. And the "one update script instead of chasing prices by hand" idea is now live via build-time models.dev pricing (PR #74). Thanks for the thorough write-up.

[Pauliehedron]

I'm excited to see what you adapted! I offer suggestions, you know you code and path, so always pick what you like which works for you. Thanks for considering and implementing what you found useful!

Edit: Removed all my local side car pieces, fully upstream now. Thanks again!

[Pauliehedron]

A couple other items which you may find helpful. Hermes Agent how has _OFFICIAL_DOCS_PRICING, which I am starting to see pop up in issues, as well as right in the TUI console with a money counter. So that may be something you could potentially leverage for the hermes agent dashboard you have?

And for getting hardware info from macOS, there is a way to get some info via userspace so may fill the gap you noticed for power, perhaps to input a base number that the user could then override (like I just use gpu watts in my simple calculation because I use my PC for other things that are CPU related), see: mostlygeek/llama-swap#814

[Pauliehedron]

Dug a little deeper on the hermes agent side:

│ Heads up — Hermes Agent just shipped agent/usage_pricing.py (~905 lines). It has a _OFFICIAL_DOCS_PRICING dict with ~50+ hardcoded model rates (Anthropic, OpenAI, Minimax), an OpenRouter live-pricingfetcher, and endpoint /v1/models probing. It writes estimated_cost_usd into state.db per session. TT reads it natively (line 2768) and only falls through to its own cost compute when bothactual_cost_usd and estimated_cost_usd are NULL. No conflict — just means TT's local power cost path only activates for sessions Hermes doesn't know the pricing for.

Comment drafted with assistance from Ensemble (ENSEMBLE Framework v3.3.3 protocol on Hermes Agent). Accountable human: @Pauliehedron, Conductor

[ddiall]

TL;DR: Anthropic clarified their new policy for Claude subscriptions applicable from June 15, it confirms the exact multi-source billing complexity PowerConfig targets:

1. Same provider, three distinct credit buckets with explicit drain priority:

• Agent SDK credit ($20/mo Pro) → claude -p, SDK, GitHub Actions (drains first)
• Subscription plan limits → interactive Claude Code, Cowork (separate, no spillover)
• Usage credits / API key → pay-as-you-go (drains after SDK credit)

2. subscriptionEndpoints pattern proposed by OP maps perfectly: The SDK credit acts like a "subscription endpoint" for programmatic use — cost=0 per session until monthly credit exhausted.

3. Critical nuance: No fallback between buckets. Interactive limits ≠ SDK credit. This means routing logic must be task-aware (interactive vs programmatic), not just credit-aware.

4. My preference: Push this complexity into TokenTelemetry upstream (PowerConfig + drain priority in calculate_cost()) rather than building custom routing layers. The schema you've drafted seems to handles it — just needs the drain-order logic and task-type awareness.

Kudos @VasiHemanth on shipping PR #74 and keep 'em coming ;-)

[VasiHemanth]

Follow-up — the Phase 1 items I'd flagged as "not done yet" are now shipped ✅ (PR #100)

Closing the loop on my last update, without repeating it:

• Measured throughput — tok/s now comes from each agent's own latency logs instead of the fixed 30, so the electricity estimate reflects how your machine actually ran.
• Per-chip default wattage — Apple Silicon machines auto-fill a sensible default; hardware we can't detect now asks you to enter your draw under load (or click Measure) instead of silently presenting a generic 80 W.
• @ddiall's multi-bucket drain priority — TT now models each agent's credit buckets and drain order, split by interactive vs programmatic use, including Anthropic's June-15 Agent-SDK pool and its "no fallback" behavior. Expand any agent under Settings → Billing to see the drain order, pool sizes and stop-at-$0 warnings.

Still open (tracked): auto-detecting wattage on non-Apple hardware (AMD/Intel/Windows), and live "pool % drained" meters — which need per-session interactive-vs-programmatic tagging to fill.

[awdemos]

Great idea. The fixed lookup table vs. external config distinction is exactly the right framing.

One extension worth considering: for subscription/flat-rate endpoints, rather than showing cost=0 (which understates the true spend), show a "breakeven tokens" metric. Given the flat monthly price, compute how many tokens the session would have needed to cost on a per-token API to equal the subscription. That lets users see when a flat environment is winning vs. when they are over-provisioned.

We run UltraWork on a flat monthly GPU box for exactly this comparison (https://vibecodingagency.com/gpu-cloud/), so this kind of TCO math is the first thing teams ask for. Disclosure: I help run Vibe Coding Agency. Happy to share the breakeven formula we use if useful.