-
Notifications
You must be signed in to change notification settings - Fork 2
plat 203
PLAT-203 — pi-cli-routed models had no rate card at all, and a genuinely-zero cost was indistinguishable from "we don't know"
| Coordination | Value |
|---|---|
| Assigned agent | unassigned |
| Ticket state | fixed |
| Last synchronized | 2026-08-29 |
- Priority: P2 — cost figures silently absent (not zero) for every pi-cli-routed call, which "could mislead operations review into under-counting cycle cost for any step pinned to a non-Claude model" per the finding's own impact assessment.
-
Owner:
agent_go/pkg/orchestrator/base_orchestrator_tokens.go(calculatePricingFromModelData),agent_go/pkg/orchestrator/cost_storage.go(buildModelTokenUsage,EnsureModelTokenUsagePricing),agent_go/pkg/orchestrator/base_orchestrator_types.go(ModelTokenUsage). -
Related:
harness:cost-ledger:reflection-attribution-and-call-count(confida-login, medium;classification: "no_issue",recommended_route: "evidence_wait"in the harness's own self-assessment — this ticket found it was in fact a real, confirmed gap). Same class of zero-vs-unset ambiguitycostledger.Aggregate(the newer, parallel cost-tracking system — see PLAT-184) already solved viaBillingBasis="unpriced"/UnpricedCallCount; this ticket applies the same pattern to the older, still-liveModelTokenUsage/cost_storage.gosystem the finding's cited file (costs/execution/<workflow>/<date>.json) actually uses.
costs/execution/confida-staging/2026-08-24.json's by_step_and_model for
execute-browser-and-capture-apis showed google/gemini-3.7-flash with
llm_call_count=4 present but no total_cost_usd key at all in that
model block.
-
pi-cli has no rate card at all, for any model.
picli.PiCLIAdapter.GetModelMetadata(multi-llm-provider-go/pkg/adapters/picli/picli_adapter.go:90-104) returns*llmtypes.ModelMetadatasuccessfully (no error) but never sets any*CostPer1MTokensfield — not specific togemini-3.7-flash, every model this adapter serves gets the same empty pricing. Confirmedgemini-3.7-flashis in fact the adapter's own current default model (DefaultModelID,picli_adapter.go:13), so this isn't a stale/missing entry for an unusual model — it's a structural gap for the whole provider. -
A genuinely-zero cost and an unknown cost were indistinguishable.
ModelTokenUsage.TotalCosthadjson:"total_cost_usd,omitempty"— whencalculatePricingFromModelDatafell through to all-zero costs (metadata missing, or present but empty as in case 1), the JSON key simply vanished, identical to what a genuinely free call would produce.
calculatePricingFromModelData now returns a pricingFound bool: true
only when the resolved model metadata actually carries a nonzero rate on at
least one of Input/Output/Reasoning/CachedInput/CachedInputWrite
cost-per-1M-token fields — not merely "metadata lookup didn't error."
ModelTokenUsage gained Unpriced bool (omitempty — a priced call's JSON
is byte-identical to before this change) and TotalCost lost its
omitempty (a priced call already always has a real total; an unpriced
call now explicitly shows "total_cost_usd":0,"unpriced":true" instead of
a missing key). Both call sites (buildModelTokenUsage,
EnsureModelTokenUsagePricing) set Unpriced from pricingFound and log
a [COST] line naming the exact provider/model with no rate card, so a
recurrence is immediately debuggable instead of silent.
- Did not add real per-token pricing for
google/gemini-3.7-flashor any other pi-cli model. That requires knowing pi-cli's actual billing model (subscription passthrough vs. metered vs. genuinely free tier) with confidence — guessing a dollar figure would be worse than an honest "unpriced" marker, since a plausible-looking wrong number is harder to catch than an explicit unknown. This is now a well-scoped follow-up if pi-cli's billing terms are confirmed. - Did not change the update-condition in
EnsureModelTokenUsagePricing(still gates writing pricing fields oninputCost>0 || ... || totalCost>0, not onpricingFound) — changing that would also affect a real rate-card model whose token counts happen to be zero, which no existing test covers; scoped this change to exactly what the finding needed. - Did not touch
costledger/costobserver(the newer, parallel system) — it already has its own equivalentBillingBasis/UnpricedCallCountdisambiguation and was not what the finding's cited file used.
-
go build ./...clean. - New tests
TestUnpricedProviderCallsAreExplicitNotAbsentandTestPricedProviderCallsAreNotMarkedUnpriced(pkg/orchestrator/unpriced_model_cost_test.go) prove: a pi-cli/gemini call is markedpricingFound=false,Unpriced=true, and its JSON explicitly shows"total_cost_usd":0,"unpriced":true; a realclaude-sonnet-5call keepspricingFound=true,Unpriced=false, a real nonzero total, and — critically — nounpricedkey at all in its JSON, so every existing priced-call consumer sees a byte-identical payload to before this change. - Existing
TestPhasePricingUsesImmutableClaudeModelRateCards(Claude rate-card exactness) passes unchanged. - Full suite: 3 pre-existing failures before and after (confirmed via
git stash), all incmd/server/guidance, none touching this ticket's packages — no regression.
Auto-synced from docs/ on main. Edit there, not here.