Fix #373: Bedrock pricing keys never match live spans — normalize provider + model id in get_rates#385
Merged
Merged
Conversation
Every [aws.*] entry in models.toml was dead for live traffic: Bedrock
spans carry provider "aws.bedrock" (direct boto3) or "bedrock"
(LiteLLM-routed) and the raw boto3 modelId ("us.amazon.nova-micro-v1:0",
dots and trailing ":0" intact), while the table keys are provider "aws"
with dot-flattened, unversioned model names. get_rates() missed on both
axes and every Bedrock call silently fell back to the $0.50/$2.00
default rate.
Fix inside get_rates() so one home covers all ingest paths (direct
boto3, LiteLLM-Bedrock, OTLP HTTP): alias aws.bedrock / aws_bedrock /
bedrock -> aws for the table lookup, and for that provider also try the
modelId in table-key form (strip a "bedrock/" prefix and a trailing
":<digits>" version, flatten dots to hyphens). The normalization is
lookup-only -- stored spans keep the raw provider/model strings, and
models.toml is unchanged; the existing #366/#370 keys simply become
reachable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015e3DZGhb1R5sXX2x52Y2Ar
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every
[aws.*]pricing entry — the original Nova rows plus the rates added in #366/#370 — was unreachable for real Bedrock traffic, so every Bedrock call silently billed at the $0.50/$2.00 default. This makes those keys live by normalizing inside the rate lookup.Closes #373
Summary
get_rates()now aliases provideraws.bedrock/aws_bedrock/bedrock→ the table'sawskey for the provider-keyed lookup.bedrock/prefix (LiteLLM-routed form), strip a trailing:<digits>version, flatten dots to hyphens (us.amazon.nova-micro-v1:0→us-amazon-nova-micro-v1).provider/modelstay raw,models.tomlis untouched.Symptom, root cause, fix
Symptom: all Bedrock spans cost out at the default flat rate; the
[aws.*]table entries never match.Root cause: two simultaneous mismatches. Live spans carry
provider = "aws.bedrock"(set verbatim byBedrockIntegration) or"bedrock"(LiteLLM), andmodel = "us.amazon.nova-micro-v1:0"(raw boto3 modelId). The table keys are[aws.us-amazon-nova-micro-v1]— provideraws, dots flattened,:0stripped.get_rates("aws.bedrock", "us.amazon.nova-micro-v1:0")looked undertable["aws.bedrock"], missed, and returnedNone.Fix: normalize in
get_rates(tokenjam/core/pricing.py) per the issue's preferred approach — one home covering every ingest path (direct boto3, LiteLLM-Bedrock, OTLP HTTP), slotted alongside the existing_strip_date_suffixfallback. The normalized model key is tried only after the exact and date-stripped forms miss, and only when the resolved provider isaws, so no other provider's lookup changes. The:Nstrip is anchored (:\d+$) — only a trailing all-digit version segment is removed; mid-string dates in Anthropic-on-Bedrock ids (...-20250805-v1) survive intact. No cache changes — the LRU-cached layers are untouched; only the post-load lookup logic changed.Tests / Verification
tests/unit/test_cost.py(whereget_ratestests live — there is notest_pricing.py): 6 new tests coveringget_rates("aws.bedrock", "us.amazon.nova-micro-v1:0")→ packaged Nova Micro rate (0.035/0.14, non-default)"bedrock"and the LiteLLMbedrock/us.amazon...model form:Nstrip applies only to a trailing:digits, not:latestor mid-string colonsanthropicstill resolves;openai+ a Bedrock id still misses; unknown pair stillNone)python3 -m pytest tests/unit/ -q→ 1313 passed;tests/integration/→ 344 passedruff checkandmypyclean on both touched filesWhat's NOT in this PR
billing_accountderivation — untouched by design; the issue explicitly scopes this out (provider_to_billing_accountalready maps Bedrock correctly).models.toml— no key changes; the issue's premise is that the existing Add Bedrock Nova Micro/Premier, Anthropic-on-Bedrock, and LiteLLM provider pricing entries #366/fix: Claude Code cost & alert correctness (4/5 from #306) #370 keys become live once the lookup normalizes.convert_otel_span/otlp_parsing) — the issue considered and rejected it (two edit sites, rewrites what users see in traces).🤖 Generated with Claude Code
https://claude.ai/code/session_015e3DZGhb1R5sXX2x52Y2Ar