Fix Anthropic model-info lookup for AWS Bedrock model IDs#7940
Fix Anthropic model-info lookup for AWS Bedrock model IDs#7940Functionhx wants to merge 3 commits into
Conversation
get_info and get_token_limit used model.startswith(prefix) to match table keys. Bedrock model IDs carry a provider prefix (us.anthropic., eu.anthropic., etc.) and a -vN:M version suffix that prevent the prefix match from ever succeeding, so every Bedrock ID falls through to the KeyError/default path. Fix: add a shared _normalize_model_id helper that strips Bedrock prefixes and -vN:M version suffixes before the lookup. Use it in both get_info and get_token_limit so Bedrock IDs resolve to the same table entries as their canonical Anthropic equivalents. Test Plan: Verified with standalone regex tests covering all Bedrock prefixes (us., eu., apac., global., bare anthropic.) and version suffixes. - us.anthropic.claude-3-5-sonnet-20240620-v1:0 -> claude-3-5-sonnet-20240620 - anthropic.claude-3-5-sonnet-20240620-v1:0 -> claude-3-5-sonnet-20240620 - canonical IDs pass through unchanged Run via: python -c "import re; ..." with assert-based checks. Fixes microsoft#7833 Signed-off-by: Yuchen Fan <functionhx@gmail.com>
Before the Bedrock normalize fix, Bedrock callers never reached the token-limit lookup path. Now they do, and Claude 4 models fell through to the 100000 default instead of the correct 200000. Add claude-opus-4-20250514, claude-sonnet-4-20250514, and claude-3-7-sonnet-latest (all 200k context). microsoft#7833 Signed-off-by: Yuchen Fan <functionhx@gmail.com>
claude-opus-4-0 and claude-sonnet-4-0 are "latest" aliases in _MODEL_INFO. Without entries here, get_token_limit for these IDs depends on a brittle partial-match heuristic that may not always hit the right table row. microsoft#7833 Signed-off-by: Yuchen Fan <functionhx@gmail.com>
|
@microsoft-github-policy-service agree |
|
This fixes the same root cause as #7930 (also open, touching the same file/functions) — worth the maintainers' attention that these overlap before merging either. I compared the two approaches directly: for every Bedrock region prefix documented today ( Not saying this one is wrong — the added Claude 4 token-limit entries here are a real, useful addition either way — just flagging the duplication so it does not land twice. |
2 participants
Added _normalize_model_id helper to strip Bedrock region prefixes and -vN:M version suffixes before model info/token limit lookup. Also added missing Claude 4 entries to _MODEL_TOKEN_LIMITS.
Checks