fix: Anthropic adapter bugs (#28568, #28562, #28576, #28580)#1
Open
openbot-coder wants to merge 2 commits into
Open
fix: Anthropic adapter bugs (#28568, #28562, #28576, #28580)#1openbot-coder wants to merge 2 commits into
openbot-coder wants to merge 2 commits into
Conversation
, BerriAI#28580) ## Bug Fixes ### Bug BerriAI#28576: is_thinking_enabled crashes when thinking=None - Guard against non-dict thinking value in is_thinking_enabled - Prevents AttributeError when thinking key exists with None value ### Bug BerriAI#28568/BerriAI#28562: Spend log ID mismatch for non-Anthropic backends - Extract litellm_call_id from response._hidden_params or kwargs - Use msg_<litellm_call_id> as Anthropic response ID for spend log correlation - Affects both streaming and non-streaming paths, sync and async handlers ### Bug BerriAI#28580: Anthropic prefill semantic lost on hosted_vllm - Auto-stamp prefix:true for trailing assistant messages in adapter handler - HostedVLLMChatConfig.transform_request translates prefix to continue_final_message ## Tests - Add comprehensive regression tests for all bug fixes - Test coverage includes all new code paths in modified files
1cb5ed5 to
3bd29ed
Compare
Fix Black formatting in 3 files that caused CI lint failure: - handler.py: line length wrapping for conditionals and method calls - streaming_iterator.py: compact msg_ format string - test_anthropic_adapter_bug_fixes.py: method defs and blank lines Co-Authored-By: Craft Agent <agents-noreply@craft.do>
2bda3f1 to
8153acd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes 4 critical Anthropic adapter bugs in LiteLLM:
Bug BerriAI#28576: is_thinking_enabled crashes when thinking=None
non_default_params.get("thinking", {}).get("type")crashes whenthinkingkey exists withNonevalue.get("type")litellm/llms/base_llm/chat/transformation.pyBug BerriAI#28568/BerriAI#28562: Spend log ID mismatch for non-Anthropic backends
chatcmpl-abc)msg_<uuid>litellm_call_id(different UUID)litellm_call_idfrom response/kwargs and use it as the response ID withmsg_prefixlitellm/llms/anthropic/experimental_pass_through/adapters/handler.py(extract litellm_call_id)litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py(use litellm_call_id as response ID)litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py(use litellm_call_id for streaming message IDs)Bug BerriAI#28580: Anthropic prefill semantic lost on hosted_vllm
continue_final_message: true, breaking prefill entirelyprefix:truefor trailing assistant messages in adapter handlercontinue_final_messageandadd_generation_promptflagslitellm/llms/anthropic/experimental_pass_through/adapters/handler.py(auto-stamp prefix)litellm/llms/hosted_vllm/chat/transformation.py(add continue_final_message support)Changes Summary
Testing