Skip to content

[Bug]: Using reviewer agent (AssistantAgent) with claude-3-5-sonnet-20240620 crashes #3162

Closed
@gilko1981

Description

@gilko1981

Describe the bug

The agent fails only when configured to llm=claude-3-5-sonnet-20240620. when using other configs, such as gpt-4o the Agent functions as expected. When assigning the claude-3-5-sonnet-20240620 model to a different agent, the agent functions as expected.

When using the reviewer agent (AssistantAgent) with claude-3-5-sonnet-20240620 crashes.
I have a simple group chat, round-robin - [analyzer, analyzer_assistant, executor, reviewer]
When configuring the reviewer agent with claude-3-5-sonnet-20240620, the execution crashes with:

raise self._make_status_error_from_response(err.response) from None

anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages: Unexpected role "function". Allowed roles are "user" or "assistant". For instructions on how to use tools, see https://docs.anthropic.com/en/docs/tool-use.'}}

Steps to reproduce

  1. Assigning claude-3-5-sonnet-20240620 model to the Analyzer Agent (only this model, only to this agent)
  2. Activating group chat, round-robin - [analyzer (simple llm call), analyzer_assistant (prepare function call), executor (function call), reviewer (simple llm call)]

Model Used

claude-3-5-sonnet-20240620

Expected Behavior

Normal group chat execution

Screenshots and logs

Traceback (most recent call last):
File "/Users/gil/Project personal/autogen_new/blogs/create_spider_config.py", line 62, in calc_config
blog_articles_patterns_results = retrieve_xpath_dict(input_htmls=[input_main_html],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/resolve_xpath_set.py", line 51, in retrieve_xpath_dict
manager.initiate_chat(
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 1018, in initiate_chat
self.send(msg2send, recipient, silent=silent)
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 655, in send
recipient.receive(message, self, request_reply, silent)
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 820, in receive
self.send(reply, sender, silent=silent)
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 655, in send
recipient.receive(message, self, request_reply, silent)
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 818, in receive
reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 1972, in generate_reply
final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/groupchat.py", line 1052, in run_chat
reply = speaker.generate_reply(sender=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 1972, in generate_reply
final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 1340, in generate_oai_reply
extracted_response = self._generate_oai_reply_from_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/agentchat/conversable_agent.py", line 1359, in _generate_oai_reply_from_client
response = llm_client.create(
^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/oai/client.py", line 722, in create
response = client.create(params)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/autogen/oai/anthropic.py", line 141, in create
response = self._client.messages.create(**anthropic_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/anthropic/_utils/_utils.py", line 277, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/anthropic/resources/messages.py", line 902, in create
return self._post(
^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 1266, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 942, in request
return self._request(
^^^^^^^^^^^^^^
File "/Users/gil/Project personal/autogen_new/.venv/lib/python3.12/site-packages/anthropic/_base_client.py", line 1046, in _request
raise self._make_status_error_from_response(err.response) from None
anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages: Unexpected role "function". Allowed roles are "user" or "assistant". For instructions on how to use tools, see https://docs.anthropic.com/en/docs/tool-use.'}}

Additional Information

No response

### Tasks

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.2Issues which are related to the pre 0.4 codebaseneeds-triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions