Skip to content

MultimodalConversableAgent system_message incompatible with groupchat #751

@ibishara

Description

@ibishara

MultimodalConversableAgent system_message is a list, but AssistantAgent system_message is a string. This raises the following error when both agents are included in a groupchat:

File ~/miniconda3/envs/pyautogen/lib/python3.10/site-packages/autogen/agentchat/groupchat.py:193, in GroupChat._participant_roles(self, agents) 191 roles = [] 192 for agent in agents: --> 193 if agent.system_message.strip() == "": 194 logger.warning( 195 f"The agent '{agent.name}' has an empty system_message, and may not work well with GroupChat." 196 ) 197 roles.append(f"{agent.name}: {agent.system_message}") AttributeError: 'list' object has no attribute 'strip'

A reproducible example:

from autogen.agentchat.contrib.multimodal_conversable_agent import MultimodalConversableAgent
from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager

gpt_config_list = [
{'model': 'gpt-3.5-turbo-1106', 'api_key': 'OAI_API_KEY'}
]

gptv_config_list = [
{'model': "gpt-4-vision-preview", 'api_key': 'OAI_API_KEY'}
]

planner = AssistantAgent(
name="Planner",
llm_config={"config_list": gpt_config_list},
system_message='Generate plans and organize tasks.'
)

image_agent = MultimodalConversableAgent(
name="Image_Agent",
max_consecutive_auto_reply=10,
llm_config={"config_list": gptv_config_list},
system_message='Specialized in image understanding.'
)

user_proxy = UserProxyAgent(
name="User_Proxy",
llm_config={"config_list": gpt_config_list},
system_message='Execute code snippets and test functionalities.'
)

def initiate_group_chat(problem_statement):
groupchat = GroupChat(
agents=[planner, user_proxy, image_agent],
messages=[],
max_round=50
)
manager = GroupChatManager(groupchat=groupchat, llm_config={"config_list": gpt_config_list})
user_proxy.initiate_chat(manager, message=problem_statement)

problem_statement = 'Task: Describe and replicate a webpage.'

initiate_group_chat(problem_statement)

Metadata

Metadata

Assignees

No one assigned

    Labels

    multimodallanguage + vision, speech etc.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions