AutoGen v0.5.4+ Agent As Tool BUG

[XavierWangHX]

What happened?

Describe the bug
When a Host AssistantAgent is configured with multiple AgentTool instances (where each tool is another AssistantAgent) and is expected to call these tools sequentially based on its thought process, it only calls the first tool and then stops. This behavior was observed when using the Qwen3 model (qwen3-235b-a22b), Qwen-Max and DeepSeekV3 (Other model may have the same question, I also tried Gemini 2.5pro via gemini official api) via the Alibaba DashScope API, even when parallel_tool_calls=True is set for the model client.
The agent's(Qwen3) internal think process correctly outlines the plan to call multiple tools in sequence, but only the first tool call is launched.

To Reproduce
Code:
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_agentchat.tools import AgentTool
from autogen_core import CancellationToken

async def main() -> None:
"""
Travel Planning System using Toolized Expert Agents

This script implements a travel planning system using autogen_agentchat.tools:
1. Travel Planner - Provides overall travel itinerary
2. Local Activities Expert - Recommends local attractions and experiences
3. Language Consultant - Provides language and communication guidance

These expert tools are coordinated by a summary agent to generate the final travel plan
"""

# Create OpenAI model client
API_BASE = "xxxx"
API_KEY = "xxxx"
model_client = OpenAIChatCompletionClient(
    model="qwen3-235b-a22b",
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": True,
        "structured_output": True,
        "family": "unknown"
    },
    api_key=API_KEY,
    base_url=API_BASE
)

try:
    # Create expert agents
    # 1. Travel Planner Agent
    planner_agent = AssistantAgent(
        name="planner_agent",
        model_client=model_client,
        model_client_stream=True,
        description="Travel planning expert that creates customized itineraries based on user requirements.",
        system_message="""You are a professional travel planner who creates detailed travel recommendations.

Your responsibilities include:

1. Analyzing travel destinations and time constraints
2. Providing comprehensive itineraries including attractions, accommodation, and transportation
3. Considering seasonal factors and local specialties
4. Offering budget recommendations

Please provide professional, detailed, and practical travel planning advice."""
)

    # 2. Local Activities Agent
    local_agent = AssistantAgent(
        name="local_agent",
        model_client=model_client,
        model_client_stream=True,
        description="Local activities expert that recommends unique experiences and hidden gems.",
        system_message="""You are a local activities and attractions expert with knowledge of unique experiences worldwide.

Your responsibilities include:

1. Recommending distinctive local activities and attractions
2. Sharing off-the-beaten-path hidden spots
3. Suggesting local cuisine and cultural experiences
4. Identifying opportunities to interact with locals

Please provide authentic, interesting, and unique local experience recommendations to help travelers immerse in local culture."""
)

    # 3. Language Consultant Agent
    language_agent = AssistantAgent(
        name="language_agent",
        model_client=model_client,
        model_client_stream=True,
        description="Language expert providing communication guidance for destination languages.",
        system_message="""You are a language and communication expert with global language and cultural knowledge.

Your responsibilities include:

1. Providing basic language information about the destination
2. Teaching practical local phrases and expressions
3. Explaining cultural differences and communication taboos
4. Offering solutions for language barriers

Please provide practical and accurate language/communication advice to help travelers interact with locals."""
)

    # Convert expert agents to tools
    planner_tool = AgentTool(agent=planner_agent)
    local_tool = AgentTool(agent=local_agent)
    language_tool = AgentTool(agent=language_agent)
    
    # Create travel summary agent using the three expert tools
    travel_summary_agent = AssistantAgent(
        name="travel_summary_agent",
        model_client=model_client,
        model_client_stream=True,
        description="Travel summary expert that integrates inputs from all specialists into a comprehensive plan.",
        system_message="""You are a travel summary expert responsible for integrating specialist recommendations into a complete travel plan.

You can use these tools to get expert advice:

1. planner_tool - Get detailed travel itinerary suggestions
2. local_tool - Get local activity and attraction recommendations
3. language_tool - Get language and communication guidance

Your responsibilities:

1. Understand user's travel requirements
2. Consult experts for specialized recommendations
3. Integrate all inputs into a coherent comprehensive plan
4. Ensure the plan covers itinerary, activities, and communication tips

Your final response must be a complete travel plan. End with "Plan completed" when all expert inputs are integrated.""",
tools=[planner_tool, local_tool, language_tool],
)

    # Run travel planning system
    print("Launching travel planning system...\n")
    
    # Example user query
    travel_query = "Please create a 3-day travel plan for Singapore including must-see attractions, local experiences, and language tips."
    print(f"User request: {travel_query}\n")

    # Process query with travel summary agent
    await Console(travel_summary_agent.run_stream(
        task=travel_query, 
        cancellation_token=CancellationToken()
    ))
    
finally:
    # Close model client connection
    await model_client.close()

if name == "main":
asyncio.run(main())

Output:
C:\Users\xxxxx\Desktop\Code>python test.py
Launching travel planning system...

User request: Please create a 3-day travel plan for Singapore including must-see attractions, local experiences, and language tips.

---------- TextMessage (user) ----------
Please create a 3-day travel plan for Singapore including must-see attractions, local experiences, and language tips.
---------- ModelClientStreamingChunkEvent (travel_summary_agent) ----------
Okay, let's tackle this user's request for a 3-day travel plan in Singapore. They want must-see attractions, local experiences, and language tips. I need to use the three tools provided: planner_agent, local_agent, and language_agent.

First, I should figure out what each tool does. The planner_agent creates itineraries, so I'll need to ask for a 3-day itinerary. The local_agent gives unique experiences and hidden gems, so that's for the local experiences part. The language_agent provides communication guidance, which covers the language tips.

So, step 1: Call planner_agent with the task of creating a 3-day itinerary. That should give the main structure with top attractions. Then, step 2: Use local_agent to find unique activities and hidden spots. Step 3: Use language_agent to get language advice for Singapore.

Once I have all three inputs, I need to integrate them into a coherent plan. The user might be a tourist who wants both the highlights and some authentic local experiences, and they probably want to communicate effectively. They might not speak the local languages, so key phrases would be helpful.

Wait, Singapore has multiple languages: English, Mandarin, Malay, Tamil. So the language tips should include common phrases in these languages. Also, the local experiences might include food markets, cultural neighborhoods, etc.

I need to make sure the plan is balanced each day, mixing main attractions with local spots. The user might also appreciate tips on transportation, but the planner tool should handle that. Let me start by calling each agent in order.
---------- ToolCallRequestEvent (travel_summary_agent) ----------
[FunctionCall(id='call_f4c5ef67cf38465dbeaf2c', arguments='{"task": "Create a 3-day travel itinerary for Singapore with must-see attractions."}', name='planner_agent')]
---------- ToolCallExecutionEvent (travel_summary_agent) ----------
[FunctionExecutionResult(content='planner_agent: Singapore 3-Day Itinerary: Iconic Attractions & Cultural Highlights\n\nDay 1: Urban Landmarks & Futuristic Gardens \nMorning: \n- Merlion Park (9:00 AM): Start at Singapore ’s iconic symbol, the Merlion, with views of the Financial District. Walk along the waterfront to the Esplanade Bridge for photo ops of Marina Bay Sands (MBS). \n- Marina Bay Sands (10:30 AM): Explore the Sands Expo and ArtScience Museum (ticket: ~S$18; must-see exhibit: "Future World"). \n\nLunch: \n- Dine at Adrift by David Myers (MBS Shoppes) for modern Asian fusion or budget-friendly options at the Shoppes Food Court. \n\nAfternoon: \n- Gardens by the Bay (1:30 PM): Stroll the Supertree Grove (free), then visit the Flower Dome (ticket: ~S$15) and Cloud Forest (ticket: ~S$15). Stay indoors if rainy. \n\nEvening: \n- Dinner at Marina Bay Sands: Splurge on sky-high views at CE LA VI (S$80–120/pers) or opt for Yan Du (contemporary Chinese, S$40–60). \n- Gardens by the Bay Light Show (~7:45 PM): Free nightly light-and-music display at the Supertrees. \n\nTransport Tip: MRT (Raffles Place → Promenade) or walk from MBS to Gardens. \n\n---\n\nDay 2: Sentosa Island Adventure \nMorning: \n- Universal Studios Singapore (8:30 AM): Arrive early to beat crowds. Must-ride attractions: Battlestar Galactica (roller coasters) and Transformers: The Ride. \n\nLunch: \n- Eat inside the park (e.g., Hollywood New York Diner) or head to Food Republic (Palawan Beach, budget-friendly). \n\nAfternoon: \n- S.E.A. Aquarium (1:30 PM): Asia’s largest aquarium (~S$20); skip if time is tight. \n- Adventure Cove Water Park (3:00 PM): Splash around or relax at Palawan Beach. \n\nEvening: \n- Dinner at Ocean Restaurant by Man Fu Yuan (6:30 PM): Dine amid aquarium views (~S$100/pers). \n- Sentosa Cable Car Ride (8:00 PM): Panoramic night views on the return trip to HarbourFront. \n\nOptional: Swap water park/aquarium for Resorts World Sentosa (casino, shopping) or Fort Siloso. \n\nTransport Tip: MRT to HarbourFront, then Sentosa Express (S$4 return). \n\n---\n\nDay 3: Culture, Heritage & Nightlife \nMorning: \n- Chinatown (9:00 AM): Visit Buddha Tooth Relic Temple, Armenian Street, and Maxwell Chambers (lunch: Hainanese chicken rice at Tian Tian). \n\nAfternoon: \n- Little India (12:30 PM): Explore Sri Veeramakaliamman Temple, Tekka Centre (lunch: banana leaf rice at Banana Leaf Apolo), and Serangoon Road’s colorful shophouses. \n\nLate Afternoon: \n- Orchard Road (2:30 PM): Shop at ION Orchard or indulge in afternoon tea at The Lobby Lounge (Shangri-La Hotel, ~S$40/pers). \n\nEvening: \n- Clarke Quay (6:00 PM): Riverfront dinner at The Boathouse (Modern Asian, S$50/pers). \n- Singapore River Cruise (7:30 PM): 30-minute scenic cruise (~S$18) followed by drinks at Quay Bar or PS.Cafe. \n\nOptional Adjustment: Swap shopping/Clarke Quay for Singapore Zoo + Night Safari (~S$40 combined). \n\nTransport Tip: MRT (Chinatown → Little India → Somerset/Orchard). \n\n---\n\nAccommodation Recommendations: \n- Luxury: Marina Bay Sands (central, skyline views). \n- Mid-Range: Park Hotel Clarke Quay (vibrant location). \n- Budget: Hotel 81 Chinatown (convenient, affordable). \n\nBudget Breakdown (Per Person): \n- Budget Traveler: S$150–200/day (hawker meals, public transport, free attractions). \n- Mid-Range: S$250–350/day (moderate dining, Sentosa attractions). \n- Splurge: S$400+/day (fine dining, luxury hotels, theme parks). \n\nPractical Tips: \n- Transport: Buy an EZ-Link card (~S$12, reusable) for MRT/bus. \n- Weather: Pack an umbrella; sudden showers common. \n- Local Eats: Try laksa (Coconut Club), chili crab (Jumbo Seafood), and kopi (traditional coffee). \n\nThis itinerary balances Singapore’s futuristic marvels, cultural depth, and vibrant nightlife. Enjoy your tropical adventure! 🌴✨', name='planner_agent', call_id='call_b4e0a7600ba14987814bae', is_error=False)]
---------- ToolCallSummaryMessage (travel_summary_agent) ----------
planner_agent: Singapore 3-Day Itinerary: Iconic Attractions & Cultural Highlights

Day 1: Urban Landmarks & Futuristic Gardens
Morning:

• Merlion Park (9:00 AM): Start at Singapore’s iconic symbol, the Merlion, with views of the Financial District. Walk along the waterfront to the Esplanade Bridge for photo ops of Marina Bay Sands (MBS).
• Marina Bay Sands (10:30 AM): Explore the Sands Expo and ArtScience Museum (ticket: ~S$18; must-see exhibit: "Future World").

Lunch:

• Dine at Adrift by David Myers (MBS Shoppes) for modern Asian fusion or budget-friendly options at the Shoppes Food Court.

Afternoon:

• Gardens by the Bay (1:30 PM): Stroll the Supertree Grove (free), then visit the Flower Dome (ticket: ~S$15) and Cloud Forest (ticket: ~S$15). Stay indoors if rainy.

Evening:

• Dinner at Marina Bay Sands: Splurge on sky-high views at CE LA VI (S$80–120/pers) or opt for Yan Du (contemporary Chinese, S$40–60).
• Gardens by the Bay Light Show (~7:45 PM): Free nightly light-and-music display at the Supertrees.

Transport Tip: MRT (Raffles Place → Promenade) or walk from MBS to Gardens.

---

Day 2: Sentosa Island Adventure
Morning:

• Universal Studios Singapore (8:30 AM): Arrive early to beat crowds. Must-ride attractions: Battlestar Galactica (roller coasters) and Transformers: The Ride.

Lunch:

• Eat inside the park (e.g., Hollywood New York Diner) or head to Food Republic (Palawan Beach, budget-friendly).

Afternoon:

• S.E.A. Aquarium (1:30 PM): Asia’s largest aquarium (~S$20); skip if time is tight.
• Adventure Cove Water Park (3:00 PM): Splash around or relax at Palawan Beach.

Evening:

• Dinner at Ocean Restaurant by Man Fu Yuan (6:30 PM): Dine amid aquarium views (~S$100/pers).
• Sentosa Cable Car Ride (8:00 PM): Panoramic night views on the return trip to HarbourFront.

Optional: Swap water park/aquarium for Resorts World Sentosa (casino, shopping) or Fort Siloso.

Transport Tip: MRT to HarbourFront, then Sentosa Express (S$4 return).

---

Day 3: Culture, Heritage & Nightlife
Morning:

• Chinatown (9:00 AM): Visit Buddha Tooth Relic Temple, Armenian Street, and Maxwell Chambers (lunch: Hainanese chicken rice at Tian Tian).

Afternoon:

• Little India (12:30 PM): Explore Sri Veeramakaliamman Temple, Tekka Centre (lunch: banana leaf rice at Banana Leaf Apolo), and Serangoon Road’s colorful shophouses.

Late Afternoon:

• Orchard Road (2:30 PM): Shop at ION Orchard or indulge in afternoon tea at The Lobby Lounge (Shangri-La Hotel, ~S$40/pers).

Evening:

• Clarke Quay (6:00 PM): Riverfront dinner at The Boathouse (Modern Asian, S$50/pers).
• Singapore River Cruise (7:30 PM): 30-minute scenic cruise (~S$18) followed by drinks at Quay Bar or PS.Cafe.

Optional Adjustment: Swap shopping/Clarke Quay for Singapore Zoo + Night Safari (~S$40 combined).

Transport Tip: MRT (Chinatown → Little India → Somerset/Orchard).

---

Accommodation Recommendations:

• Luxury: Marina Bay Sands (central, skyline views).
• Mid-Range: Park Hotel Clarke Quay (vibrant location).
• Budget: Hotel 81 Chinatown (convenient, affordable).

Budget Breakdown (Per Person):

• Budget Traveler: S$150–200/day (hawker meals, public transport, free attractions).
• Mid-Range: S$250–350/day (moderate dining, Sentosa attractions).
• Splurge: S$400+/day (fine dining, luxury hotels, theme parks).

Practical Tips:

• Transport: Buy an EZ-Link card (~S$12, reusable) for MRT/bus.
• Weather: Pack an umbrella; sudden showers common.
• Local Eats: Try laksa (Coconut Club), chili crab (Jumbo Seafood), and kopi (traditional coffee).

This itinerary balances Singapore’s futuristic marvels, cultural depth, and vibrant nightlife. Enjoy your tropical adventure! 🌴✨

Expected behavior
The travel_summary_agent should follow its think plan and sequentially call all three tools (planner_tool, local_tool, and language_tool). After receiving the response from planner_tool, it should proceed to call local_tool, and then language_tool, before finally synthesizing the complete travel plan, but it only called the first AgentTool and stopped.

Which packages was the bug in?

Python AgentChat (autogen-agentchat>=0.4.0)

AutoGen library version.

Python 0.5.4

Other library version.

No response

Model used

Qwen3,DeepSeek,Gemini

Model provider

None

Other model provider

No response

Python version

3.10

.NET version

None

Operating system

Ubuntu

[ekzhu]

@XavierWangHX this is not a bug, it is intended behavior of AssistantAgent. You need to invoke the agent in loop if you want it to automatically calling the tools in sequence, otherwise, the agent stops after first tool call.

See the tutorial on AssistantAgent: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#running-an-agent-in-a-loop.

[ekzhu]

My recommendation is to implement your custom agent rather than using builtin AssistantAgent. See the note on why not using AssistantAgent for everything: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#assistant-agent

[XavierWangHX]

Thank you I understood，by the way，can it make three fuction calls simultaneously？ 在 2025年5月13日，上午8:25，Eric Zhu ***@***.***> 写道：  - External Email - [https://avatars.githubusercontent.com/u/320302?s=20&v=4]ekzhu left a comment (microsoft/autogen#6516)<#6516 (comment)> @XavierWangHX<https://github.com/XavierWangHX> this is not a bug, it is intended behavior of AssistantAgent. You need to invoke the agent in loop if you want it to automatically calling the tools in sequence, otherwise, the agent stops after first tool call. See the tutorial on AssistantAgent: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#running-an-agent-in-a-loop. — Reply to this email directly, view it on GitHub<#6516 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BEICP6FJDZODD5WI7PYTPJD26E3XXAVCNFSM6AAAAAB45ODBOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZUGY2TAMJYGU>. You are receiving this because you were mentioned.Message ID: ***@***.***> Notice: This email is generated from the account of an NUS alumnus. Contents, views, and opinions therein are solely those of the sender.