Fix delegation visualization labels to show correct sender/receiver #1119

simonrosenberg · 2025-11-10T10:12:50Z

Description

This PR fixes all visualization issues in the delegation example (25_agent_delegation.py) by ensuring consistent "Agent" suffix usage across all event titles.

Changes Made

1. System Prompt Visualization

Before: "System Prompt"
After: "[Agent Name] Agent System Prompt"
Added SystemPromptEvent handling in DelegationVisualizer._create_event_panel()
Imported SystemPromptEvent and _SYSTEM_COLOR from SDK

2. User Messages

Before: "Message from User to Delegator"
After: "User Message to Delegator Agent"
Updated message formatting to include "Agent" suffix consistently

3. Agent-to-Agent Messages

Before: "Delegator Message to Lodging Expert" (sender without "Agent")
After: "Delegator Agent Message to Lodging Expert Agent" (both with "Agent")
Updated delegated message formatting to include "Agent" suffix for both sender and receiver

4. Agent-to-User Messages

Before: "Message from Delegator to User"
After: "Message from Delegator Agent to User"
Updated response message formatting to include "Agent" suffix

5. Actions and Observations

Before: "Delegator Action", "Lodging Expert Observation"
After: "Delegator Agent Action", "Lodging Expert Agent Observation"
Updated action and observation titles to include "Agent" suffix

Testing

✅ All existing tests updated to match new label formats
✅ All tests passing (tests/tools/delegate/test_visualizer.py)
✅ Pre-commit hooks passing (ruff format, ruff lint, pycodestyle, pyright)
✅ Verified visualization output with 25_agent_delegation.py

Implementation Details

Changes are isolated to openhands-tools package (delegation-specific)
No changes needed to core openhands-sdk (maintains backward compatibility)
Consistent "Agent" suffix applied to all delegation event titles
Proper name formatting (snake_case to Title Case) preserved

Example Output

With these changes, delegation visualizations now show:

╭────────────── Delegator Agent System Prompt ──────────────╮
│ System Prompt: You are OpenHands agent...                 │
╰────────────────────────────────────────────────────────────╯

╭────────── User Message to Delegator Agent ────────────╮
│ Ask the lodging sub-agent about Covent Garden.         │
╰─────────────────────────────────────────────────────────╯

╭─────────────── Delegator Agent Action ─────────────────╮
│ Action: DelegateAction                                  │
│ Arguments: {"lodging_expert": "What about...?"}         │
╰─────────────────────────────────────────────────────────╯

╭──── Delegator Agent Message to Lodging Expert Agent ────╮
│ What do you think about staying in Covent Garden?       │
╰──────────────────────────────────────────────────────────╯

╭──── Lodging Expert Agent Message to Delegator Agent ────╮
│ **Short answer: Covent Garden is NOT a good budget..    │
╰──────────────────────────────────────────────────────────╯

╭────────────── Delegator Agent Observation ──────────────╮
│ Tool: delegate                                           │
│ Result: Completed delegation of 1 tasks                 │
╰──────────────────────────────────────────────────────────╯

╭───── Message from Delegator Agent to User ─────╮
│ Here's the consolidated report...               │
╰─────────────────────────────────────────────────╯

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.12-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:7e73ce6-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-7e73ce6-python \
  ghcr.io/openhands/agent-server:7e73ce6-python

All tags pushed for this build

ghcr.io/openhands/agent-server:7e73ce6-golang-amd64
ghcr.io/openhands/agent-server:7e73ce6-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:7e73ce6-golang-arm64
ghcr.io/openhands/agent-server:7e73ce6-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:7e73ce6-java-amd64
ghcr.io/openhands/agent-server:7e73ce6-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:7e73ce6-java-arm64
ghcr.io/openhands/agent-server:7e73ce6-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:7e73ce6-python-amd64
ghcr.io/openhands/agent-server:7e73ce6-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:7e73ce6-python-arm64
ghcr.io/openhands/agent-server:7e73ce6-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:7e73ce6-golang
ghcr.io/openhands/agent-server:7e73ce6-java
ghcr.io/openhands/agent-server:7e73ce6-python

About Multi-Architecture Support

Each variant tag (e.g., 7e73ce6-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 7e73ce6-python-amd64) are also available if needed

- Add optional 'sender' parameter to MessageEvent to track message origin - Update send_message() in BaseConversation and LocalConversation to accept sender - Modify DefaultConversationVisualizer to use sender info for accurate message titles - Update DelegateExecutor to pass formatted sender names when delegating tasks - Add helper function to format agent IDs (snake_case -> Title Case) - Add comprehensive tests for sender parameter behavior Fixes #1118 Co-authored-by: openhands <openhands@all-hands.dev>

github-actions · 2025-11-10T10:17:34Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/conversation
base.py	69	5	92%	106, 111–112, 156, 167
openhands-sdk/openhands/sdk/conversation/impl
local_conversation.py	189	53	71%	121, 123–124, 157–158, 181, 186, 202, 219, 227–229, 233–234, 287–288, 291, 298, 319–321, 324, 333, 349, 351, 353, 357, 359–361, 363, 365, 371–372, 385–386, 388, 390, 394–397, 414–415, 420, 425, 427, 432, 434–436, 454, 456
remote_conversation.py	374	105	71%	55–61, 68–71, 100, 107, 115, 117–120, 130, 139, 143–144, 149–152, 187, 201, 218, 229, 238–239, 291, 311, 319, 331, 339–342, 345, 350–353, 355, 360–361, 366–370, 375–379, 384–387, 390, 401–402, 406, 410, 413, 484, 490, 492, 508, 510–511, 522, 539–540, 546, 561, 579, 596, 598, 600–601, 605–606, 615–616, 625, 633, 638–640, 642, 645, 647–648, 665, 672, 678–679, 693–694, 701–702
openhands-sdk/openhands/sdk/conversation/visualizer
base.py	16	1	93%	67
default.py	118	20	83%	98, 135, 161–162, 175, 201–202, 211–212, 220–221, 230–231, 258, 262, 271, 273, 275, 278, 297
openhands-sdk/openhands/sdk/event/llm_convertible
message.py	68	18	73%	56, 74, 79–85, 89, 95, 98–99, 102, 120–121, 126, 142
openhands-tools/openhands/tools/delegate
impl.py	107	90	15%	30, 32–33, 42–43, 47, 53–54, 57–60, 62, 81–82, 88–89, 99–102, 104, 106, 108, 116–117, 124, 126, 132–133, 135–137, 142–144, 161–162, 169–171, 180, 182–184, 187–192, 194, 201–202, 204–205, 208–211, 213–214, 218–221, 224–226, 231–232, 235–236, 239, 241–245, 247, 250–252, 254–255, 258, 260, 265–267
visualizer.py	66	55	16%	58, 62, 88–89, 92–93, 96, 99–100, 118–119, 122–125, 128–129, 131, 133–134, 138, 145, 147–148, 153, 157, 166, 170, 179, 201–203, 205, 208–211, 213, 216, 218–219, 221–222, 228, 234–239, 241, 243–244, 250, 255
TOTAL	12221	5681	53%

- Fix typo: visualize -> visualizer parameter in DelegateExecutor - Remove string formatting logic from SDK visualizer (keep it in delegate layer) - Update tests to expect properly formatted agent names The delegate executor already formats agent names using _format_agent_name() which converts snake_case to Title Case (e.g., 'lodging_expert' -> 'Lodging Expert'). The SDK visualizer should not do any string transformation, just display the name as-is. Co-authored-by: openhands <openhands@all-hands.dev>

- Added recipient field to MessageEvent to track message destination - Agent now derives recipient from last user message in event history - Updated visualizer to use recipient field for agent response titles - Removed hidden state tracking for cleaner, more maintainable design - Updated tests to use new recipient field instead of sender This supports multi-agent scenarios (like delegation) by making message routing explicit without adding complexity to the conversation state. Co-authored-by: openhands <openhands@all-hands.dev>

simonrosenberg · 2025-11-10T11:15:27Z

Architecture Improvement

I've updated the implementation to use a cleaner approach that eliminates hidden state:

Previous Approach (first commit)

Tracked _last_message_sender as hidden state in LocalConversation
Required explicit tracking on every send_message() call
Added complexity to conversation state management

New Approach (current commit)

Agent derives recipient from event history when responding
Looks at the last MessageEvent with role="user" to determine who to respond to
No hidden state tracking needed in LocalConversation
More explicit and maintainable

Benefits

No Hidden State: Everything is derived from the event log
Simpler API: No state management complexity in LocalConversation
More Explicit: The logic is clear and contained in agent.py
Flexible: Works for any multi-party conversation, not just delegation

Key Changes

Added recipient field to MessageEvent (complementing existing sender)
Agent derives recipient when creating response messages
Visualizer uses recipient for agent message titles
All tests passing (235/235)

This approach is much cleaner and more maintainable! 🎉

Move recipient derivation logic from agent.py to the visualizer for cleaner separation of concerns. The agent now only sets the sender field when delegating tasks, and the visualizer derives the recipient by examining the event history when displaying agent responses. Changes: - Remove recipient field from MessageEvent in message.py - Remove recipient derivation logic from agent.py - Add recipient derivation in default.py visualizer by looking back through event history to find the last user message sender - Update test to mock event history for recipient derivation - Keep sender field for delegation messages (set by delegation tool) This approach maintains backward compatibility while providing correct sender/receiver information for delegation scenarios. Co-authored-by: openhands <openhands@all-hands.dev>

- Implemented DelegationVisualizer in openhands-tools to display delegation-specific message labels - Added _format_agent_name() function to convert snake_case/camelCase agent names to Title Case - Updated DefaultConversationVisualizer in SDK to show simpler labels ('Message from User', 'Message from Agent') - Updated 25_agent_delegation.py example to use DelegationVisualizer - Added comprehensive test suite for DelegationVisualizer (5 tests) - Updated SDK visualizer tests to match simplified labels - All tests passing (21 tests for visualizers) Fixes #1118 Co-authored-by: openhands <openhands@all-hands.dev>

The default visualizer now shows generic 'Message from Agent' labels without using the agent's specific name, keeping it simple and generic. Agent-specific names are only shown in the DelegationVisualizer which extends the default visualizer for delegation scenarios. This makes the separation of concerns clearer: - DefaultConversationVisualizer: Generic, no agent names needed - DelegationVisualizer: Shows specific agent names for delegation Updated tests to match the new generic labels. Co-authored-by: openhands <openhands@all-hands.dev>

The name parameter and _name attribute have been removed from: - ConversationVisualizerBase: No longer has name parameter or _name attribute - DefaultConversationVisualizer: No longer uses name, shows generic labels The DelegationVisualizer now independently adds: - name parameter in its __init__ - _name attribute for delegation-specific functionality - Properly calls parent __init__ without name parameter This maintains backward compatibility through proper inheritance while keeping the base visualizers generic and delegation-specific features in the appropriate subclass. All tests passing (16 SDK visualizer tests, 5 delegation visualizer tests). Co-authored-by: openhands <openhands@all-hands.dev>

This commit completes the delegation visualization improvements by ensuring agent names appear consistently in ALL event types, not just messages. Changes: - Override _create_event_panel() in DelegationVisualizer to add agent names to actions and observations - ActionEvent now shows '[Agent Name] Action' - ObservationEvent now shows '[Agent Name] Observation' - Added tests to verify action and observation events show agent names correctly This ensures the complete fix for issue #1118 where users expected to see: - 'Main Agent Action' instead of generic 'Agent Action' - 'Lodging Expert Observation' instead of generic 'Observation' Co-authored-by: openhands <openhands@all-hands.dev>

This reverts commit e049427.

- Use DelegationVisualizer explicitly in delegate impl for sub-agents - Check for DelegationVisualizer instance before accessing _name attribute - Update test mock to match BaseConversation.send_message signature - Remove name parameter from example custom visualizer Co-authored-by: openhands <openhands@all-hands.dev>

…isualizer This commit extends the DelegationVisualizer to show agent names for all event types: - Action events now show '[Agent Name] Action' instead of just 'Action' - Observation events now show '[Agent Name] Observation' instead of just 'Observation' - Message events continue to use the existing detailed sender/receiver format This provides consistent agent context across all event types in delegation scenarios, making it easier to track which agent is performing which action or receiving which observation during multi-agent delegation workflows. Implementation: - Override _create_event_panel() to handle ActionEvent and ObservationEvent - Extract agent name formatting and prepend to default titles - Delegate to parent class for other event types Tests: - Added test for action event with agent name - Added test for observation event with agent name Co-authored-by: openhands <openhands@all-hands.dev>

- Add 'Agent' suffix to delegated messages: 'Delegator Message to Lodging Agent' - Add 'to User' suffix to agent responses to user: 'Message from Delegator to User' - Update tests to match new labels - All 28 tests passing Co-authored-by: openhands <openhands@all-hands.dev>

… event titles This commit addresses all visualization issues in the delegation example: 1. System Prompt now shows: '[Agent Name] Agent System Prompt' 2. User messages now show: 'User Message to [Agent Name] Agent' 3. Agent-to-agent messages now show: '[Sender] Agent Message to [Receiver] Agent' 4. Agent-to-user messages now show: 'Message from [Agent Name] Agent to User' 5. Actions now show: '[Agent Name] Agent Action' 6. Observations now show: '[Agent Name] Agent Observation' Changes: - Added SystemPromptEvent handling in DelegationVisualizer - Updated all message, action, and observation labels to include 'Agent' suffix - Updated tests to match new label formats - Imported _SYSTEM_COLOR and SystemPromptEvent Fixes #1118 Co-authored-by: openhands <openhands@all-hands.dev>

@staticmethod

Changed _format_agent_name from a module-level function to a static method of the DelegationVisualizer class for better encapsulation. Changes: - Moved _format_agent_name function inside DelegationVisualizer as @staticmethod - Updated all calls from _format_agent_name() to self._format_agent_name() - Updated docstring examples to reflect class method usage Co-authored-by: openhands <openhands@all-hands.dev>

Consolidated agent name formatting by removing the duplicate implementation from impl.py and using the centralized version in DelegationVisualizer. Changes: - Removed _format_agent_name function from impl.py - Updated all calls to use DelegationVisualizer._format_agent_name() - Maintains single source of truth for agent name formatting logic Co-authored-by: openhands <openhands@all-hands.dev>

…zer requirement All agent name formatting is now handled exclusively by the visualizer. The impl.py passes raw agent IDs and the DelegationVisualizer is responsible for formatting them for display. Changes: - Removed all calls to _format_agent_name from impl.py - Pass raw agent_id to DelegationVisualizer constructor - Pass raw parent_name to send_message (visualizer formats it) - Enforce that delegation tool requires DelegationVisualizer - Removed fallback code paths for other visualizer types Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-11-10T14:41:26Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- Run tests
- Pre-commit checks

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1119 at branch `openhands/fix-delegation-visualization`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

Delegation should not require DelegationVisualizer - it can work with no visualizer at all. Only create DelegationVisualizer for sub-agents if the parent conversation is already using DelegationVisualizer. Changes: - Removed ValueError that enforced DelegationVisualizer requirement - Sub-agents get DelegationVisualizer only if parent uses it - Sub-agents get no visualizer (None) if parent has no visualizer Co-authored-by: openhands <openhands@all-hands.dev>

…izer The skip_user_messages parameter doesn't make sense for delegation scenarios where inter-agent messages (which appear as user messages to sub-agents) are essential to understanding the delegation flow. Changes: - Removed skip_user_messages parameter from DelegationVisualizer.__init__ - Removed passing of skip_user_messages in impl.py - Simplified visualizer initialization Co-authored-by: openhands <openhands@all-hands.dev>

…onVisualizer" This reverts commit c00319c.

…andling The merge from main introduced CondensationRequest event handling that still referenced self._name which was removed in our refactoring. Updated to match the pattern used for other event types. Co-authored-by: openhands <openhands@all-hands.dev>

The base DefaultConversationVisualizer does not use the sender field at all. This test was added to verify sender functionality but is only relevant for the DelegationVisualizer subclass which overrides the behavior. The test has been removed from the SDK tests and only exists in the delegation-specific tests. Co-authored-by: openhands <openhands@all-hands.dev>

examples/01_standalone_sdk/26_custom_visualizer.py

enyst · 2025-11-10T15:48:39Z

openhands-sdk/openhands/sdk/event/llm_convertible/message.py

    extended_content: list[TextContent] = Field(
        default_factory=list, description="List of content added by agent context"
    )
+    sender: str | None = Field(


Please let me try to see if I got this right, so right now, on the PR, we save a sender in the MessageEvent, although

we don't tell the LLM the sender

we don't use it in visualize

🤔

Agreed that's a bit sketchy!
We could use it in the default visualize so it makes more sense

enyst

LGTM! Interesting which way it's going, maybe someday we'll tell the delegate who the sender really was 😇

I'd love it if @xingyaoww could take a look since it's changing the visualizer interface we just changed 😅

simonrosenberg · 2025-11-11T08:04:16Z

LGTM! Interesting which way it's going, maybe someday we'll tell the delegate who the sender really was 😇

I'd love it if @xingyaoww could take a look since it's changing the visualizer interface we just changed 😅

This PR reverts the visualizer to what is was before delegation "Message from Agent" "User Message" etc... @xingyaoww

xingyaoww · 2025-11-11T14:52:28Z

openhands-sdk/openhands/sdk/conversation/visualizer/base.py

    Conversation will then calls `MyVisualizer()` followed by `initialize(state)`
    """

-    _name: str | None


is there anyway that we are not breaking this?

removing this is a breaking changes. But i do see doing so would simplify our code and hopefully have minimal impact to users

Do we have any other breaking change for 1.0.1 ?

I mean if you know, off-hand. Don't worry if you don't! I'll find out automatically

this _name was introduce solely for delegation visualization. So I think it does make sense to remove it so the default viz remains as simple as possible... Cf other comment: no need for init anymore in minimal custom visualizer

examples/01_standalone_sdk/26_custom_visualizer.py

xingyaoww

LGTM

openhands-ai bot mentioned this pull request Nov 10, 2025

Improve Visualisations for Delegation #1118

Closed

simonrosenberg force-pushed the openhands/fix-delegation-visualization branch from c1dbb4d to 792ff4c Compare November 10, 2025 10:53

openhands-agent added 13 commits November 10, 2025 11:39

Revert "Show agent names in all event types (actions and observations)"

51a554f

This reverts commit e049427.

openhands-agent and others added 6 commits November 10, 2025 14:44

Revert "Remove unnecessary skip_user_messages parameter from Delegati…

d8b7b58

…onVisualizer" This reverts commit c00319c.

Merge branch 'main' into openhands/fix-delegation-visualization

64fbecd

simonrosenberg requested review from enyst and xingyaoww November 10, 2025 15:18

enyst reviewed Nov 10, 2025

View reviewed changes

examples/01_standalone_sdk/26_custom_visualizer.py Show resolved Hide resolved

enyst reviewed Nov 10, 2025

View reviewed changes

xingyaoww reviewed Nov 11, 2025

View reviewed changes

simplify

b75ae23

xingyaoww approved these changes Nov 13, 2025

View reviewed changes

simonrosenberg merged commit 57b3657 into main Nov 13, 2025
21 checks passed

simonrosenberg deleted the openhands/fix-delegation-visualization branch November 13, 2025 19:41

Fix delegation visualization labels to show correct sender/receiver #1119

Fix delegation visualization labels to show correct sender/receiver #1119

Uh oh!

Conversation

simonrosenberg commented Nov 10, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

1. System Prompt Visualization

2. User Messages

3. Agent-to-Agent Messages

4. Agent-to-User Messages

5. Actions and Observations

Testing

Implementation Details

Example Output

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

simonrosenberg commented Nov 10, 2025

Architecture Improvement

Previous Approach (first commit)

New Approach (current commit)

Benefits

Key Changes

Uh oh!

openhands-ai bot commented Nov 10, 2025

Uh oh!

Uh oh!

enyst Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

simonrosenberg Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

simonrosenberg commented Nov 11, 2025

Uh oh!

xingyaoww Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

enyst Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

enyst Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

simonrosenberg Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xingyaoww left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

simonrosenberg commented Nov 10, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

simonrosenberg Nov 12, 2025 •

edited

Loading