Skip to content

Python: Fix AG-UI reasoning role and multimodal media parsing to follow specification#5389

Open
moonbox3 wants to merge 3 commits intomicrosoft:mainfrom
moonbox3:agent/fix-5340-2
Open

Python: Fix AG-UI reasoning role and multimodal media parsing to follow specification#5389
moonbox3 wants to merge 3 commits intomicrosoft:mainfrom
moonbox3:agent/fix-5340-2

Conversation

@moonbox3
Copy link
Copy Markdown
Contributor

Motivation and Context

The AG-UI adapter emitted role="assistant" on ReasoningMessageStartEvent instead of the spec-required role="reasoning", causing reasoning messages to render incorrectly in AG-UI clients. Additionally, _parse_multimodal_media_part read the deprecated "data" field instead of the current spec's "value" field, silently dropping all user-submitted multimodal media.

Fixes #5340

Description

Changed ReasoningMessageStartEvent role from "assistant" to "reasoning" in both the flow and non-flow code paths of _emit_text_reasoning in _run_common.py. Updated _parse_multimodal_media_part in _message_adapters.py to read source_dict.get("value") first (with fallback to "data" for backward compatibility) for base64/data source types and the unknown-type fallback branch. Bumped the ag-ui-protocol dependency from pined 0.1.13 to >=0.1.16,<0.2 to pick up the upstream SDK fix for the reasoning role type. Added targeted regression tests for both the reasoning role and multimodal value-field parsing.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Note: PR autogenerated by moonbox3's agent

Copilot and others added 2 commits April 21, 2026 06:58
Fix two spec compliance issues in the AG-UI integration:

1. ReasoningMessageStartEvent now uses role='reasoning' instead of
   role='assistant', matching the AG-UI specification for reasoning
   messages.

2. _parse_multimodal_media_part now reads the 'value' field from source
   dicts (with fallback to 'data' for backward compatibility), matching
   the current AG-UI InputContentSource specification.

Bump ag-ui-protocol dependency from ==0.1.13 to >=0.1.16,<0.2 to pick
up the SDK fix that accepts role='reasoning' in ReasoningMessageStartEvent.

Fix pre-existing pyright reportMissingImports errors for orjson in sample
files, and fix import ordering in foundry-hosted-agents sample.

Fixes microsoft#5340

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 21, 2026 07:04
@moonbox3 moonbox3 self-assigned this Apr 21, 2026
Copy link
Copy Markdown
Contributor Author

@moonbox3 moonbox3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 96% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach


Automated review by moonbox3's agents

@moonbox3
Copy link
Copy Markdown
Contributor Author

moonbox3 commented Apr 21, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/ag-ui/agent_framework_ag_ui
   _message_adapters.py5975590%102–103, 112–115, 118–122, 124–129, 132, 141–147, 187, 318, 408–411, 413–415, 462–464, 518, 521, 523, 526, 529, 545, 562, 584, 684, 700–701, 772, 794, 864, 899–900, 968, 1011
   _run_common.py3071096%381–382, 388–393, 701–702
TOTAL28982347688% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
5760 30 💤 0 ❌ 0 🔥 1m 31s ⏱️

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes the Python AG-UI adapter to align with the AG-UI specification for reasoning event roles and multimodal media payload parsing, and updates the AG-UI protocol dependency/tests accordingly.

Changes:

  • Emit ReasoningMessageStartEvent(role="reasoning") (instead of "assistant") in both streaming and non-streaming reasoning paths.
  • Parse multimodal media payloads from source.value first (with fallback to deprecated source.data) for base64/data/unknown source types.
  • Update ag-ui-protocol dependency (and lockfile) and add regression tests for both fixes.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
python/packages/ag-ui/agent_framework_ag_ui/_run_common.py Sets reasoning message start role to "reasoning" in both flow and non-flow paths.
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py Prefers source.value over source.data when parsing multimodal media payloads.
python/packages/ag-ui/pyproject.toml Relaxes ag-ui-protocol pin to >=0.1.16,<0.2 to pick up upstream fix.
python/uv.lock Locks ag-ui-protocol to 0.1.17 and updates metadata specifier accordingly.
python/packages/ag-ui/tests/ag_ui/test_run.py Updates existing reasoning role assertion and adds targeted flow/non-flow role regression tests.
python/packages/ag-ui/tests/ag_ui/test_message_adapters.py Adds regression tests ensuring value is used (and preferred) for multimodal media parsing.
python/samples/04-hosting/foundry-hosted-agents/responses/02_local_tools/main.py Import ordering adjustment (moves Annotated import into the stdlib section).
python/samples/02-agents/conversations/file_history_provider.py Adds inline Pyright ignore for optional orjson import.
python/samples/02-agents/conversations/file_history_provider_conversation_persistence.py Adds inline Pyright ignore for optional orjson import.
.maf-runtime-ready Adds a new repo-root marker file.

Comment thread .maf-runtime-ready Outdated
Copy link
Copy Markdown
Contributor Author

@moonbox3 moonbox3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 94% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach


Automated review by moonbox3's agents

Address PR review feedback: the .maf-runtime-ready file is not referenced anywhere in the repo and was left over from automation.

Fixes microsoft#5340
@Rickyneer
Copy link
Copy Markdown

Rickyneer commented Apr 22, 2026

To: @moonbox3
Thank you for the fast response to my issue.

I do want to note that this does not fix the issue completely and with this fix, another part of the bug has popped up.
The place in which you patched the multimodal media parsing code, is not the only code that needs to be modified. An identical snippet of code that does the exact same thing (I recall in a function that is called _legacy_binary_parsing or something similar, also in the same file), which I think was for creating snapshot events, also looks for the wrong field "data" which should be field "value". This results in ag-ui compatible clients to crash when trying to ingest the snapshot event.
Maybe it's better to put both of these same code snippets behind a single function so it'll be easier to update in the future?
I hope to hear your opinion on this.

Thank you in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: AG-UI reasoning and multimodal media parsing doesn't correctly follow the specification

5 participants