.NET - AG-UI Support more AG-UI event types.

[javiercn]

We currently support emitting the following AG-UI events:

RUN_STARTED
RUN_FINISHED
RUN_ERROR
TEXT_MESSAGE_START
TEXT_MESSAGE_CONTENT
TEXT_MESSAGE_END
TOOL_CALL_START
TOOL_CALL_ARGS
TOOL_CALL_END
TOOL_CALL_RESULT
STATE_SNAPSHOT
STATE_DELTA

This is because we are able to map these events to specific events and AIContent instances:

• RUN_STARTED: We unconditionally emit this event at the beginning of a run.

• RUN_FINISHED: We unconditionally emit this event at the end of a run.

• RUN_ERROR: We emit this event when an error happens.

• TEXT_MESSAGE_START: We emit this event when we see an update with a TextContent instance for the first time in a message.

• TEXT_MESSAGE_CONTENT: We emit this event every time we see a TextContent instance for a message.

• TEXT_MESSAGE_END: We emit this whenever we start see an AIContent instance for another message.

• TOOL_CALL_START: We emit this whenever we see a ToolCallContent in the update.

• TOOL_CALL_ARGS: We emit this for the ToolCallContent arguments.

• TOOL_CALL_END: We emit this right after the ToolCallContent.

• TOOL_CALL_RESULT: We emit this when we see the ToolCallResultContent on a message.

• STATE_SNAPSHOT: We map this as a DataContent with type application/json.

  • This is a convention that we defined to map from Microsoft.Extensions.AI to AG-UI.

• STATE_DELTA: We map this as a DataContent with type application/json-patch+json.

  • This is a convention that we defined to map from Microsoft.Extensions.AI to AG-UI.

There are however other events we don't support because we don't have a reasonable way to map them.

• STEP_STARTED: Signals the start of a step within an agent run.
• STEP_FINISHED: Signals the completion of a step within an agent run.
• MESSAGES_SNAPSHOT: Provides a snapshot of all messages in a conversation.
• This is useful to sync the UI with existing conversations.
• ACTIVITY_SNAPSHOT: Delivers a complete snapshot of an activity message.
• This represents a step that is happening as part of a run, like "PLAN" or "SEARCH" and includes
  custom information related to that activity.
• ACTIVITY_DELTA: Provides incremental updates to an activity snapshot using JSON Patch.
• RAW: Used to pass through events from external systems.
• CUSTOM: Used to pass application specific events.

We don't have good ways of representing this information using Microsoft.Extensions.AI concepts. There are three ways that we can go about this.

• Create/make public the AG-UI event types that we use internally. This way, people can provide a AgentRunResponseUpdate with a RawRepresentation being the AG-UI event and we can take that at face value and serialize it.

  • We would need to add public API for this and define a strategy for evolving it.
  • One option is to mark all these types as [Experimental] on top of the package preview so we can choose when to "graduate them".
  • The implications of changing/moving them if we ever want to do so are:
    • Apps will have to install a new library/recompile (this might be fine).
    • Libraries that build on top will have to recompile and publish a new version against the new updated location.

• Create a set of custom AIContent subtypes to represent certain updates (the equivalent of what foundry does in the SDK, like with RunUpdate https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/ai/Azure.AI.Agents.Persistent/src/Custom/Streaming/RunUpdate.cs#L13 and StreamingUpdateReason https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/ai/Azure.AI.Agents.Persistent/src/Custom/Streaming/StreamingUpdateReason.cs#L11

  • Not great because we would need to add to the JsonSerializationOptions used to serialize and deserialize AIContent. Doable but potentially painful.

• Abuse DataContent as a general-purpose mechanism for delivering updates.

  • We could follow the same approach we do for STATE_SNAPSHOT and STATE_DELTA and take it one step further. If you create a data content and define it with a content type of application/ag-ui-<<event-name>>+json we will take that payload at face value and serialize it into the event stream.
  • It won't require public API.
  • It will support all scenarios, current and future.
  • It provides a terrible user experience.

At some point we need to make a decision as there are asks that are depending on this.

#2270
#2494
#2510

[stephentoub]

If we need to do something here, my preference is option 1 ("Create/make public the AG-UI event types that we use internally."), ideally with the types as part of a dedicated AG-UI library, which the agent layer wraps (as it does today).

[MaciejSzczepanskiRedslim]

The frustrating part is that we wanted to consume an AG-UI endpoint exposed by the Python Agent Framework from the .NET SDK (python endpoint produced: MESSAGES_SNAPSHOT). This discrepancy between the two implementations shouldn’t exist. One of the key advantages of the framework is the ability to switch between different technology stacks while keeping a consistent API and familiar concepts.

[yinyihao1]

I saw support for the AG UI protocol on the official website. However, it wasn't until I spent a week discussing the technical details with the front-end engineers that I discovered so many event types are currently unsupported.

I hope support will be added soon.

[paulguz-datapa]

The failure to fully support AG-UI (and what is a protocol if it is not implemented fully?) drives me away from using .Net for agentic front-ends.