Skip to content

[Blazor] Add Components.AI client tool rendering - #68325

Open
javiercn wants to merge 4 commits into
javiercn-components-ai-02-rich-textfrom
javiercn-components-ai-03-client-tools
Open

[Blazor] Add Components.AI client tool rendering#68325
javiercn wants to merge 4 commits into
javiercn-components-ai-02-rich-textfrom
javiercn-components-ai-03-client-tools

Conversation

@javiercn

@javiercn javiercn commented Aug 10, 2026

Copy link
Copy Markdown
Member

Overview

Position 3 in native stack #68340, this layer depends on #68324 and adds browser-owned tool execution to the chat/rich-text foundation. Across four commits and 17 files, it introduces a provider-neutral client-action contract, continues the same conversation after the browser returns a tool result, and lights up the canonical Agentic Chat change_background action. The cross-cutting constraint is that executable client functions stay inside the Blazor circuit: the model receives declarations only, while DojoClient still reaches AGUIDojoApi through AGUI.Client 0.0.5, HTTP/SSE, and AGUI.Server 0.0.5.

Design

// src/Components/AI/src/Pipeline/UIAgentOptions.cs
// Public registration associates an exact tool name with code owned by this UIAgent instance.
// The implementation stores the function for browser invocation; it is not handed to the server as executable code.
public void RegisterUIAction(AIFunction function)
{
    ArgumentNullException.ThrowIfNull(function);
    UIActions.Add(function.Name, function);
}
// src/Components/AI/src/Blocks/UIActionBlock.cs
// Public render contract: consumers can inspect the model call, render by ToolName, invoke once,
// and observe the correlated FunctionResultContent that resumes the conversation.
public class UIActionBlock : ContentBlock
{
    public FunctionCallContent Call { get; }
    public string ToolName => Call.Name;
    public FunctionResultContent? Result { get; private set; }
    public bool IsComplete => Result is not null;

    public Task InvokeAsync(CancellationToken cancellationToken = default)
    {
        lock (_invocationLock)
        {
            // Two renderer callbacks receive the same Task; the browser action executes once.
            _invocation ??= InvokeCoreAsync(cancellationToken);
            return _invocation;
        }
    }
}

The action name dictionary uses ordinal matching, so change_background is claimed while CHANGE_BACKGROUND remains unhandled; this avoids accidentally dispatching a different tool. Registered actions are added to cloned ChatOptions with AsDeclarationOnly(), preserving any existing server tools without exposing browser closures to the API. All client actions share this one contract and handler pattern; change_background is the representative canonical use rather than a special-purpose product API.

Implementation

// src/Components/AI/src/Pipeline/UIActionHandler.cs
// The pipeline claims only FunctionCallContent whose exact name was registered on this agent.
// Every registered action follows this path; unmatched calls pass to later handlers unchanged.
if (content is FunctionCallContent call &&
    _actions.TryGetValue(call.Name, out var function))
{
    context.MarkHandled(call);
    return BlockMappingResult<State>.Emit(
        new UIActionBlock(function, call)
        {
            // Preserve the model call ID so the continuation result correlates over AG-UI.
            Id = call.CallId ?? Guid.NewGuid().ToString("N")
        },
        state);
}
// src/Components/AI/src/Engine/AgentContext.cs
// A turn streams normally until one or more UIActionBlocks appear, then pauses for browser results.
ChatMessage? currentMessage = message;
while (currentMessage is not null)
{
    var actions = new List<UIActionBlock>();

    await foreach (var block in _agent.SendMessageAsync(currentMessage, cancellationToken)
        .WithCancellation(cancellationToken))
    {
        // Existing request/response block routing is unchanged (omitted).
        if (block is UIActionBlock action)
        {
            actions.Add(action);
        }
    }

    if (actions.Count == 0)
    {
        currentMessage = null;
        continue;
    }

    Status = ConversationStatus.AwaitingInput;
    NotifyStatusChanged();

    // One action or many parallel actions are the same equivalence class: wait for all,
    // then send one tool-role message containing exactly one result per action.
    var results = await Task.WhenAll(
        actions.Select(action => action.GetResultAsync(cancellationToken)));
    currentMessage = new ChatMessage(ChatRole.Tool, [.. results]);

    Status = ConversationStatus.Streaming;
    NotifyStatusChanged();
}
@* src/Components/AI/testassets/DojoClient/Components/Scenarios/AgenticChat/AgenticChatScenario.razor *@
@* The renderer auto-invokes only the canonical action, and ChangeBackground mutates component-local state,
   which makes the effect circuit-local rather than server-global. *@
<BlockRenderer TBlock="UIActionBlock"
               Context="action"
               When="@(action => action.ToolName == "change_background")">
    <AutoInvokeAction Block="action" />
</BlockRenderer>

@code {
    private async Task<string> ChangeBackground(string background)
    {
        _background = background;
        await InvokeAsync(StateHasChanged);
        return "Background changed successfully.";
    }
}

The deterministic browser fixture replaces only the API's underlying model client. Its recording asserts the change_background declaration on the first request and exactly one matching tool-result call ID on the continuation; AGUI.Server serialization, HTTP/SSE transport, AGUI.Client parsing, product mapping, and Blazor invocation remain real. Test cases are compressed into two equivalence classes: product tests cover registration/matching/single execution/continuation, while browser tests cover visible completion and isolation between two circuits.

Outcome

Equivalence class Proven behavior
Product contract Declaration-only client tools preserve configured server tools, match names ordinally, invoke once, transition Streaming → AwaitingInput → Streaming → Idle, and resume with one correlated result.
Canonical Agentic Chat change_background updates the owning circuit, streams confirmation text after the continuation, and leaves a second circuit unchanged.
Transport/replay Only the API model is recorded; the separate DojoClient/AGUIDojoApi AG-UI HTTP/SSE boundary is exercised end to end.

Review guidance: read the declaration-only option construction, exact-name handler, UIActionBlock single-invocation gate, and AgentContext continuation loop closely. The action styling and generated recording need only a spot-check for scenario identity and sensitive-data absence.

Acceptance criteria: one circuit changes only its own background; the model receives the declared change_background tool and exactly one result with the original call ID; confirmation text arrives after that continuation.

Validation: dependency-aware build 0 warnings / 0 errors; AgentContextUIActionTests 4/4 passed; AgenticChatScenarioTests 5/5 passed.

@javiercn
javiercn requested a review from a team as a code owner August 10, 2026 17:15
Register browser-owned functions as declaration-only chat tools and map matching model calls into UIActionBlock instances that custom Blazor renderers can invoke exactly once. The product remains provider-neutral and does not expose executable client functions to the server transport.

Continue the same conversation turn after every client action completes by sending one tool-result message back through IChatClient. AwaitingInput and Streaming transitions make the pause observable while preserving tool-name matching and per-agent action isolation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 323b0ef8-7905-4041-8388-a08586c0bd34
Register the canonical change_background declaration in the Agentic Chat UI and render its UIActionBlock with a circuit-local auto-invoker. The action updates only the owning Blazor circuit and returns a tool result so the agent can continue with confirmation text.

Teach the credential-free API model to request the same sunset-gradient action and confirm its result. DojoClient still reaches AGUIDojoApi through AGUI.Client 0.0.5 over HTTP/SSE, with no in-process model shortcut.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 323b0ef8-7905-4041-8388-a08586c0bd34
Cover ordinal tool-name matching, declaration-only registration, preservation of configured tools, single invocation, AwaitingInput transitions, and exactly one continuation result in focused product tests.

Extend API-side replay to validate tool declarations and continuation results, then exercise change_background through the real dual-host AG-UI transport. The browser coverage verifies the action completes with confirmation text and cannot mutate a second Blazor circuit; its generated recording remains separate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 323b0ef8-7905-4041-8388-a08586c0bd34
Record the declaration-validated change_background call and the single-result continuation used by the Agentic Chat client-tool browser tests. Replay still replaces only the API model, leaving AGUI.Server, HTTP/SSE, AGUI.Client, and the Blazor circuit action on the real path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 323b0ef8-7905-4041-8388-a08586c0bd34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant