.NET: [Feature]: Native Async Tool/Function Support — Agent Suspend & Resume on Long-Running Tool Calls

[saikir1994]

Description

What problem does it solve?

Currently, all tool/function invocations in the Agent Framework are synchronous within the agent's request-response cycle. When the LLM requests a function call, FunctionInvokingChatClient immediately invokes it, waits for the result, and feeds it back — all within a single RunAsync call. This doesn't work for real-world async operations where a tool triggers an external process (API call, workflow, human task) that takes seconds to hours to complete. The agent should not hold a connection/thread waiting; it should yield control and resume when the result arrives.

The existing requireApproval (human-in-the-loop) pattern is close — it suspends the agent via ExternalInputRequest/ExternalInputResponse — but it gates before the tool call (accept/reject). This feature would use the same suspend/resume principle but for the post-dispatch phase: the tool call is actually made, then the agent suspends until the result comes back.

What would the expected behavior be?

The pattern should mirror exactly how ApprovalRequiredAIFunction / FunctionApprovalRequestContent works today for HITL, but for async tool dispatch:

How HITL approval works today (for reference):

1. Tool is wrapped in ApprovalRequiredAIFunction
2. When the LLM requests that tool, FunctionInvokingChatClient does NOT execute it — instead it returns immediately with FunctionApprovalRequestContent in the response messages
3. The caller parses response.Messages for FunctionApprovalRequestContent items
4. The caller decides (approve/reject), then calls approvalRequest.CreateResponse(approved) to create a FunctionApprovalResponseContent
5. The caller sends the response back: agent.RunAsync([new ChatMessage(ChatRole.Tool, [approvalResponse])], session)
6. The framework maps the response back by CallId, executes (or skips) the tool, and the LLM continues

Proposed async tool pattern (same shape, different semantics):

1. Tool is wrapped in AsyncAIFunction (analogous to ApprovalRequiredAIFunction)
2. When the LLM requests that tool, the framework dispatches the actual call (the tool's method runs, sends request to external system, returns a correlation/tracking ID)
3. The agent returns immediately with AsyncFunctionRequestContent in the response messages — containing the CallId, function name, arguments, and the dispatch result (e.g., tracking ID)
4. The caller parses response.Messages for AsyncFunctionRequestContent items, stores the pending call info
5. Later, when the external system provides the result, the caller calls asyncRequest.CreateResponse(resultPayload) to create an AsyncFunctionResponseContent
6. The caller sends the response back: agent.RunAsync([new ChatMessage(ChatRole.Tool, [asyncResponse])], session)
7. The framework maps the response back by CallId, the async function's registered callback processes the result, and it becomes available to the LLM to continue

Key difference from HITL: The tool method does execute during step 2 (dispatching the request), but the final result is not yet available. The response message signals "I started this work, here's how to identify it — send me the result when it's done."

Caller                          Agent Framework                    LLM / External System
  |                                   |                                  |
  |-- RunAsync(messages) ------------>|                                  |
  |                                   |-- LLM requests tool call         |
  |                                   |-- Detect AsyncAIFunction wrapper |
  |                                   |-- Execute tool (dispatch only) -->|  (sends request)
  |                                   |-- Tool returns tracking ID       |
  |                                   |-- Return AsyncFunctionRequestContent
  |<-- AgentResponse ----------------|                                  |
  |    { Messages containing          |                                  |
  |      AsyncFunctionRequestContent  |                                  |
  |      (callId, name, trackingId) } |                                  |
  |                                   |                                  |
  |  (caller stores pending call,     |                                  |
  |   time passes...)                 |                                  |
  |                                   |                                  |
  |  (external system completes) <----|----------------------------------| 
  |                                   |                                  |
  |  asyncRequest.CreateResponse(result)                                 |
  |-- RunAsync([ChatMessage(          |                                  |
  |    ChatRole.Tool,                 |                                  |
  |    [asyncResponse])], session) -->|                                  |
  |                                   |-- Map by CallId                  |
  |                                   |-- Process via callback           |
  |                                   |-- Feed result to LLM             |
  |                                   |-- LLM continues conversation     |
  |<-- AgentResponse (final) ---------|                                  |

Are there any alternatives you've considered?

1. Using requireApproval as a workaround: Gate the call, have the external handler execute the function and return the result. But this conflates "approval" semantics with "async execution" and requires the external handler to know function implementation details or have workaround logic to pass in approval and internally mask the response as immediatly available.
2. DurableTask orchestrations: Works for durable scenarios but requires DurableTask infrastructure. Many deployments need lightweight async tools without orchestration overhead.
3. Custom middleware via FunctionInvocationDelegatingAgent: Could intercept the call and throw a "suspend" exception, but the framework has no first-class support for capturing and restoring the invocation state. This is the current approach that I am using and it needs some additional management code in the caller code.

Code Sample

1. Defining the async tool (analogous to ApprovalRequiredAIFunction)

// The tool method dispatches work and returns a tracking/correlation ID.
// It does NOT wait for the final result — that comes later via callback.
[Description("Submits order to fulfillment system")]
static async Task<string> SubmitOrder(string orderId, string[] items)
{
    // Sends request to external system, returns immediately with tracking ID
    var trackingId = await fulfillmentClient.SubmitAsync(orderId, items);
    return trackingId;
}

// Wrap with AsyncAIFunction — just like ApprovalRequiredAIFunction wraps for approval
AIAgent agent = chatClient.AsAIAgent(
    instructions: "You are a helpful order assistant",
    tools: [new AsyncAIFunction(AIFunctionFactory.Create(SubmitOrder))]);

2. Caller-side: detect async requests, collect results, re-invoke (mirrors HITL approval loop exactly)

// STEP 1: Call agent
AgentSession session = await agent.CreateSessionAsync();
AgentResponse response = await agent.RunAsync("Submit order #123 with items A, B, C", session);

// STEP 2: Detect async function requests (same pattern as FunctionApprovalRequestContent)
List<AsyncFunctionRequestContent> asyncRequests = response.Messages
    .SelectMany(m => m.Contents)
    .OfType<AsyncFunctionRequestContent>()
    .ToList();

// STEP 3: Loop until all async calls resolved (mirrors the approval while-loop)
while (asyncRequests.Count > 0)
{
    // BUILD RESPONSE MESSAGES — for each pending async call, wait for the result
    List<ChatMessage> asyncResponses = new();
    foreach (var asyncRequest in asyncRequests)
    {
        Console.WriteLine($"Async tool dispatched: {asyncRequest.FunctionCall.Name}");
        Console.WriteLine($"  Dispatch result (tracking ID): {asyncRequest.DispatchResult}");

        // ... wait for external system to complete (poll, webhook, queue, etc.) ...
        string externalResult = await WaitForExternalResult(asyncRequest.DispatchResult);

        // Create response — same pattern as approvalRequest.CreateResponse(approved)
        asyncResponses.Add(new ChatMessage(ChatRole.Tool,
            [asyncRequest.CreateResponse(externalResult)]));
    }

    // STEP 4: Send responses back to agent (same as sending approval responses)
    response = await agent.RunAsync(asyncResponses, session);

    // Check for more async requests (agent may chain multiple async tools)
    asyncRequests = response.Messages
        .SelectMany(m => m.Contents)
        .OfType<AsyncFunctionRequestContent>()
        .ToList();
}

// Final response — LLM has all results, conversation continues
Console.WriteLine($"\nAgent: {response}");

3. Streaming variant (mirrors HITL sample)

List<AIContent> asyncResponses = [];

do
{
    asyncResponses.Clear();
    List<AgentResponseUpdate> updates = [];

    await foreach (var update in agent.RunStreamingAsync(messages, session))
    {
        updates.Add(update);
        foreach (AIContent content in update.Contents)
        {
            switch (content)
            {
                case AsyncFunctionRequestContent asyncRequest:
                    asyncResponses.Add(asyncRequest);
                    break;
                case TextContent textContent:
                    Console.Write(textContent.Text);
                    break;
            }
        }
    }

    // Wait for all async results AFTER streaming completes
    for (int i = 0; i < asyncResponses.Count; i++)
    {
        var asyncRequest = (AsyncFunctionRequestContent)asyncResponses[i];
        string result = await WaitForExternalResult(asyncRequest.DispatchResult);
        asyncResponses[i] = asyncRequest.CreateResponse(result);
    }

    AgentResponse response = updates.ToAgentResponse();
    messages.AddRange(response.Messages);

    foreach (AIContent asyncResponse in asyncResponses)
    {
        messages.Add(new ChatMessage(ChatRole.Tool, [asyncResponse]));
    }
}
while (asyncResponses.Count > 0);

Language/SDK

• Both (.NET and Python — code examples above are .NET; the same pattern should apply to the Python SDK)

Additional Context

Design follows the established HITL approval pattern. The proposed implementation mirrors the existing ApprovalRequiredAIFunction / FunctionApprovalRequestContent / FunctionApprovalResponseContent flow exactly:

HITL Approval (exists today)	Async Tool (proposed)
ApprovalRequiredAIFunction wrapper	AsyncAIFunction wrapper
FunctionApprovalRequestContent in response	AsyncFunctionRequestContent in response
approvalRequest.CreateResponse(approved)	asyncRequest.CreateResponse(resultPayload)
FunctionApprovalResponseContent sent back	AsyncFunctionResponseContent sent back
Framework maps by CallId, executes/skips tool	Framework maps by CallId, processes result via callback
Caller loop: parse requests → respond → re-invoke	Caller loop: parse requests → get results → re-invoke

Intent to contribute: I intend to submit a PR implementing this feature. Proposed implementation approach:

1. New content types (in Microsoft.Extensions.AI or Agent Framework abstractions):

   • AsyncFunctionRequestContent — like FunctionApprovalRequestContent, carries CallId, FunctionCall, and DispatchResult (the tracking/correlation ID from the initial tool dispatch)
   • AsyncFunctionResponseContent — like FunctionApprovalResponseContent, carries CallId and the final result payload
   • CreateResponse(result) method on the request type for easy response construction

2. AsyncAIFunction wrapper (analogous to ApprovalRequiredAIFunction):

   • Wraps any AIFunction
   • When FunctionInvokingChatClient invokes it: executes the tool (dispatch), captures the return value as DispatchResult, then emits AsyncFunctionRequestContent instead of FunctionResultContent
   • When the caller sends back AsyncFunctionResponseContent, the framework maps by CallId and makes the final result available to the LLM

3. Streaming event generators (for OpenAI Hosting / DevUI):

   • AsyncFunctionRequestEventGenerator — emits response.async_function.requested events (mirrors response.function_approval.requested)
   • AsyncFunctionResponseEventGenerator — emits response.async_function.responded events

4. Declarative workflow support: async: true on InvokeFunctionTool, leveraging existing ExternalInputRequest/ExternalInputResponse plumbing

5. Sample & tests: Getting started sample mirroring Agent_Step04_UsingFunctionToolsWithApprovals, unit tests for the request/response lifecycle

I'd appreciate feedback on the design direction before starting the PR.