Skip to content

FunctionInvokingChatClient breaks tool_callstool adjacency in service-managed mode when the caller supplies trailing messages after an approval response #7616

Description

@westey-m

Description

When FunctionInvokingChatClient (FICC) resumes an approval-gated function call — the
caller sends back a ToolApprovalResponseContent to continue — it rebuilds the
assistant(FunctionCallContent) message for the approved call. In service-managed
history mode
(a non-empty ConversationId is set on the continuation request), FICC
omits that reconstructed assistant(tool_calls) message, because it assumes the
service already holds it. However, the executed tool result (FunctionResultContent) is
still appended to the end of the outgoing message list.

If the caller supplies any message after the approval response on the continuation turn,
that trailing message ends up wedged between the (service-held) assistant(tool_calls)
and the appended tool result:

assistant(tool_calls)  ->  user(<caller-supplied trailing message>)  ->  tool(result)

Providers that enforce tool_callstool adjacency (e.g. OpenAI ChatCompletion) reject this with
HTTP 400:

An assistant message with 'tool_calls' must be followed by tool messages responding to
each 'tool_call_id'.

The bug manifests only when both:

  1. FICC is in service-managed mode (non-empty ConversationId), and
  2. the caller supplies at least one message after the ToolApprovalResponseContent.

A plain FICC over a stateless Chat Completions client with no ConversationId does not
reproduce it, because FICC inserts the reconstructed assistant(tool_calls) right before
the tool result, keeping them adjacent.

Reproduction Steps

Three conditions are required:

  1. FICC is in service-managed mode — a non-empty ConversationId on the continuation
    request.
  2. The caller supplies at least one message after the ToolApprovalResponseContent.
  3. (For the exact 400) the service actually holds the assistant(tool_calls) so it
    precedes the trailing message and tool result.

Expected behavior

When resuming an approved call in service-managed mode, the FunctionResultContent should
be positioned so it stays adjacent to the assistant message carrying the matching
tool_call_id (the one the service holds) — not appended after unrelated caller-supplied
messages. Equivalently, FICC should not leave a trailing caller message wedged between the
(service-held) assistant(tool_calls) and the tool result.

The valid outgoing ordering is:

1. system     "You are a helpful assistant ..."
2. user       "get the current path"
3. assistant  tool_calls: get_current_path  id=call_kTM...
4. tool       result                        tool_call_id=call_kTM...
5. user       "By the way, please keep the answer concise."

Actual behavior

The tool result is appended to the tail of the outgoing message list, after any
caller-supplied trailing messages. In service-managed mode (where FICC omits the
reconstructed assistant(tool_calls) and relies on the service to hold it), this places
the trailing message between the assistant tool_calls and its tool result:

Mode Effective outgoing messages Result
no ConversationId (client-managed) … user(extra), assistant(tool_calls), tool(result) assistant → tool adjacentvalid
ConversationId set (service-managed) assistant(tool_calls), user(extra), tool(result) — assistant held by service trailing message breaks adjacency → invalid

The provider (e.g. OpenAI) fails the request with HTTP 400:

An assistant message with 'tool_calls' must be followed by tool messages responding to
each 'tool_call_id'. The following tool_call_ids did not have response messages: call_kTM...

The defect is message positioning — the function executes correctly; only the placement
of its result in the outgoing message list is wrong.

Regression?

Not known to be a regression.

Known Workarounds

Do not send any additional caller-supplied messages after the ToolApprovalResponseContent
on the continuation turn when operating in service-managed mode. Send the approval response
by itself, let FICC complete the tool round-trip, and only then send further messages on a
subsequent turn.

Configuration

  • Package: Microsoft.Extensions.AIFunctionInvokingChatClient.
  • Inner client / service: an IChatClient that manages conversation state (returns a
    ConversationId and holds prior turns), backed by a provider enforcing
    tool_callstool adjacency (e.g. OpenAI Chat Completions).
  • Independent of OS/.NET version; specific to the service-managed usage
    pattern described above.

Other information

Root cause. Resumed approvals are handled before the main loop in
FunctionInvokingChatClient.ProcessFunctionApprovalResponses. The reconstructed
assistant(tool_calls) message is only added to the outgoing list when there is no
conversation id:

// FunctionInvokingChatClient.cs — ProcessFunctionApprovalResponses
preDownstreamCallHistory = [.. allPreDownstreamCallMessages]; // the assistant(tool_calls)
if (!hasConversationId)                                        // <-- the gate
{
    originalMessages.AddRange(preDownstreamCallHistory);
}

The executed tool result is subsequently appended to the end of the same list
(ProcessFunctionCallsAsyncmessages.AddRange(addedMessages)). In service-managed mode
that tail-append places the result after any caller-supplied trailing messages, breaking
adjacency with the service-held assistant(tool_calls).

Suggested fix direction. Position the reconstructed tool-call/tool-result messages so
the tool result stays adjacent to the assistant message that owns the matching
tool_call_id, with any trailing caller-supplied messages after the tool result — rather
than always appending the result at the tail. This applies to both the streaming and
non-streaming approval paths. A fix along these lines has been prepared and validated
against the existing approval test suite.

Related types. ApprovalRequiredAIFunction, ToolApprovalRequestContent,
ToolApprovalResponseContent, FunctionCallContent, FunctionResultContent.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue describes a behavior which is not expected - a bug.untriaged

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions