How should an Agent expose an underlying IChatClient without Agent middleware?

[1448376744 (soul-soft)]

How should an Agent expose an underlying IChatClient without Agent middleware?

I would like to discuss whether an Agent should provide a supported way to access the underlying model IChatClient without the agent middleware pipeline.

This is useful for host-side tasks that are related to the current agent session but should not run as part of the agent loop. A common example is generating a short conversation title for the current agent session.

Today, ChatClientAgent.ChatClient may return the original client, but by default it returns a decorated pipeline. Some decorators are only valid inside an active AIAgent.RunAsync / RunStreamingAsync context. For example, context providers and message injection rely on AIAgent.CurrentRunContext or the current AgentSession.

That makes it awkward to reuse the agent's model client for auxiliary host work:

• The host wants to use the same provider/model/configuration as the agent.
• The host does not want tool invocation, approval handling, message injection, session mutation, or agent context providers.
• Calling the decorated ChatClient outside an agent run can fail because some middleware requires an active agent context.
• Keeping a separate copy of the original IChatClient in host code works, but it is easy for configuration to drift from the agent.

UseProvidedChatClientAsIs = true does not solve this scenario. It gives the agent the raw client, but it also disables the default agent pipeline for normal agent execution. What I need is different: the agent should still use its full pipeline when running, while the host can also access the base model client for auxiliary calls.

Would it make sense to expose a clearer API for this scenario?

For example:

IChatClient rawClient = agent.GetBaseChatClient();

or, if this should only apply to ChatClientAgent:

IChatClient rawClient = chatClientAgent.InnerChatClient;
IChatClient pipelineClient = chatClientAgent.ChatClient;

The exact shape is not important. The important distinction is that there are two different use cases:

• the agent pipeline client, used by the agent loop
• the base model client, used for auxiliary host-side calls

Having an explicit API would make these scenarios easier to implement without depending on middleware ordering or private construction details.

[1448376744 (soul-soft)]

westey (@westey-m) Roger Barreto (@rogerbarreto) would appreciate your thoughts on this API shape, since it affects ChatClientAgent and the
default agent middleware pipeline.

[westey (westey-m)]

1448376744 (@soul-soft), thanks for the question.

The problem with requiring ChatClientAgent to expose the leaf ChatClient is that it doesn't necessarily have access to this either. When using UseProvidedChatClientAsIs = true, the expectation is that the developer would have built a custom stack, which would include decorators for Function Invocation and potentially many more. In this case ChatClientAgent faces the same issue as you are describing.

Even with UseProvidedChatClientAsIs = true the ChatClient provided may not be a leaf node. It could already be decorated or be a ChatClient that switches between other chat clients.

I believe that the only option that will work reliably is option 4 from your list: Keeping a separate copy of the original IChatClient in host code works, but it is easy for configuration to drift from the agent..
Even if we didn't have the abovementioned problems, I think option 4 makes more sense from a design perspective. Using an agent as a store for a ChatClient in this way is requiring it to perform a function that is outside of the goal of an agent. This is essentially straying into dependency injection territory.

Aside: Do note that if you know the type of the leaf ChatClient (e.g. OpenAIResponsesChatClient) you could get a reference to that via AIAgent.GetService or IChatClient.GetServce, e.g. agent.GetService<OpenAIResponsesChatClient>(). This would only be useful if you were always using a single type of leaf ChatClient of course.