.NET: ChatClientAgent having internal ChatOptions is overly restrictive

[PederHP]

ChatOptions being internal means that IChatClient-based agents are restricted to being very static in nature. For example:

• Claude-based agents cannot change extended thinking parameters on the fly.
• OpenAI-based agents cannot change reasoning effort levels.
• Tools and ToolMode are static

Having agents that change extended thinking / reasoning effort levels based on user input or other signals is a common pattern, and used in many coding agents. Dynamic Tool strategies are also relatively common.

Making ChatOptions public and mutable would enable a lot of patterns. Instructions should also be mutable - having a dynamic system prompt / instructions is also a very common pattern.

[stephentoub]

Note that you can send a different ChatOptions in as part of a ChatClientAgentRunOptions on each request, and it will be merged with the agent-level settings for that request.

[PederHP]

Note that you can send a different ChatOptions in as part of a ChatClientAgentRunOptions on each request, and it will be merged with the agent-level settings for that request.

True, and I did consider if that was enough, but it's awkward to keep a parallel datastructure and making sure it's passed whenever the agent is run. When deriving from ChatClientAgent (and keeping the overrides in the derived class state) it means having to merge with incoming ChatClientAgentRunOptions before passing them on.

Also for collections it will unionize and not replace the property, which I think is the right choice for per-request overrides, but not really for a mutable agent.

I couldn't really see any downside to making them mutable, as I think stateful agents are pretty common pattern. It's much simpler and I don't see any reason an agent should be restricted to immutability (although that is a reasonable choice for some architectures, but not all).

[stephentoub]

(You can also insert ConfigureOptions as part of the IChatClient.)

Encouraging mutating the agent seems like a footgun. Serialized tests will show it all working fine, but then expected concurrent use will lead to hard to diagnose failures.

[PederHP]

But that won't work for instructions will it? And again, it's very awkward for something that is trivial to do when using a custom wrapper around an IChatClient, instead of agent. I see your point about concurrency, though.

Maybe an AgentThread specific set of overrides?

[stephentoub]

But that won't work for instructions will it?

ConfigureOptions? Why not?

[stephentoub]

The options effectively define what the agent is. If you need to change instructions and tools, and you don't want to do it per request but instead for the agent as a whole, wouldn't you really just want a new agent instance at that point?

[PederHP]

But that won't work for instructions will it?

ConfigureOptions? Why not?

Because they are part of ChatClientAgentOptions, so the ConfigureOptions on the IChatClient can't change them?

The options effectively define what the agent is. If you need to change instructions and tools, and you don't want to do it per request but instead for the agent as a whole, wouldn't your really just want a new agent instance at that point?

That's a very narrow definition of agent. Claude Code for example will change thinking parameters during a thread. And dynamic system prompt is something that I use a lot when building agents.

[PederHP]

Maybe a MutableChatClientAgent for those of us who don't mind handling the concurrency? It simplifies so many things just being able to modify Instructions and ChatOptions.

[stephentoub]

Because they are part of ChatClientAgentOptions, so the ConfigureOptions on the IChatClient can't change them?

At the end of the day the ChatClientAgent is a thin wrapper around an IChatClient that's submitting everything to an IChatClient. Anything it sends will be through either messages or options. A ConfigureOptions then in the IChatClient pipeline has full control over changing what those options are as they flow through.

[PederHP]

Because they are part of ChatClientAgentOptions, so the ConfigureOptions on the IChatClient can't change them?

At the end of the day the ChatClientAgent is a thin wrapper around an IChatClient that's submitting everything to an IChatClient. Anything it sends will be through either messages or options. A ConfigureOptions then in the IChatClient pipeline has full control over changing what those options are as they flow through.

Ah yes, my bad, I had forgotten that Instructions are also a property on ChatOptions (and presumably mapped to ChatRole.System messages or the request-level System property based on provider).

Ok, so deriving from ChatClientAgent and using ConfigureOptions to be able change options would work (except I think it might not play nice with request-based overrides). But it's still awkward as it happens outside of managing the lifecycle of the agent object.

I think having the option of having a MutableChatClientAgent is preferable, as it's much cleaner to work with for developers who want a mutable agent (and are prepared to handle the concurrency issues). I'll change my PR accordingly. I can see why you might not want mutability as the default behavior, but I think it's a reasonable type of agent to want.

Another argument is that the behavior of merging request ChatOptions with agent ChatOptions is relatively complex for some of them (especially as null is both a valid value and no-value for some of them). It's much more intuitive to mutate the agent ChatOptions when that is the desired behavior.

[stephentoub]

@westey-m