Skip to content

Add OpenAIRequestPolicies extension hook for MEAI OpenAI clients#7495

Merged
jozkee merged 2 commits intodotnet:mainfrom
rogerbarreto:feature/openai-responses-request-options-factory
Apr 29, 2026
Merged

Add OpenAIRequestPolicies extension hook for MEAI OpenAI clients#7495
jozkee merged 2 commits intodotnet:mainfrom
rogerbarreto:feature/openai-responses-request-options-factory

Conversation

@rogerbarreto
Copy link
Copy Markdown
Contributor

@rogerbarreto rogerbarreto commented Apr 28, 2026

Summary

Adds a new experimental OpenAIRequestPolicies type that downstream SDKs can use to append System.ClientModel.PipelinePolicy instances to the RequestOptions built by Microsoft.Extensions.AI for every outbound OpenAI request. The motivating scenario: an SDK author is handed an IChatClient constructed by a customer from an OpenAIClient they do not own, and needs to brand the outgoing User-Agent header (or attach correlation metadata) without mutating the customer's pipeline.

Surface

The type is reachable via GetService<T> on the three MEAI OpenAI clients that already funnel through the ToRequestOptions chokepoint (OpenAIChatClient, OpenAIEmbeddingGenerator, OpenAIResponsesChatClient):

[Experimental(""MEAI001"")]
public sealed class OpenAIRequestPolicies
{
    public void AddPolicy(PipelinePolicy policy, PipelinePosition position = PipelinePosition.PerCall);
}

// Usage:
chatClient.GetService<OpenAIRequestPolicies>()?.AddPolicy(myUserAgentPolicy);

Customer policies run after MEAI's internal User-Agent policy, so message.Request.Headers.Set(""User-Agent"", ...) replaces the value and Headers.Add(...) stacks an additional value, with predictable last-writer-wins semantics. The instance is per client (no shared state, no weak tables) and reachable through any ChatClientBuilder decorator chain via the standard GetService traversal.

Design choices

  • Append-only: the surface is a single AddPolicy method. Remove and Clear are intentionally deferred until a real consumer asks; the experimental attribute keeps that door open.
  • Locked-down access: callers cannot reach the underlying RequestOptions, so CancellationToken, BufferResponse, and MEAI's own User-Agent policy stay safe from accidental tampering.
  • Storage: lock-free reads on a copy-on-write T[] swapped via Interlocked.CompareExchange. No System.Collections.Immutable dependency (would not flow through netstandard2.0 / net462).
  • Diagnostic ID: reuses the generic MEAI001 (AIExperiments) alias rather than OPENAI001. OPENAI001 is reserved for surfaces that wrap the OpenAI SDK's experimental APIs; this hook is generic MEAI extensibility.
  • Scope: limited to the three clients with the existing ToRequestOptions chokepoint. The other OpenAI MEAI clients (Realtime, S2T, T2S, Image, Assistants, HostedFile) bypass ToRequestOptions today, do not currently receive MEAI's User-Agent either, and would each require their own refactor (or a websocket-specific story for Realtime). Out of scope here; revisit if asked.

Why not configure the underlying OpenAIClient?

The OpenAI .NET SDK already supports OpenAIClientOptions.AddPolicy at construction time. That is the right path when you control construction. This PR addresses the case where you do not, the SDK author receives only an IChatClient and must add behavior without reaching into the customer's pipeline configuration.

Tests

New OpenAIRequestPoliciesTests (10 tests, all passing on net462 / net8 / net9 / net10):

  • AddPolicy(null) throws ArgumentNullException.
  • GetService<OpenAIRequestPolicies>() returns a stable per-client instance on each of the three wired client types.
  • Two MEAI client wrappers around the same OpenAIClient get distinct OpenAIRequestPolicies instances.
  • The instance is reachable through a ChatClientBuilder decorator chain.
  • A custom policy that calls Headers.Set(""User-Agent"", ...) replaces MEAI's value (proves ordering: customer-after-MEAI).
  • A custom policy that calls Headers.Add(""User-Agent"", ...) stacks alongside MEAI's value.
  • When no policy is registered, MEAI's own User-Agent is still emitted (regression guard).
  • 200 concurrent AddPolicy invocations retain all entries (CAS-loop concurrency smoke).

Existing 356 Microsoft.Extensions.AI.OpenAI.Tests continue to pass.

API baseline

Microsoft.Extensions.AI.OpenAI.json regenerated via ./scripts/MakeApiBaselines.ps1. The version line moves from 10.5.0.0 to 10.6.0.0 because the previous baseline predates a repo-wide MinorVersion bump in eng/Versions.props; the sibling Abstractions.json baseline was already at 10.6.0.0.

Microsoft Reviewers: Open in CodeFlow

Introduces a new experimental sealed type OpenAIRequestPolicies retrievable via IChatClient.GetService<T>() / IEmbeddingGenerator.GetService<T>() on the three Microsoft.Extensions.AI OpenAI clients that go through the ToRequestOptions chokepoint (OpenAIChatClient, OpenAIEmbeddingGenerator, OpenAIResponsesChatClient).

The type exposes a single AddPolicy(PipelinePolicy, PipelinePosition = PerCall) method so downstream SDKs that receive a customer-built IChatClient (and therefore cannot reconfigure the underlying OpenAIClient pipeline) can append their own pipeline policies, for example to stamp or replace the User-Agent header. Customer policies run after MEAI's internal user-agent policy, so Headers.Set replaces and Headers.Add stacks.
Copilot AI review requested due to automatic review settings April 28, 2026 16:36
@rogerbarreto rogerbarreto requested review from a team as code owners April 28, 2026 16:36
@github-actions github-actions Bot added the area-ai Microsoft.Extensions.AI libraries label Apr 28, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an experimental extensibility hook (OpenAIRequestPolicies) to the MEAI OpenAI adapters so downstream components can append PipelinePolicy instances to every RequestOptions built at the ToRequestOptions chokepoint (e.g., for User-Agent branding / correlation headers) without mutating the underlying OpenAIClient pipeline.

Changes:

  • Introduces experimental OpenAIRequestPolicies with an append-only AddPolicy API and internal ApplyTo(RequestOptions) integration.
  • Wires per-client OpenAIRequestPolicies instances into OpenAIChatClient, OpenAIEmbeddingGenerator, and OpenAIResponsesChatClient and applies them after MEAI’s internal User-Agent policy.
  • Adds unit tests validating service reachability/stability, per-client isolation, ordering semantics, and basic concurrent registration behavior; updates API baseline.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIRequestPoliciesTests.cs Adds coverage for policy registration, GetService exposure, ordering semantics, and concurrent add smoke test.
src/Shared/DiagnosticIds/DiagnosticIds.cs Adds a named experiment constant for the new experimental surface under the shared AI experiments diagnostic ID.
src/Libraries/Microsoft.Extensions.AI.OpenAI/RequestOptionsExtensions.cs Extends ToRequestOptions to optionally apply caller-registered OpenAIRequestPolicies after MEAI’s internal policies.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIChatClient.cs Exposes a per-client OpenAIRequestPolicies via GetService and applies it to request options.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIEmbeddingGenerator.cs Exposes a per-generator OpenAIRequestPolicies via GetService and applies it to request options.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs Exposes a per-client OpenAIRequestPolicies via GetService and applies it to request options for create/get + streaming reflection paths.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIRequestPolicies.cs New experimental type implementing copy-on-write policy registration and application to RequestOptions.
src/Libraries/Microsoft.Extensions.AI.OpenAI/Microsoft.Extensions.AI.OpenAI.json Updates API baseline (version + new OpenAIRequestPolicies type).

Copy link
Copy Markdown
Member

@jozkee jozkee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense for cases where you don't control OpenAIClient creation.

Comment thread test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIRequestPoliciesTests.cs Outdated
@jozkee
Copy link
Copy Markdown
Member

jozkee commented Apr 28, 2026

Scope: limited to the three clients with the existing ToRequestOptions chokepoint. The other OpenAI MEAI clients (Realtime, S2T, T2S, Image, Assistants, HostedFile) bypass ToRequestOptions today, do not currently receive MEAI's User-Agent either, and would each require their own refactor (or a websocket-specific story for Realtime). Out of scope here; revisit if asked.

Will it be possible to use OpenAIRequestPolicies for the clients currently out of scope? Just asking to make sure that this will be compatible for those and no other RequestPolicies types will be needed.

…tion message

Replaces ThrowUserAgentExceptionHandler with a CapturingUserAgentHandler that records request.Headers.UserAgent.ToString() and asserts on the captured value, addressing jozkee's feedback that Message-based assertions were too loose. Also surfaced that the runtime User-Agent is 'OpenAI/x.y.z (...) MEAI/x.y.z' (OpenAI prepends, MEAI appends), so the no-policy test now uses Contains rather than StartsWith.
@rogerbarreto
Copy link
Copy Markdown
Contributor Author

rogerbarreto commented Apr 28, 2026

Will it be possible to use OpenAIRequestPolicies for the clients currently out of scope? Just asking to make sure that this will be compatible for those and no other RequestPolicies types will be needed.

@jozkee, Yes, no additional types should be needed. I checked the openai-dotnet public surface and every one of the currently out-of-scope clients exposes public protocol methods that accept a RequestOptions:

  • AudioClient.GenerateSpeechAsync / TranscribeAudioAsync / TranslateAudioAsync(BinaryContent, string, RequestOptions)
  • ImageClient.GenerateImagesAsync / GenerateImageEditsAsync / GenerateImageVariationsAsync(..., RequestOptions)
  • AssistantClient.CreateRunAsync / CreateMessageAsync / CancelRunAsync / CreateThreadAsync / etc., all (..., RequestOptions)
  • OpenAIFileClient.UploadFileAsync / DownloadFileAsync / GetFileAsync / DeleteFileAsync (and the upload/multipart family) (..., RequestOptions)
  • ContainerClient.CreateContainerFileAsync / DeleteContainerFileAsync / etc. (..., RequestOptions)
  • RealtimeSessionClient.SendCommandAsync / ReceiveUpdatesAsync(RequestOptions)

@jozkee jozkee merged commit 086fa24 into dotnet:main Apr 29, 2026
6 checks passed
jeffhandley pushed a commit that referenced this pull request May 1, 2026
* Add OpenAIRequestPolicies extension hook for MEAI OpenAI clients

Introduces a new experimental sealed type OpenAIRequestPolicies retrievable via IChatClient.GetService<T>() / IEmbeddingGenerator.GetService<T>() on the three Microsoft.Extensions.AI OpenAI clients that go through the ToRequestOptions chokepoint (OpenAIChatClient, OpenAIEmbeddingGenerator, OpenAIResponsesChatClient).

The type exposes a single AddPolicy(PipelinePolicy, PipelinePosition = PerCall) method so downstream SDKs that receive a customer-built IChatClient (and therefore cannot reconfigure the underlying OpenAIClient pipeline) can append their own pipeline policies, for example to stamp or replace the User-Agent header. Customer policies run after MEAI's internal user-agent policy, so Headers.Set replaces and Headers.Add stacks.

* Address review: capture request headers instead of asserting on exception message

Replaces ThrowUserAgentExceptionHandler with a CapturingUserAgentHandler that records request.Headers.UserAgent.ToString() and asserts on the captured value, addressing jozkee's feedback that Message-based assertions were too loose. Also surfaced that the runtime User-Agent is 'OpenAI/x.y.z (...) MEAI/x.y.z' (OpenAI prepends, MEAI appends), so the no-policy test now uses Contains rather than StartsWith.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai Microsoft.Extensions.AI libraries

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants