[API Proposal, Microsoft.Extensions.Http.Resilience]: API for removing and replacing the standard resilience/hedging handlers

[iliar-turdushev]

Background and motivation

.NET provides the ConfigureHttpClientDefaults method that allows you to configure the default behavior of the HttpClient. With the help of this method you can register the StandardResiliencePipeline as part of the default configuration of the HttpClient:

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

This is convenient, since you don't need to configure the resilience pipeline for each HttpClient. But with the API we currently provide users face a few issues/challenges when they need to change the default configuration. Below are a couple of known issues/challanges.

Issue 1: Removing the existing default resilience pipeline and adding a custom one

For example, a user registers the StandardResilienceHandler as default configuration, but at the same time he/she wants to use the StandardHedgingHandler for a particular named HttpClient:

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to remove the StandardResilienceHandler and add instead the StandardHedgingHandler.
services.AddHttpClient("custom")./*Remove StandardResilienceHandler*/.AddStandardHedgingHandler();

Currently, there is no API that allows users to remove the StandardResilienceHandler and register a new one. Now users can work that around as described here #4814 (comment).

Issue 2: Override configuration of the default resilience pipeline for a particular HttpClient

A user registers the StandardResilienceHandler as default configuration, and wants to provide custom configuration for a particular HttpClient:

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to customize settings of the StandardResilienceHandler.
services.AddHttpClient("custom")./*Configure default StandardResilienceHandler*/;

Currently, there is no API to override the default configuration, see #4814.

Issue 3: Replace the default resilience pipeline with a custom one

In oppose to the "Issue 1" a user wants to replace the default resilience pipeline with a custom one. He/she wants to replace it, because it might be important to keep the order where the default resilience pipeline was registered, for example, it could be important for metering purposes.

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to replace the StandardResilienceHandler with a custom one.
services.AddHttpClient("custom")./*We want to replace default StandardResilienceHandler, i.e. the new pipeline will be in the same order as the default one*/;

Supporting such scenario could be useful to fix issues like the following #5021. Because the default resilience pipeline and its state are shared by all HttpClient instances it could lead to issues, like the one mentioned in the previous sentence. And having an ability to replace the default pipeline with a custom one for a particular HttpClient could be a solution to that issue.

API Proposal

Remove all resilience handlers:

namespace Microsoft.Extensions.DependencyInjection;

public static class ResilienceHttpClientBuilderExtensions
{
+   /// <summary>
+   /// Removes all resilience handlers registered earlier.
+   /// </summary>
+   /// <param name="builder">The HTTP client builder.</param>
+   /// <returns>The value of <paramref name="builder"/>.</returns>
+   public static IHttpClientBuilder RemoveAllResilienceHandlers(this IHttpClientBuilder builder);
}

Add or replace the StandardResilienceHandler:

namespace Microsoft.Extensions.DependencyInjection;

public static class ResilienceHttpClientBuilderExtensions
{
+    /// <summary>
+    /// If there is no standard Resilience or Hedging handlers registered,
+    /// then the method adds the standard Resilience handler to the HttpClient request pipeline.
+    /// Otherwise, the existing standard Resilience or Hedging handler is replaced with the standard Resilience handler.
+    /// </summary>
+    /// <param name="builder">The HTTP client builder.</param>
+    /// <returns>A <see cref="IHttpStandardResiliencePipelineBuilder"/> instance that can be used to configure the standard resilience handler behavior.</returns>
+    public static IHttpStandardResiliencePipelineBuilder AddOrReplaceStandardResilienceHandler(
+        this IHttpClientBuilder builder);

+    /// <summary>
+    /// If there is no standard Resilience or Hedging handlers registered,
+    /// then the method adds the standard Resilience handler to the HttpClient request pipeline.
+    /// Otherwise, the existing standard Resilience or Hedging handler is replaced with the standard Resilience handler.
+    /// </summary>
+    /// <param name="builder">The HTTP client builder.</param>
+    /// <param name="section">The section that the options will bind against.</param>
+    /// <returns>A <see cref="IHttpStandardResiliencePipelineBuilder"/> instance that can be used to configure the standard resilience handler behavior.</returns>
+    public static IHttpStandardResiliencePipelineBuilder AddOrReplaceStandardResilienceHandler(
+        this IHttpClientBuilder builder,
+        IConfigurationSection section);

+    /// <summary>
+    /// If there is no standard Resilience or Hedging handlers registered,
+    /// then the method adds the standard Resilience handler to the HttpClient request pipeline.
+    /// Otherwise, the existing standard Resilience or Hedging handler is replaced with the standard Resilience handler.
+    /// </summary>
+    /// <param name="builder">The HTTP client builder.</param>
+    /// <param name="configure">The callback that configures the options.</param>
+    /// <returns>A <see cref="IHttpStandardResiliencePipelineBuilder"/> instance that can be used to configure the standard resilience handler behavior.</returns>
+    public static IHttpStandardResiliencePipelineBuilder AddOrReplaceStandardResilienceHandler(
+        this IHttpClientBuilder builder,
+        Action<HttpStandardResilienceOptions> configure);
}

Add or replace the StandardHedgingHandler:

namespace Microsoft.Extensions.DependencyInjection;

public static partial class ResilienceHttpClientBuilderExtensions
{
+    /// <summary>
+    /// If there is no standard Resilience or Hedging handlers registered,
+    /// then the method adds the standard Hedging handler to the HttpClient request pipeline.
+    /// Otherwise, the existing standard Resilience or Hedging handler is replaced with the standard Hedging handler.
+    /// </summary>
+    /// <param name="builder">The HTTP client builder.</param>
+    /// <returns>A <see cref="IStandardHedgingHandlerBuilder"/> instance that can be used to configure the standard hedging behavior.</returns>
+    public static IStandardHedgingHandlerBuilder AddOrReplaceStandardHedgingHandler(
+        this IHttpClientBuilder builder);

+    /// <summary>
+    /// If there is no standard Resilience or Hedging handlers registered,
+    /// then the method adds the standard Hedging handler to the HttpClient request pipeline.
+    /// Otherwise, the existing standard Resilience or Hedging handler is replaced with the standard Hedging handler.
+    /// </summary>
+    /// <param name="builder">The HTTP client builder.</param>
+    /// <param name="configure">Configures the routing strategy associated with this handler.</param>
+    /// <returns>A <see cref="IStandardHedgingHandlerBuilder"/> instance that can be used to configure the standard hedging behavior.</returns>
+    public static IStandardHedgingHandlerBuilder AddOrReplaceStandardHedgingHandler(
+        this IHttpClientBuilder builder,
+        Action<IRoutingStrategyBuilder> configure);
}

API Usage

With the help of the proposed API the issues described in the beginning could be fixed as following.

Issue 1: Removing the existing default resilience pipeline and adding a custom one

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to remove the StandardResilienceHandler and add instead the StandardHedgingHandler.
services.AddHttpClient("custom")
    .RemoveAllResilienceHandlers()
    .AddStandardHedgingHandler();

Issue 2: Override configuration of the default resilience pipeline for a particular HttpClient

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to customize settings of the StandardResilienceHandler.
services.AddHttpClient("custom")
    .AddOrReplaceStandardResilienceHandler(options =>
    {
        // Configure options here.
    });

Issue 3: Replace the default resilience pipeline with a custom one

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to replace the StandardResilienceHandler with a custom one. There are 2 cases:

// Case 1: we replace the original StandardResilienceHandler with a new StandardResilienceHandler with custom configuration options.
// This case is similar to Issue 2.
services.AddHttpClient("custom")
    .AddOrReplaceStandardResilienceHandler(options =>
    {
        // Configure options here.
    });

// Case 2: we replace the original StandardResilienceHandler with the StandardHedgingHandler.
services.AddHttpClient("custom").AddOrReplaceStandardHedgingHandler();

Alternative Designs

The API design above considers HttpClient's delegating handlers as a list (the order of the items is important), therefore it introduces the AddOrReplace methods that implicitly "say" that the standard resilience handler (which is a delegating handler) will be replaced if any. Otherwise, it will be added to the end of the list.

The alternative design suggests not to introduce AddOrReplace methods, but instead to implement AddOrReplace semantics in the existing Add methods. The justification behind that is: having more than one standard resilience handlers in the HttpClient is INCORRECT.

Since it is not correct to have more than one standard resilience handlers in the HttpClient, we consider reasonable to implement AddOrReplace semantics in the existing Add methods:

• if there is already a standard resilience handler in the HttpClient, then we replace it and reconfigure
• otherwise, we add a new handler to the HttpClient

With such an approach we'll need to introduce the only method RemoveAllResilienceHandlers mentioned above, and then the issues described above could be fixed as following:

Issue 1: Removing the existing default resilience pipeline and adding a custom one

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to remove the StandardResilienceHandler and add instead the StandardHedgingHandler.
services.AddHttpClient("custom")
    .RemoveAllResilienceHandlers()
    .AddStandardHedgingHandler();

Issue 2: Override configuration of the default resilience pipeline for a particular HttpClient

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to customize settings of the StandardResilienceHandler.
services.AddHttpClient("custom")
    .AddStandardResilienceHandler(options =>
    {
        // Configure options here.
    });

Issue 3: Replace the default resilience pipeline with a custom one

services.ConfigureHttpClientDefaults(builder => builder.AddStandardResilienceHandler());

// For a named HttpClient "custom" we want to replace the StandardResilienceHandler with a custom one. There are 2 cases:

// Case 1: we replace the original StandardResilienceHandler with a new StandardResilienceHandler with custom configuration options.
// This case is similar to Issue 2.
services.AddHttpClient("custom")
    .AddStandardResilienceHandler(options =>
    {
        // Configure options here.
    });

// Case 2: we replace the original StandardResilienceHandler with the StandardHedgingHandler.
services.AddHttpClient("custom").AddStandardHedgingHandler();

Risks. It could be confusing for customers that Add methods has AddOrReplace semantics. We'll need to properly document that.

Risks

There are a few risks:

1. Names of the methods AddOrReplace... are not perfect

Names AddOrReplaceStandardResilienceHandler and AddOrReplaceStandardHedgingHandler are not perfect. The suffixes ReplaceStandardResilienceHandler and ReplaceStandardHedgingHandler might bring confusion that the methods replace only the corresponding standard resilience handler type, i.e. either the Resilience handler or the Hedging handler. In fact each method replaces any of the standard handlers. So the names of the methods don't fully describe what the methods do. The name that will fully describe the methods would be AddStandardResilienceHandler_Or_ReplaceAnyStandardHandlerWithStandardResilienceHandler, but it is too long, therefore we have to find some balance.

2. We'll need to introduce a breaking change to the behavior of the existing methods registering standard Resilience and Hedging handlers

In order to introduce AddOrReplace methods we'll need to ensure that HttpClient request pipeline has only one Resilience or Hedging handler. For that we'll need to change the behavior of the existing methods registering Resilience and Hedging handlers in HttpClient request pipeline. In particular, the new behavior will throw an exception when a user attempts to register one of those handlers twice. Currently, we don't throw an exception but register the handler twice which is also not correct. Therefore this risk could be not so high/important.

3. Additional effort to maintaining new AddOrReplace methods

1. With the introduction of new AddOrReplace methods we'll have to maintain them, meaning that adding some capability to existing AddStandardResilienceHandler methods would result in mirroring it to the AddOrReplace methods.

2. Currently we have 2 standard handlers: Resilience and Hedging. Probably, introducing a new standard handler would lead to creating similar AddOrReplace methods to have feature parity.

[martintmk]

Looks great!

Just one note here. Given that having multiple standard resilience handlers is anti-pattern and almost always a bug, rather than introducing AddOrReplaceStandardHedgingHandler and AddOrReplaceStandardResilienceHandler, should we just update the behavior of existing extensions to use the same logic.

Essentially, calling AddStandardResilienceHandler always ensures the replacement if previously registered one (if any).

[mobratil]

Looks good to me. I agree with @martintmk, if having multiple handlers is a bug anyway, I think it's a legit to change the behavior of existing methods.

[iliar-turdushev]

Just one note here. Given that having multiple standard resilience handlers is anti-pattern and almost always a bug, rather than introducing AddOrReplaceStandardHedgingHandler and AddOrReplaceStandardResilienceHandler, should we just update the behavior of existing extensions to use the same logic.

Good point @martintmk. Actually, I was proposing that in this comment #5021 (comment). But then I started thinking of HttpClient's delegating handlers as a list, therefore I decided that it could be confusing that the method AddStandardResilienceHandler replaces existing handler. That is why I proposed AddOrReplace methods. If not thinking of the delegating handlers as a list, then implementing AddOrReplace semantics in Add method seems better, since it helps us avoid introducing more APIs and don't introduce changes that could lead to runtime exceptions.

[iliar-turdushev]

@joperezr @RussKie please, take a look. Thank you.

[RussKie]

@iliar-turdushev, I'm marking this as ready for the ARB, please coordinate with @terrajobst for the suitable time.

[iliar-turdushev]

@RussKie At the moment, I'd like to continue discussion here. Especially, given the comments above (#5695 (comment) and #5695 (comment)) we might end up introducing the only method RemoveAllResilienceHandlers.

BTW Do we really need for every new API have that online API review session? As far as I know, for some APIs that are not that complex, we can discuss and approve it without bringing it to the online API review session, right?

[RussKie]

BTW Do we really need for every new API have that online API review session? As far as I know, for some APIs that are not that complex, we can discuss and approve it without bringing it to the online API review session, right?

The rules are somewhat fluent. But in general, if we're adding an API which more-or-less looks or behaves like an existing API (e.g., provide a new implementation to an interface, which already has other implementations), then we can have a discussion about the nuances of the implementation and pretty much rubber-stamp it. However, for brand new API we, generally, want the API Review Board to grace with their approval because the ARB may provide us with perspectives we may miss.

This proposal looks to me as though it falls in the latter category. Please correct me, if I misread it.

[terrajobst]

My understanding is that System.* and Microsoft.Extensions.* need API approval for every single API.

We generally don't review Microsoft.*, such as ASP.NET Core, EF Core etc.

[iliar-turdushev]

This proposal looks to me as though it falls in the latter category. Please correct me, if I misread it.

@RussKie Thank you for the information. No, you didn't misread that :). I think that it would be helpful to have API Review team's perspective on the proposed API.

[iliar-turdushev]

My understanding is that System.* and Microsoft.Extensions.* need API approval for every single API.

@terrajobst I just want to clarify. By saying Microsoft.Extensions.* need API approval for every single API, do you mean that we need to go through the online API review session for every new API? Or is the process more like @RussKie described it in this comment #5695 (comment)? Thank you.