Request timeouts middleware

[Tratcher]

Background and Motivation

A common customer request is to be able to apply timeouts to their requests. AspNetCore servers don't do this by default since request times vary widely by scenario and we don't have good ways to predict that. E.g. WebSockets, static files, expensive APIs, etc..

• RequestTimeout in ASP.NET Core 3.1 is not availablewhen using InProcess Hosting #23160 - Out-of-proc had request timeouts.
• Kestrel execution timeout support #10079
• [Needs Tests] Add timeouts aspnet/KestrelHttpServer#485 (comment)
• How to specify / modify request timeout aspnet/KestrelHttpServer#611

To provide more control we can provide timeouts via middleware that are configured per-endpoint, as well as a global timeout if desired. These timeouts would link to the RequestAborted CancellationToken and be entirely cooperative. E.g. we won't call Abort when the timeout fires. It's up to the application logic to consume RequestAborted and decide how to handle the cancellation.

Proposed API

Project/Assembly: Microsoft.AspNetCore.Http

namespace Microsoft.Extensions.DependencyInjection;

+ public static class RequestTimeoutsIServiceCollectionExtensions
+ {
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services) { }
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services, Action<RequestTimeoutOptions> configure) { }
+     // We need to consider how these policies would integrate with IConfiguration, but that may be substantially different from this API.
+     // public static IServiceCollection AddRequestTimeouts(this IServiceCollection services, IConfigurationSection section) { }
+ }

namespace Microsoft.AspNetCore.Builder;

+ public static class RequestTimeoutsIApplicationBuilderExtensions
+ {
+     public static IApplicationBuilder UseRequestTimeouts(this IApplicationBuilder builder) { }
+ }

+ public static class RequestTimeoutsIEndpointConventionBuilderExtensions
+ {
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, TimeSpan timeout)  { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, string policyName) { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, RequestTimeoutPolicy policy) { }
+     public static IEndpointConventionBuilder DisableRequestTimeout(this IEndpointConventionBuilder builder) { }
+ }

+ namespace Microsoft.AspNetCore.Http.Timeouts;

+ public class RequestTimeoutOptions
+ {
+     // Applied to any request without a policy set. No value by default.
+     public TimeSpan? DefaultTimeout { get; set; }
+     public RequestTimeoutOptions AddPolicy(string policyName, TimeSpan timeout) { }
+     public RequestTimeoutOptions AddPolicy(string policyName, RequestTimeoutPolicy policy) { }
+     public bool TryGetPolicy(string policyName, out RequestTimeoutPolicy policy) { }
+     public void RemovePolicy(string policyName) { }
+ }

+ public class RequestTimeoutPolicy
+ {
+     public TimeSpan? Timeout { get; }
+     public RequestTimeoutPolicy(TimeSpan? timeout) { }
+ }

+ // Overrides the global timeout, if any.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class RequestTimeoutAttribute : Attribute
+ {
+    public TimeSpan? Timeout { get; }
+    public string? PolicyName { get; }
+    public RequestTimeoutAttribute(int seconds) { }
+    public RequestTimeoutAttribute(string policyName) { }
+ }

+ // Disables all timeouts, even the global one.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class DisableRequestTimeoutAttribute : Attribute
+ {
+    public DisableRequestTimeoutAttribute() { }
+ }

Usage Examples

services.AddRequestTimeout(options =>
{
    options.DefaultTimeout = TimeSpan.FromSeconds(10);
    options.DefaultTimeoutLocator = context => TimeSpan.Parse(context.Request.Query["delay"]);
    options.AddPolicy("MyTimeoutPolicy", TimeSpan.FromSeconds(1));
    options.AddPolicy("DynamicPolicy", new RequestTimeoutPolicy(context => TimeSpan.Parse(context.Request.Query["timeout"])));
});
...
app.UseRequestTimeouts();
...
app.UseEndpoints(endpoints => {
    endpoints
        .MapGet("/", (context) => { ... })
        .WithRequestTimeout(TimeSpan.FromSeconds(1));
app.UseEndpoints(endpoints => {
    endpoints
        .MapGet("/", (context) => { ... })
        .WithRequestTimeout("MyTimeoutPolicy");
...
    [RequestTimeout(seconds: 1)]
    public ActionResult<TValue> GetHello()
    {
        return "Hello";
    }
    [RequestTimeout("MyTimeoutPolicy")]
    public ActionResult<TValue> GetHello()
    {
        return "Hello";
    }
    [DisableRequestTimeout]
    public ActionResult<TValue> ImVerySlow()
    {
        Thread.Sleep(TimeSpan.FromHours(1);
        return "Hello";
    }

Alternative Designs

Risks

• We need to ensure this system is flexible enough to cover a wide variety of scenarios, while being easy enough for people to not mis-configure.
• What about components that don't use endpoints?
• What if the middleware is placed in the wrong location?

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[davidfowl]

1. What is the purpose of DefaultTimeoutLocator?
2. There should be an overload of WithRequestTimeout that directly takes a policy instance.
3. AddRequestTimeouts that takes the IConfigurationSection should be removed.

It feels a bit odd to have a mix of endpoint specific metadata that further filters the request to discover the timeout. The point of metadata is to bake in the decision about which endpoints have which metadata at startup so cheap decision can be made per request.

[Tratcher]

1. What is the purpose of DefaultTimeoutLocator?

This supports determining the timeout for any non-endpoint-aware resources like static files. It's optional for now, but I anticipate customers asking for it.

2. There should be an overload of WithRequestTimeout that directly takes a policy instance.

So that policy instance would get stored in the endpoint metadata and then checked for along with the attributes?

3. AddRequestTimeouts that takes the IConfigurationSection should be removed.

@sebastienros is working on an effort to make more of our components config aware, reloadable, etc. since this is a common requirement from real apps. They don't want to recompile to change timeouts. The API shape for that experience is negotiable. I don't think this can be removed without a proposed alternative.

It feels a bit odd to have a mix of endpoint specific metadata that further filters the request to discover the timeout. The point of metadata is to bake in the decision about which endpoints have which metadata at startup so cheap decision can be made per request.

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

[davidfowl]

This supports determining the timeout for any non-endpoint-aware resources like static files. It's optional for now, but I anticipate customers asking for it.

Whenever we add middleware that has predicate as part of the options, I question the need. We have routing matching and pipeline branching, why do we need middleware specific filters on top of that? This also came up when we were discussing output caching.

So that policy instance would get stored in the endpoint metadata and then checked for along with the attributes?

Right, much like the authorization policies

@sebastienros is working on an effort to make more of our components config aware, reloadable, etc. since this is a common requirement from real apps. They don't want to recompile to change timeouts. The API shape for that experience is negotiable. I don't think this can be removed without a proposed alternative.

I think that's fine, but I'm not a fan of making one off changes like this to new middleware. This proposal needs to be all encompassing, and we shouldn't add this API to this one place. As an example, the hosting filtering middleware supports config reloading and it doesn't take IConfiguration as input (which is good, it should be using IOptionsMonitor<T>). I don't want to end up in a situation like this again.

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

Endpoints have already been routed to (routing is the filtering). Routing can applies policies. e.g.

app.MapGet("/long", (Cancellation token) => client.GetSomethingAsync(token))
     .RequireHost("*:80");
      .WithTimeout(TimeSpan.FromSeconds(5));

The above endpoint is already filtered to GET requests with /long in the route and port 80.

[Tratcher]

I don't follow. Where is there further filtering? Endpoints store a specific timeout, a policy name for a centrally configured timeout, or an opt-out. Or are you expecting the policy name to be resolved to a timeout when building the endpoint?

Endpoints have already been routed to (routing is the filtering). Routing can applies policies. e.g.

app.MapGet("/long", (Cancellation token) => client.GetSomethingAsync(token))
     .RequireHost("*:80");
      .WithTimeout(TimeSpan.FromSeconds(5));

The above endpoint is already filtered to GET requests with /long in the route and port 80.

I'm still not sure which part of the design you're talking about? Which API is this for?

[Tratcher]

I've updated the proposal to remove the TimeoutLocator from RequestTimeoutPolicy, but left the one on RequestTimeoutOptions for further discussion.

I've commented out the IConfigurationSection API and left a note about considering the IConfiguration integration story.

I've also added a WithRequestTimeout overload that directly takes a RequestTimeoutPolicy.

[sebastienros]

1. If a RequestTimeoutPolicy's locator returns null, does the DefaultTimeoutLocator apply?
2. If the DefaultTimeoutLocator returns null, does the DefaultTimeout apply?
3. Named can be added and fetched. Should we be able to delete them? Probably fine not to be, since I assume we'd only want to be able to change one using TryGet, if nothing is using it it can remain in the list.
4. Duration in seconds in the attribute seems limiting. Would keep TimeSpan at least too.
5. You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

[Tratcher]

1. If a RequestTimeoutPolicy's locator returns null, does the DefaultTimeoutLocator apply?

Removed.

2. If the DefaultTimeoutLocator returns null, does the DefaultTimeout apply?

No?

3. Named can be added and fetched. Should we be able to delete them? Probably fine not to be, since I assume we'd only want to be able to change one using TryGet, if nothing is using it it can remain in the list.

We should consider Update/Remove as part of the reloadable configuration story.

4. Duration in seconds in the attribute seems limiting. Would keep TimeSpan at least too.

TimeSpan isn't a supported input for attributes. I wanted to avoid string input as both error prone and it would conflict with the policy overload.

5. You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

👍

[davidfowl]

You mention DefaultTimeoutLocator to handle things like static files. Something that was discussed in Output Caching too, is how we should be able to trigger these middleware when there is no specific endpoint, for specific routing patterns that would be combined with the selected endpoint. For instance be able to do app.AddEndpointMetadata("/static/css/*"}).WithTimeout(TimeSpan.FromSeconds(5)). Could even be using regexes if required. The metadata would then be added to the resolved endpoint's metadata. The same way metadata can be applied on endpoint "groups".

#43642

[Tratcher]

Removed DefaultTimeoutLocator.

Added RemovePolicy, though at this point maybe we should be using a thread safe collection instead.

[Kahbazi]

@Tratcher Is this issue up for grab?

[Kahbazi]

How about also adding DisableRequestTimeout on IEndpointConventionBuilder?

+ public static class RequestTimeoutsIEndpointConventionBuilderExtensions
+ {
+     public static IEndpointConventionBuilder DisableRequestTimeout(this IEndpointConventionBuilder builder) { }
+ }

[Kahbazi]

Also Why does RequestTimeoutPolicy.Timeout need to be nullable? What would happen if it is null? Should it behave the same as if it has DisableRequestTimeoutAttribute metadata?

[Tratcher]

@Tratcher Chris Ross FTE Is this issue up for grab?

Not until we finish the API review.

[ghost]

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[Tratcher]

How about also adding DisableRequestTimeout on IEndpointConventionBuilder?

+ public static class RequestTimeoutsIEndpointConventionBuilderExtensions
+ {
+     public static IEndpointConventionBuilder DisableRequestTimeout(this IEndpointConventionBuilder builder) { }
+ }

Added

[Tratcher]

Also Why does RequestTimeoutPolicy.Timeout need to be nullable? What would happen if it is null? Should it behave the same as if it has DisableRequestTimeoutAttribute metadata?

Null would allow you to conditionally/programatically disable the policy without updating all of the endpoints to DisableRequestTimeout. Timeout.InfiniateTimeSpan is an alternative, but I'd like to avoid special values.

[halter73]

API Review Notes:

• Is AddRequestTimeouts(this IServiceCollection services) necessary? Maybe not. Will it reset anything? No. It's idempotent.
• What happens if you set a policy that's already set? It probably throws. Or it overwrites. Let's make it consistent auth middleware.
• Do we want the policies to be enumerable? Can we make it a dictionary?
• Are seconds granular enough for request timeouts? Maybe not. TimeSpan doesn't work with attributes. Let's make it milliseconds.
• Let's leave the IConfigurationSection binding for a later API review, but we should consider it in the shape of our options type, but a straightforward bind is unlikely to work.
• Is this going to be a new project/assembly? No. Put it in Microsoft.AspNetCore.Http.
• DisableRequestTimeoutAttribute always wins no matter the order, then policy instance, then RequestTimeoutAttribute, then global default.

namespace Microsoft.Extensions.DependencyInjection;

+ public static class RequestTimeoutsIServiceCollectionExtensions
+ {
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services) { }
+     public static IServiceCollection AddRequestTimeouts(this IServiceCollection services, Action<RequestTimeoutOptions> configure) { }
+ }

namespace Microsoft.AspNetCore.Builder;

+ public static class RequestTimeoutsIApplicationBuilderExtensions
+ {
+     public static IApplicationBuilder UseRequestTimeouts(this IApplicationBuilder builder) { }
+ }

+ public static class RequestTimeoutsIEndpointConventionBuilderExtensions
+ {
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, TimeSpan timeout)  { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, string policyName) { }
+     public static IEndpointConventionBuilder WithRequestTimeout(this IEndpointConventionBuilder builder, RequestTimeoutPolicy policy) { }
+     public static IEndpointConventionBuilder DisableRequestTimeout(this IEndpointConventionBuilder builder) { }
+ }

+ namespace Microsoft.AspNetCore.Http.Timeouts;

+ public sealed class RequestTimeoutOptions
+ {
+     // Applied to any request without a policy set. No value by default.
+     public TimeSpan? DefaultTimeout { get; set; }
+     public RequestTimeoutOptions AddPolicy(string policyName, TimeSpan timeout) { }
+     public RequestTimeoutOptions AddPolicy(string policyName, RequestTimeoutPolicy policy) { }
+     public IDictionary<string, RequestTimeoutPolicy> Policies { get; }
+ }

+ public sealed class RequestTimeoutPolicy
+ {
+     public TimeSpan? Timeout { get; }
+     public RequestTimeoutPolicy(TimeSpan? timeout) { }
+ }

+ // Overrides the global timeout, if any.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class RequestTimeoutAttribute : Attribute
+ {
+    public TimeSpan? Timeout { get; }
+    public string? PolicyName { get; }
+    public RequestTimeoutAttribute(int milliseconds) { }
+    public RequestTimeoutAttribute(string policyName) { }
+ }

+ // Disables all timeouts, even the global one.
+ [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
+ public sealed class DisableRequestTimeoutAttribute : Attribute
+ {
+    public DisableRequestTimeoutAttribute() { }
+ }

API Approved!

[halter73]

I know we just approved IDictionary<string, RequestTimeoutPolicy> Policies, but would NamedPolicies be better since it doesn't include policies defined without a name like those added directly with builder.WithRequestTimeout(RequestTimeoutPolicy)? I can send an email to the api review alias if we think we should change this.

[Tratcher]

Policies seems fine, the fact that you need a name to add/get them seems self documenting that the others are inaccessible.

[Tratcher]

@Kahbazi, this should be ready, any questions before you get started?

[davidfowl]

Why is there no callback API on AddPolicy? Is it because it only has the timeout?

[Tratcher]

RequestTimeoutPolicy

RequestTimeoutPolicy is currently designed to be immutable which conflicts with the callback pattern. Which would you prefer?

[Kahbazi]

@Kahbazi, this should be ready, any questions before you get started?

Everything looks clear for now. I'll start coding!

[Tratcher]

Bumping back to API review.

There's a customer ask for making the timeout response configurable. There's already a question in the PR about what the default status code should be, so being able to configure at least that seems obvious. Allowing a RequestDelegate to generate the response seems the most flexible.

+ public int? TimeoutStatusCode { get; set; }
+ public RequestDelegate WriteTimeoutResponse { get; set; }

It would be called here:
https://github.com/dotnet/aspnetcore/pull/46115/files#diff-2da119fe32b8dd3db23382b34e7bb138a2c5c0cde95b970aef87602548bb7ad4R94

Question: Should this be on RequestTimeoutOptions, RequestTimeoutPolicy, or both?

[ghost]

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[halter73]

API Review Notes:

• Should this be on RequestTimeoutOptions, RequestTimeoutPolicy, or both? We probably want per-policy configuration.
• Can we add a "default" RequestTimeoutOptions RequestTimeoutPolicy? This would require a mutable policy type or setter. Or setting a "default" policy by name?
• Do we want to revisit a non-null default timeout? No. We do not want to timeout every request by default.

API Approved! - @Kahbazi

+ public sealed class RequestTimeoutOptions
+ {
+     // Applied to any request without a policy set. No value by default.
+     public RequestTimeoutPolicy? DefaultPolicy { get; set; }

+     public RequestTimeoutOptions AddPolicy(string policyName, TimeSpan timeout) { }
+     public RequestTimeoutOptions AddPolicy(string policyName, RequestTimeoutPolicy policy) { }
+     public IDictionary<string, RequestTimeoutPolicy> Policies { get; }
+ }

+ public sealed class RequestTimeoutPolicy
+ {
+     public TimeSpan? Timeout { get; init; }
+     public int? TimeoutStatusCode { get; init; }
+     public RequestDelegate? WriteTimeoutResponse { get; init; }
+ }

[Kahbazi]

API Approved! - @Kahbazi

I'm on it!

[Tratcher]

Here's an addition to address long running request scenarios that aren't endpoint aware and need to opt-out of the timeout.

public interface IHttpRequestTimeoutFeature
{
  void CancelTimeout();
}

• This is only added to requests with a timeout enabled (default or endpoint).
• Can be called multiple times, no-ops on subsequent calls.
• Has no effect if the timeout has already fired.
• I was originally thinking bool TryCancelTimeout(), but there was too much ambiguity around returning false if the timeout had already fired vs if there was no timeout.

We'll review this on Monday.

@davidfowl

[halter73]

Are we going to use this issue to review the proposed IHttpRequestTimeoutFeature? If so, can we remark this as ready-for-review along with any other issues intended for the one-off API review Monday?

[ghost]

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

• The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
• The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
• Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

[halter73]

API Review Notes:

• Will the IHttpRequestTimeoutFeature be set if timeouts are disabled. No?
• If we added a boolean property this could be a set feature even without a timeout in progress, but we don't think this is necessary
• So what do we think about IHttpRequestTimeoutFeature vs IRequestTimeoutFeature? IHttp is a common prefix for HTTP-related features?
• You cannot disable a timeout with CancelTimeout(). The timeout needs to be in progress for this to work.
• What do we think about CancelRequestTimeout vs RequestTimeout? DisableTimeout?
• Should CancelTimeout() return a bool? Can we detect if a full response has already been flushed? Cancel is supposed to be cooperative. Can we expose the timeout CancellationToken on the feature? From @Tratcher earlier comment "I was originally thinking bool TryCancelTimeout(), but there was too much ambiguity around returning false if the timeout had already fired vs if there was no timeout."
• If we expose RequestTimeoutToken is exposed, can we still pool it? Yes when middleware unwinds.

var timeoutFeature = context.Features.Get<IHttpRequestTimeoutFeature>();
timeoutFeature.DisableTimeout();
if (!timeoutFeature.RequestTimeoutToken.IsCancellationRequested)
{
    // Timeout was successfully disabled
}

API Approved!

namespace Microsoft.AspNetCore.Http.Timeouts;

+ public interface IHttpRequestTimeoutFeature
+ {
+     CancellationToken RequestTimeoutToken { get; }
+     void DisableTimeout();
+ }