Middleware Pipeline backed by OptionsMonitor to be reloaded at runtime.

[dazinator]

Is your feature request related to a problem? Please describe.

When configuring the middleware pipeline on application startup, there are a lot of things that can get built, based on the application configuration at that moment in time. Here is a contrived example, real application middleware pipelines are obviously much more complex:

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                //app.UseWebAssemblyDebugging();
            }
            else
            {

                app.UseExceptionHandler("/Error");

                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                if (Configuration["UseHsts"] == "True")
                {
                    app.UseHsts();
                }
            }

1. Once the RequestDelegate has been built, there is no opportunity to respond to any configuration change and re-build it at runtime. For example, if the configuration value for UseHsts was changed shown above, it would be good for the change in this configuration to cause a new RequestDelegate to be built based on latest configuration.

2. It would be nice imho to be able to use / leverage the options pattern, so one can have an options instance passed in when configuring / building the middleware pipeline. Note in the case above, the mixed access to various sources that impact the way the pipeline is built, like: Configuration["UseHsts"] but also env.IsDevelopment() plus customers might also bind options instances directly to various config sections etc - it would be perhaps easier to reason about more complex pipelines if values influencing the control flow of the pipeline could be consolidated into one TOptions object like this:

    public class PipelineOptions
    {
        public bool UseDeveloperExceptionPage { get; set; }
        public bool UseHsts { get; set; }
    }

 

Describe the solution you'd like

Here are a couple of very loose suggestions, for how this feature might look, but I don't want to come across as prescriptive in terms of the API's here..

1. I'd like to be able to use a TOptions instance when building the middleware pipeline.
2. I'd like for that to happen automatically whenever TOptions changes at runtime.

This means I'd probably need some way of registering this somewhere - perhaps in ConfigureServices:

        public void ConfigureServices(IServiceCollection services)
        {
            // Registers an action that can be used to configure a pipeline that will be called whenever TOptions changes.
            services.ConfigureRequestDelegate<PipelineOptions>(Configuration, Configure); // registring this is pointless by itself, something would have to monitor for changes and use this factory etc.
        }


        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env, PipelineOptions options)
        {
            // Note: PipelineOptions has been passed in reflecting up to date / current options from which I can build the pipeline.
            if (options.UseDeveloperExceptionPage)
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                if (options.UseHsts) // I get to use my options class everywhere important. Much cleaner in large pipelines that use a lot of config.
                {
                    app.UseHsts();
                }
            }  
        }

    public class PipelineOptions
    {
        public bool UseDeveloperExceptionPage { get; set; }
        public bool UseHsts { get; set; }
    }

To make this less verbose, its possible the Hosting layer could add something smart for detecting Configure() methods that should be backed by Options Manager / Monitor (I always forget which one of those is the one!) by detecting a variant of a Configure() method with a signature like this indicating a TOptions type:

        /// This method would be called at application startup but also whenever `PipelineOptions` changes.
        public void Configure<PipelineOptions>(IApplicationBuilder app, IWebHostEnvironment env, PipelineOptions options)
        {
            // Note: PipelineOptions has been passed in reflecting up to date / current options from which I can build the pipeline.
            if (options.UseDeveloperExceptionPage)
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                if (options.UseHsts) // I get to use my options class everywhere important. Much cleaner in large pipelines that use a lot of config.
                {
                    app.UseHsts();
                }
            }  
        }

If that method was detected, then by convention it could configure the TOptions instance (perhaps to some default config section prefix like "PipelineOptions:"?) and then register a factory service to provide RequestDelegate at runtime which would be backed by OptionsMonitor to return current version of RequestDelegate and rebuld it when changes occur.

Additional context

Add any other context or screenshots about the feature request here.

[Tratcher]

Rebuilding the pipeline might be overkill. Another approach would be to directly reconfigure the middleware options. This is possible today but we've only enabled it in a few middleware. Here's one example:

aspnetcore/src/Middleware/HostFiltering/src/HostFilteringMiddleware.cs

Lines 53 to 59 in ca1f9f5

	_optionsMonitor.OnChange(options =>
	{
	// Clear the cached settings so the next EnsureConfigured will re-evaluate.
	_options = options;
	_allowedHosts = new List<StringSegment>();
	_allowAnyNonEmptyHost = null;
	});

I do have another sample that shows how to add and remove individual or groups of middleware on-demand.
https://github.com/Tratcher/MiddlewareInjector/blob/master/MiddlewareInjector/Startup.cs

[dazinator]

I think rebuilding the pipeline becomes necessary when you have config settings for things like "enable / disable saml authentication" etc, i.e depending on these types of settings, dictates whether certain middleware should even be present in the pipeline or not (not all middleware has enable or disable options), and can sometimes also dictate order. At the moment without developing something similar to what I have suggested, you have to restart the website to apply these types of changes.

Still, I have just spent a couple of hours and managed to put something together that works well, involving 3 classes its not like the framework doesn't make it possible! Its just a little bit of extra work.. The approach I took was to add a middleware that forks the main IApplicationBuilder and builds its own RequestDelegate from a monitored TOptions. The middleware just forwards requests through that cached request delegate, but there is a lock placed when a rebuild is in progress so that it can't build more than once simultaneously. It consists of about 3 classes and I am happy to continue using my solution, but I do wonder if it would be a helpful function to have out of the box. Being able to do this for particular chunks of the pipeline like my middleware does is actually quite nice as I guess thats more efficient than just applying it globall, I.e rebuilding the entire pipeline when most of it may not be subject to any change.

[dazinator]

@Tratcher interesting - a sample on adding groups of middleware on demand? I'll take a closer look at that, perhaps my assumptions regarding needing to rebuild a pipeline to enable / disable particular middleware was wrong! I guess there multiple possible solutions.

[Tratcher]

Note things like authentication are controlled primarily through services and already offer their own dynamic reload capabilities. The middleware is only an execution point.

I'd be curious to see your system in action. Are you interested in sharing a sample repo? That would also help gauge interest.

We only add features to the framework that we've seen many requests for and have a clear grasp of the requirements. So far there's been limited interest in rebuilding the pipeline, and the needs were quite custom.

[davidfowl]

We wouldn't build this in as it affects the performance for everyone for the rare case that you're requesting. I know you're building an uber plugin system with multi tenancy on top of ASP.NET Core and it might make sense to share more of the overall picture rather than filing these very specific issues that we've not motivated to fix.

I'm personally extremely sensitive to anything that can negatively impact the performance for everyone (that might not be the case here). I'm also interested in understanding the impact across the stack if we choose to go forward with something like this.

[dazinator]

rather than filing these very specific issues that we've not motivated to fix.

This is a feature suggestion not a bug report - just to be clear, I'm not suggesting any kind of fix. Just wanted to gauge interest on whether something like this could/ should be present in asp.net core framework. I am not sure there is any wider context I can add that wouldn't be off topic.

Middleware pipeline today is built often with configuration as a dependency. Therefore to me it was natural to want to rebuild it (or forks / branches of it) when configuration changes rather than try to address this in individual middlewares.

@Tratcher

I'd be curious to see your system in action. Are you interested in sharing a sample repo? That would also help gauge interest.

Ok will do!

[dazinator]

@Tratcher
I've pushed a sample here: https://github.com/dazinator/Sample.ReloadableMiddlewarePipeline

Run it under kestrel and view the console output. Save a change to the Pipeline section in appsettings.json and submit a new request (reload page):

There are things I am not 100% happy with yet, i'll probably push further improvements in future. Like - rather than couple the cache invalidation to OptionsMonitor, using a ChangToken (IChangeTokenSource pattern) would make this more general purpose with OptionsMonitor being only one possible source for triggering the rebuild.

[dazinator]

The way this is shown in my sample, this functionality could easily remain in an external nuget package and not in the framework. The area that gets more framework oriented is if it would be considered useful / valuable enough to allow this to be enabled on the root pipeline by some option or convention out of the box, or be available as a method on IApplicationBuider (similar to Use() or Map()) out of the box, and be adopted as an official pattern.

[Tratcher]

There doesn't need to be significant request perf overhead. You can build the first pipeline from the middleware constructor, rebuild in the background when changes are detected, and swap them out using a lock free exchange. That's about how we handle dynamic route reloads in YARP.

[dazinator]

I've pushed some changes to the sample to address a couple of things:

1. No longer coupled with OptionsMonitor - uses IChangeToken's instead, OptionsMonitor is supported via an extension method for ease of use.
2. As @Tratcher is aware, change tokens -> they can fire twice in quick succession from a single change so I have added "debouncing" based on @cocowalla's implementation that uses a specified delay https://github.com/dazinator/Sample.ReloadableMiddlewarePipeline/blob/master/src/Server/RequestDelegateFactory.cs#L25

[dazinator]

I've taken the opportunity to evolve my sample into a reusable library and published on nuget, project is here and it comes with a sample, but no tests at present.

I have used this to enable a scenario where I rebuild a tenants pipeline at runtime when:

• Options change - mainly around authentication
• Admin user "Restarts" the tenant through a button click in the UI.

Supplying a CompositeChangeToken wrapping both those sources worked well.

It seems there is a choice (and perhaps a clear winner?) over whether the pipeline is rebuilt "out of band" from the request, and then swapped in for the next request, or whether the pipeline is just "invalidated" in the background, and then re-built happens within a lock with the next subsequent request. The former option as @Tratcher explained is much better for not impacting requests. However in my library I started with the latter as for some reason I thought access to the request / httpcontext might be useful during the rebuild but.. this is something I may revisit in future, for my case, doesn't matter very much as rebuilds take less than a second and happen fairly infrequently.