Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exotic MVC routing throws System.IndexOutOfRangeException at Microsoft.AspNetCore.Routing.RouteValueDictionary #8499

Open
Agentis opened this Issue Mar 14, 2019 · 5 comments

Comments

Projects
None yet
4 participants
@Agentis
Copy link

Agentis commented Mar 14, 2019

We have a .NET Core 2.1 application in production on Azure. It has been, infrequently, coming down with an unrecoverable 500 error which necessitates a site restart. We have noticed the following exception/stack trace appearing in our Azure application insights log hundreds of times at these times.

We cannot easily reproduce the issue unfortunately; it happens unpredictably but with increasingly likelihood as time goes by.

Exception:

System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Microsoft.AspNetCore.Routing.RouteValueDictionary.ListStorage.Clear()
   at Microsoft.AspNetCore.Routing.RouteValueDictionary.Clear()
   at Microsoft.AspNetCore.Routing.RouteData.RouteDataSnapshot.Restore()
   at Microsoft.AspNetCore.Routing.RouteCollection.<RouteAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.<Invoke>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.<Invoke>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware.<Invoke>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Frame`1.<ProcessRequestsAsync>d__2.MoveNext()

Potentially relevant code; our route configuration:

app.UseMvc(routes =>
{
    var section = Configuration.GetSection("AppSettings:HostNames");
    var hostnames = section.Get<string[]>();

    // Prepare special routes
    routes.MapRoute("takeSurveyApi", "TakeSurvey/{action}/{id?}", new { controller = "TakeSurvey", action = "Index" });
    routes.MapRoute("takeSurvey", "TakeSurvey/{id?}", new { controller = "TakeSurvey", action = "Index" });
    routes.MapRoute("stripeWebhook", "Billing/WebHook", new { controller = "Billing", action = "WebHook" });

    routes.MapSubdomainRoute(
        hostnames,
        "MHP",
        "{tenant}",
        "MHP/{action}",
        new { controller = "MHP", action = "GetTenantDetails" });

    routes.MapSubdomainRoute(
        hostnames,
        "Localisation",
        "{tenant}",
        "Localisation/{action}",
        new { controller = "Localisation", action = "Get" });

    routes.MapSubdomainRoute(
        hostnames,
        "TenantInSubdomain",
        "{tenant}",
        "{*url}",
        new { controller = "Home", action = "Index" });

    routes.MapRoute(
        hostnames,
        "DefaultRoute",
        "{controller}/{action}/{id?}",
        new { controller = "Home", action = "Index" });               

    routes.MapRoute(
        hostnames, "reactRouter", "{*url}", new {controller = "Home", action = "Index"});
});

Additional context

dotnet --info` = 
.NET Core SDK (reflecting any global.json):
 Version:   2.1.500
 Commit:    b68b931422

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.500\

Host (useful for support):
  Version: 2.1.6
  Commit:  3f4f8eebd8

.NET Core SDKs installed:
  1.0.0-preview2-003131 [C:\Program Files\dotnet\sdk]
  1.1.0 [C:\Program Files\dotnet\sdk]
  2.1.101 [C:\Program Files\dotnet\sdk]
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.403 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
@mkArtakMSFT

This comment has been minimized.

Copy link
Member

mkArtakMSFT commented Mar 14, 2019

Thanks for contacting us, @Agentis.
@JamesNK can you please look into this? Thanks!

@rynowak

This comment has been minimized.

Copy link
Member

rynowak commented Mar 15, 2019

Are you doing anything that involves accessing the HTTP request or route data on another thread?

Are you able to share the implementation of MapSubdomainRoute?

@Agentis

This comment has been minimized.

Copy link
Author

Agentis commented Mar 16, 2019

Isn’t MapSubdomainRoute straight from MVC? I don’t currently have access to the solution to double check.

But as for threading, there is actually processing occurring on other threads, powered by Hangfire. Most of it revolves around doing some activity tracking; almost every controller action queues a Hangfire job which record the interaction.

Is there a known issue with this combination of effects?

@rynowak

This comment has been minimized.

Copy link
Member

rynowak commented Mar 18, 2019

Isn’t MapSubdomainRoute straight from MVC?

It might be part of some sample code or test somewhere, but it isn't a feature we provide.

But as for threading, there is actually processing occurring on other threads, powered by Hangfire. Most of it revolves around doing some activity tracking; almost every controller action queues a Hangfire job which record the interaction.

Is there a known issue with this combination of effects?

The HttpContext, RouteValueDictionary and most other things that are part of the request are not thread-safe. They are intended to be used inside the context of the currently executing request, and not in other threads.

@Agentis

This comment has been minimized.

Copy link
Author

Agentis commented Mar 24, 2019

Sorry for the delay in reply, other projects took over...

The developer who integrated this subdomain routing has since left the organisation; but it looks like this is the library he used in this instance:

https://github.com/mariuszkerl/AspNetCoreSubdomain

We're using the current version, 1.1.0.

As this library doesn't appear in the stack trace, it did not occur to me to examine it.

Are there any clues in the error as to where in this library a problem may have occurred (if the problem is not instead due to the HttpContext being accessed off-thread)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.