Unhandled DbUpdateConcurrencyException leading to HTTP 500 errors

[rosslovas]

Version

2024.2.2605

Latest Version

The problem does not exist in the latest build

What happened?

Intermittent HTTP 500 errors may appear in the UI on various pages. The logs indicate unhandled DbUpdateConcurrencyExceptions as the cause.

Reproduction

Intermittent, no reliable way to reproduce known

Error and Stacktrace

System.ObjectDisposedException: The response has been aborted due to an unhandled application exception.
 ---> Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See https://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
   at Octopus.Core.Persistence.Database.OctopusDbContext.SaveChangesAsync(CancellationToken cancellationToken) in ./source/Octopus.Core/Persistence/Database/OctopusDbContext.cs:line 155
   at Octopus.Core.RelationalStorage.RawRelationalTransaction.Commit(CancellationToken cancellationToken) in ./source/Octopus.Core/RelationalStorage/RawRelationalTransaction.cs:line 245
   at Octopus.Core.Repositories.UserSessionStore.TryRefreshUserSession(UserSessionId userSessionId, DateTimeOffset validToDate, CancellationToken cancellationToken) in ./source/Octopus.Core/Repositories/UserSessionStore.cs:line 97
   at Octopus.Server.Web.Infrastructure.Authentication.UserSessionService.RevokeOrRefreshUserSessionAsNecessary(HttpContext context, CancellationToken cancellationToken) in ./source/Octopus.Server/Web/Infrastructure/Authentication/UserSessionService.cs:line 326
   at Octopus.Server.Web.Middleware.OctopusAuthenticationMiddleware.<>c__DisplayClass2_0.<<InvokeAsync>b__0>d.MoveNext() in ./source/Octopus.Server/Web/Middleware/OctopusAuthenticationMiddleware.cs:line 33
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<FireOnStarting>g__ProcessEvents|241_0(HttpProtocol protocol, Stack`1 events)
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.FirstWriteAsyncInternal(ReadOnlyMemory`1 data, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.FirstWriteAsync(ReadOnlyMemory`1 data, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.WritePipeAsync(ReadOnlyMemory`1 data, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at Octopus.Server.Web.Infrastructure.HttpResponseHelper.WriteErrorToResponse(HttpResponse response, Int32 statusCode, Error error) in ./source/Octopus.Server/Web/Infrastructure/HttpResponseHelper.cs:line 61
   at Octopus.Server.Web.Middleware.ErrorResponseWritingMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/ErrorResponseWritingMiddleware.cs:line 128
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Octopus.Server.Web.Middleware.RedirectToHttpsMiddleware.InvokeAsync(HttpContext context, IWebConfigurationMemento webConfiguration) in ./source/Octopus.Server/Web/Middleware/RedirectToHttpsMiddleware.cs:line 28
   at Octopus.Server.Web.Middleware.CorsPreflightRequestMiddleware.InvokeAsync(HttpContext context, IWebConfigurationMemento webConfiguration) in ./source/Octopus.Server/Web/Middleware/CorsPreflightRequestMiddleware.cs:line 28
   at Octopus.Server.Web.Middleware.CommonHeadersMiddleware.InvokeAsync(HttpContext context, IWebConfigurationMemento webConfiguration) in ./source/Octopus.Server/Web/Middleware/CommonHeadersMiddleware.cs:line 43
   at Octopus.Server.Web.Middleware.GenerateNonceMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/GenerateNonceMiddleware.cs:line 22
   at Octopus.Server.Web.Middleware.WebServerStatusMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/WebServerStatusMiddleware.cs:line 34
   at Octopus.Server.Web.Middleware.AllowedHttpMethodsMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/AllowedHttpMethodsMiddleware.cs:line 28
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at Octopus.Server.Web.Middleware.RequestLoggingMiddleware.InvokeAsync(HttpContext httpContext, ILogger logger, IEnumerable`1 featureToggles) in ./source/Octopus.Server/Web/Middleware/RequestLoggingMiddleware.cs:line 56
   at Octopus.Server.Web.Middleware.LogContextEnrichingMiddleware.InvokeAsync(HttpContext httpContext, ICorrelationIdProvider correlationIdProvider) in ./source/Octopus.Server/Web/Middleware/LogContextEnrichingMiddleware.cs:line 32
   at Octopus.Server.Web.Middleware.CorrelationIdMiddleware.InvokeAsync(HttpContext httpContext, ICorrelationIdProvider correlationIdProvider) in ./source/Octopus.Server/Web/Middleware/CorrelationIdMiddleware.cs:line 30
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

[octoreleasebot]

Release Note: Fixed a regression causing HTTP requests to intermittently respond with HTTP 500 errors

[Octobob]

🎉 The fix for this issue has been released in:

Release stream	Release
2024.2	2024.2.3210
2024.3+	all releases