Null Reference Exception cancelling Guided Failure Tasks

[borland]

Severity

Blocking a customer who has recently upgraded to 2023.2

Version

Bug was introduced in 2023.2.4119

Latest Version

I could reproduce the problem in the latest build

What happened?

As reported by a customer, attempting to cancel a Guided Failure task can result in an error being shown in the web portal - "object reference not set to an instance of an object" or "Value cannot be null. (Parameter 'source')"

Reproduction

This is not reproducible with a new install of Octopus on any recent version, however for customers that have older databases (the affected customer had data back to 2017 and earlier) there may be rows in the Interruptions database table that can't be loaded by Octopus Server 2023.2 - specifically where the JSON column does not contain any value for LinkedInterruptionIds. Such rows are what trigger the bug.

Error and Stacktrace

2023-06-13 08:06:19.3928   5248    316  INFO  "HTTP" "POST" to "[readacted URL]""/api/interruptions/Interruptions-31179/submit" "completed" with 400 in 00:00:00.1739324 (173ms) by "[redacted username]"
System.ArgumentNullException: Value cannot be null. (Parameter 'source')
   at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
   at System.Linq.Utilities.<>c__DisplayClass1_0`1.<CombinePredicates>b__0(TSource x)
   at System.Linq.Enumerable.WhereListIterator`1.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Octopus.Core.Model.ServerTasks.UpdateLinkedInterruptionsHandler.AnySiblingInterruptionsStillPending(Interruption interruption, List`1 linkedInterruptions, CancellationToken cancellationToken) in ./source/Octopus.Core/Model/ServerTasks/UpdateLinkedInterruptionsHandler.cs:line 59
   at Octopus.Core.Model.ServerTasks.UpdateLinkedInterruptionsHandler.UpdateLinkedInterruptions(Interruption interruption, CancellationToken cancellationToken) in ./source/Octopus.Core/Model/ServerTasks/UpdateLinkedInterruptionsHandler.cs:line 34
   at Octopus.Core.Model.ServerTasks.UpdateLinkedInterruptionsHandler.Handle(InterruptionSubmittedDomainEvent domainEvent, CancellationToken cancellationToken) in ./source/Octopus.Core/Model/ServerTasks/UpdateLinkedInterruptionsHandler.cs:line 23
   at Octopus.Core.Infrastructure.DomainEvents.AutofacDomainEventBroker.Publish[TDomainEvent](TDomainEvent domainEvent, CancellationToken cancellationToken) in ./source/Octopus.Core/Infrastructure/DomainEvents/AutofacDomainEventBroker.cs:line 25
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWork.PublishDomainEvents(CancellationToken cancellationToken) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWork.cs:line 149
   at Octopus.Core.Infrastructure.UnitsOfWork.UnitOfWork.Complete(CancellationToken cancellationToken) in ./source/Octopus.Core/Infrastructure/UnitsOfWork/UnitOfWork.cs:line 124
   at Octopus.Server.Web.Middleware.ActionFilters.UnitOfWorkActionFilter.OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next) in ./source/Octopus.Server/Web/Middleware/ActionFilters/UnitOfWorkActionFilter.cs:line 36
   at Octopus.Server.Web.Middleware.ActionFilters.UnitOfWorkActionFilter.OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next) in ./source/Octopus.Server/Web/Middleware/ActionFilters/UnitOfWorkActionFilter.cs:line 36
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Octopus.Server.Web.Middleware.BoundaryTrailerRewriteMiddleware.Invoke(HttpContext context, IAutomationContext automationContext) in ./source/Octopus.Server/Web/Middleware/BoundaryTrailerRewriteMiddleware.cs:line 44
   at Octopus.Server.Web.Infrastructure.Authentication.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult) in ./source/Octopus.Server/Web/Infrastructure/Authentication/AuthorizationMiddlewareResultHandler.cs:line 50
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, IUnitOfWork unitOfWork) in ./source/Octopus.Server/Web/UnitOfWorkMiddleware.cs:line 46
   at Octopus.Server.Web.UnitOfWorkMiddleware.InvokeAsync(HttpContext httpContext, IUnitOfWork unitOfWork) in ./source/Octopus.Server/Web/UnitOfWorkMiddleware.cs:line 46
   at Octopus.Server.Web.Middleware.OctopusClientOldVersionWarningMiddleware.InvokeAsync(HttpContext context, IAutomationContext automationContext) in ./source/Octopus.Server/Web/Middleware/OctopusClientOldVersionWarningMiddleware.cs:line 52
   at Octopus.Server.Web.Middleware.DynamicContentHeadersMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/DynamicContentHeadersMiddleware.cs:line 48
   at Octopus.Server.Web.Middleware.MaintenanceModeMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/MaintenanceModeMiddleware.cs:line 59
   at Octopus.Server.Web.Middleware.OctopusAuthenticationMiddleware.InvokeAsync(HttpContext context, IUserAuthenticator userAuthenticator, IAuthCookieService authCookieService, IWebAuthCache authCache, ILogger logger) in ./source/Octopus.Server/Web/Middleware/OctopusAuthenticationMiddleware.cs:line 58
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Octopus.Server.Web.Middleware.LegacyRequestLoggerMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/LegacyRequestLoggerMiddleware.cs:line 41
   at Octopus.Server.Web.Middleware.TelemetryMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/TelemetryMiddleware.cs:line 65
   at Octopus.Server.Web.Middleware.ErrorHandlingMiddleware.InvokeAsync(HttpContext context) in ./source/Octopus.Server/Web/Middleware/ErrorHandlingMiddleware.cs:line 50

More Information

R&D have traced this internally to a code change introduced in version 2023.2.4119 of Octopus Deploy Server

Workaround

No response

[borland]

FYI @SeanStanway-Octopus

[borland]

Release Note: Fix possible null reference exception when cancelling interrupted server tasks

[Octobob]

🎉 The fix for this issue has been released in:

Release stream	Release
2023.2	2023.2.12482
2023.3	2023.3.4421
2023.4+	all releases