External feed triggers fail with an InvalidOperationException

[YuKitsune]

Severity

One customer affected, workaround exists.

Version

Bug was introduced in 2024.2.6377

Latest Version

None

What happened?

When the External Feed Trigger task runs, it may fail with an InvalidOperationException with the following message:

There is already an open DataReader associated with this Connection which must be closed first.

Reproduction

N/A. Race condition.

Error and Stacktrace

System.InvalidOperationException: There is already an open DataReader associated with this Connection which must be closed first.
   at Microsoft.Data.SqlClient.SqlCommand.ValidateCommand(Boolean isAsync, String method)
   at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
   at Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.InitializeReader(Enumerator enumerator)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
   at Octopus.Core.Serialization.Ocl.TinyTypeConversion.TinyTypeConversionStrategyProvider.GetRequiredDocuments[TDocument](IQueryable`1 queryable, Type tinyType, TinyTypeConversionIdentifiers identifiers)
   at Octopus.Core.Serialization.Ocl.TinyTypeConversion.TinyTypeConversionStrategyProvider.GetCollectionResultsForTinyType(IOctopusQueryExecutor queryExecutor, Type tinyType, TinyTypeConversionIdentifiers identifiers, ProjectId projectId)
   at Octopus.Core.Serialization.Ocl.TinyTypeConversion.TinyTypeConversionStrategyProvider.LoadLookups(ProjectId projectId, TinyTypeConversionIdentifiers identifiers, Func`2 createLookup) in ./source/Octopus.Core/Serialization/Ocl/TinyTypeConversion/TinyTypeConversionStrategyProvider.cs:line 317
   at Octopus.Core.Serialization.Ocl.TinyTypeConversion.TinyTypeConverter.TryConvert(ProjectId projectId, Object obj, TinyTypeConversionStrategy conversionStrategyType) in ./source/Octopus.Core/Serialization/Ocl/TinyTypeConversion/TinyTypeConverter.cs:line 38
   at Octopus.Core.Serialization.Ocl.Transformation.Transforms.Decorators.IdToSlugOclTransformDecorator`1.TransformFromOcl(String ocl, ProjectGitContext context, CancellationToken cancellationToken) in ./source/Octopus.Core/Serialization/Ocl/Transformation/Transforms/Decorators/IdToSlugOclTransformDecorator.cs:line 31
   at Octopus.Core.Serialization.Ocl.OclHandler.FromOcl[T](String ocl, ProjectGitContext context, CancellationToken cancellationToken)
   at Octopus.Core.Serialization.Ocl.Documents.GitOclDocumentService.GetDocument[TDocument,TKey](TKey id, GitTransactionContext context, IReadWriteVirtualFileSystem fileSystem, CancellationToken cancellationToken)
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.<>c__DisplayClass15_0`2.<<GetDocument>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.MigrateBeforeExecute[T](Func`1 innerOperation, CancellationToken cancellationToken)
   at Octopus.Core.Git.Transactions.GitDocumentTransaction.GetDocument[TDocument,TKey](TKey id, CancellationToken cancellationToken)
   at Octopus.Core.Persistence.Git.GitDocumentStore`2.Get(TKey id, CancellationToken cancellationToken)
   at Octopus.Core.Persistence.ChangeTracking.NevermoreAndGit.NonTrackingDocumentStoreDecorator`2.Get(TKey id, CancellationToken cancellationToken)
   at Octopus.Core.Persistence.DocumentStore`2.Get(TKey id, CancellationToken cancellationToken)
   at Octopus.Core.Persistence.ReadOnlyDocumentStore`2.Get(TKey id, CancellationToken cancellationToken)

More Information

No response

Workaround

If the customer notices a deployment hasn't been run due to the broken external trigger processor, they could manually queue deployments.

[octoreleasebot]

Release Note: Fixed an issue where external feed triggers could fail with an InvalidOperationException.

[Octobob]

🎉 The fix for this issue has been released in:

Release stream	Release
2024.2	2024.2.7483
2024.3+	all releases