Master key rotation erroring on ADO and Github issue tracker

[veochen-octopus]

Severity

Makes master key rotation unusable for certain customers

Version

any

Latest Version

I could reproduce the problem in the latest build

What happened?

The master key rotation process does not handle ADO and Github issuetracker configurations properly. If a customer has these enabled and configured, a rotation will corrupt the db.

Reproduction

-Configure ADO/Github issue tracker

• Run master key rotation
• Start the server and see errors

Error and Stacktrace

2023-03-23 12:25:07.6074   3804      5 FATAL  Unhandled AppDomain exception occurred: "Error reading row 1, column 3. Padding is invalid and cannot be removed..
Compiled reader expression:

(DbDataReader reader, DocumentReaderContext context) => 
{
    Type deserializeAsType = Octopus.Server.Extensibility.IssueTracker.AzureDevOps.Configuration.AzureDevOpsConfiguration
    context.Column = 0
    String temp0 = IIF(reader.IsDBNull(0), null, reader.GetString(0))
    context.Column = 1
    String temp1 = IIF(reader.IsDBNull(1), null, reader.GetString(1))
    context.Column = 2
    String temp2 = IIF(reader.IsDBNull(2), null, reader.GetString(2))
    context.Column = 3
    AzureDevOpsConfiguration deserializedFromJson = context.DeserializeText(reader, 3, deserializeAsType)
    AzureDevOpsConfiguration result = deserializedFromJson
    if (result != null)
    {
        result.Name = temp1
        result.ExtensionAuthor = temp2
    }

    result
}
"
Nevermore.ReaderException: Error reading row 1, column 3. Padding is invalid and cannot be removed..
Compiled reader expression:

(DbDataReader reader, DocumentReaderContext context) => 
{
    Type deserializeAsType = Octopus.Server.Extensibility.IssueTracker.AzureDevOps.Configuration.AzureDevOpsConfiguration
    context.Column = 0
    String temp0 = IIF(reader.IsDBNull(0), null, reader.GetString(0))
    context.Column = 1
    String temp1 = IIF(reader.IsDBNull(1), null, reader.GetString(1))
    context.Column = 2
    String temp2 = IIF(reader.IsDBNull(2), null, reader.GetString(2))
    context.Column = 3
    AzureDevOpsConfiguration deserializedFromJson = context.DeserializeText(reader, 3, deserializeAsType)
    AzureDevOpsConfiguration result = deserializedFromJson
    if (result != null)
    {
        result.Name = temp1
        result.ExtensionAuthor = temp2
    }

    result
}

 ---> System.Security.Cryptography.CryptographicException: Padding is invalid and cannot be removed.
   at Internal.Cryptography.UniversalCryptoDecryptor.GetPaddingLength(ReadOnlySpan`1 block)
   at Internal.Cryptography.UniversalCryptoDecryptor.UncheckedTransformFinalBlock(ReadOnlySpan`1 inputBuffer, Span`1 outputBuffer)
   at Internal.Cryptography.UniversalCryptoDecryptor.UncheckedTransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
   at Internal.Cryptography.UniversalCryptoTransform.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
   at System.Security.Cryptography.CryptoStream.FlushFinalBlockAsync(Boolean useAsync, CancellationToken cancellationToken)
   at System.Security.Cryptography.CryptoStream.FlushFinalBlock()
   at System.Security.Cryptography.CryptoStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at Octopus.Core.Security.MasterKey.MasterKeyEncryption.ToPlaintext(Byte[] masterKey, EncryptedBytes encrypted) in ./source/Octopus.Core/Security/MasterKey/MasterKeyEncryption.cs:line 74
   at Octopus.Core.Security.MasterKey.StoredMasterKeyEncryption.ToPlaintext(EncryptedBytes encrypted) in ./source/Octopus.Core/Security/MasterKey/StoredMasterKeyEncryption.cs:line 75
   at Octopus.Core.Security.MasterKey.MasterKeyEncryptionExtensionMethods.ToPlaintextString(IMasterKeyEncryption encryption, EncryptedBytes encrypted) in ./source/Octopus.Core/Security/MasterKey/MasterKeyEncryptionExtensionMethods.cs:line 21
   at Octopus.Core.RelationalStorage.TypeHandlers.SensitiveStringConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in ./source/Octopus.Core/RelationalStorage/TypeHandlers/SensitiveStringConverter.cs:line 56
   at Octopus.Core.RelationalStorage.TypeHandlers.SensitiveStringTypeHandler.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in ./source/Octopus.Core/RelationalStorage/TypeHandlers/SensitiveStringTypeHandler.cs:line 53
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Nevermore.Advanced.Serialization.NewtonsoftDocumentSerializer.DeserializeSmallText(String text, Type type)
   at Nevermore.Advanced.ReaderStrategies.Documents.DocumentReaderContext.DeserializeText[TDocument](DbDataReader reader, Int32 index, Type concreteType)
   at lambda_method2364(Closure , DbDataReader , DocumentReaderContext )
   at Nevermore.Advanced.ReaderStrategies.Documents.DocumentReaderStrategy.<>c__DisplayClass4_1`1.<CreateReader>b__1(DbDataReader dbDataReader)
   --- End of inner exception stack trace ---
   at Nevermore.Advanced.ReaderStrategies.Documents.DocumentReaderStrategy.<>c__DisplayClass4_1`1.<CreateReader>b__1(DbDataReader dbDataReader)
   at Nevermore.Advanced.ReadTransaction.ProcessReader[TRecord](DbDataReader reader, PreparedCommand command)+MoveNext()
   at Nevermore.Advanced.ReadTransaction.<>c__DisplayClass100_0`1.<<Stream>g__Execute|0>d.MoveNext()
   at Nevermore.Advanced.ThreadSafeEnumerable`1.ThreadSafeEnumerator.MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
   at Nevermore.Advanced.ReadTransaction.Load[TDocument,TKey](TKey id)
   at Octopus.Core.RelationalStorage.RawRelationalTransaction.Load[TDocument](String id) in ./source/Octopus.Core/RelationalStorage/RawRelationalTransaction.cs:line 26
   at Octopus.Core.Repositories.RawConfigurationStore.Get[TDocument](String id) in ./source/Octopus.Core/Repositories/RawConfigurationStore.cs:line 32
   at Octopus.Core.Repositories.RawConfigurationStoreCacheDecorator.Get[TDocument](String id) in ./source/Octopus.Core/Repositories/RawConfigurationStoreCacheDecorator.cs:line 27
   at Octopus.Server.Extensibility.IssueTracker.AzureDevOps.Configuration.DatabaseInitializer.Execute() in ./source/Octopus.Server.Extensibility.IssueTracker.AzureDevOps/Configuration/DatabaseInitializer.cs:line 21
   at Octopus.Core.Initialization.BuiltIn.ConfigurationInitializer.Initialize(IRawRelationalStore store) in ./source/Octopus.Core/Initialization/BuiltIn/ConfigurationInitializer.cs:line 24
   at Octopus.Core.Initialization.StoreInitializer.Initialize() in ./source/Octopus.Core/Initialization/StoreInitializer.cs:line 19
   at Octopus.Server.OctopusServerEngine.Start() in ./source/Octopus.Server/OctopusServerEngine.cs:line 70
   at Octopus.Server.Commands.RunCommand.Start() in ./source/Octopus.Server/Commands/RunCommand.cs:line 95
   at Octopus.Shared.Startup.AbstractCommand.Start(String[] commandLineArguments, ICommandRuntime commandRuntime, OptionSet commonOptions) in ./source/Octopus.Shared/Startup/AbstractCommand.cs:line 101
   at Octopus.Shared.Startup.OctopusProgram.Start(ICommandRuntime commandRuntime) in ./source/Octopus.Shared/Startup/OctopusProgram.cs:line 504
   at Octopus.Shared.Startup.WindowsServiceHost.<>c__DisplayClass2_0.<Run>b__0() in ./source/Octopus.Shared/Startup/WindowsServiceHost.cs:line 26
   at Octopus.Shared.Startup.WindowsServiceAdapter.RunService() in ./source/Octopus.Shared/Startup/WindowsServiceAdapter.cs:line 48
   at System.Threading.Thread.StartCallback()

More Information

No response

Workaround

No response

[octoreleasebot]

Release Note: Include all extension configurations in master key rotation

[Octobob]

🎉 The fix for this issue has been released in:

Release stream	Release
2022.4	2022.4.8617
2023.1	2023.1.9879
2023.2	2023.2.9010
2023.3+	all releases