Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some upgrades may fail if malformed artifacts exist #5497

Closed
jburger opened this issue Apr 18, 2019 · 4 comments

Comments

@jburger
Copy link

commented Apr 18, 2019

The bug

Some upgrades may fail if malformed artifacts exist, due to an upgrade script that expects to process such rows, that fails in an unexpected way.

Artifacts are now expected to have the same permission scopes as the ServerTasks that they link to.

The upgrade script that fails here is attempting to collect data regarding violations of this rule, to aid us in reaching a better design, as instances where the scope is misaligned, may indicate an unhandled data integrity issue. However, this script is failing due to a poorly crafted query.

What I expected to happen

  1. Data shouldn't be in this state, however this script was not designed to fix that condition, only to detect and inform the customer to report some infromation to support. However, the mechanism to harvest that information was flawed. At the very least, that should detection mechanism needs to work :)

  2. Ideally, in addition to fixing 1. we'd understand how the data got to be in this state, and make some changes to handle those instances.

Steps to reproduce

  1. On a version prior to 2019.4.4 create artifacts with links to server tasks with a different permission scope to the artifact. Rows in this state would match the following query
SELECT 
                a.[Id]
                ,a.[ServerTaskId]
                ,s.[ProjectId] as ServerTaskProjectId
                ,s.[EnvironmentId] as ServerTaskEnvironmentId
                ,s.[TenantId] as ServerTaskTenantId
                FROM [dbo].[Artifact] a 
                INNER JOIN [dbo].[ServerTask] s on a.[ServerTaskId] = s.Id
                WHERE a.[ProjectId] != s.[ProjectId] or a.[EnvironmentId] != s.[EnvironmentId] or a.[TenantId] != s.[TenantId]
  1. Commence an upgrade to a version post 2019.4.4 and observe the upgrade error

Log excerpt

System.IndexOutOfRangeException: ServerTenantId
Error: at System.Data.ProviderBase.FieldNameLookup.GetOrdinal(String fieldName)
Error: at System.Data.SqlClient.SqlDataReader.GetOrdinal(String name)
Error: at System.Data.SqlClient.SqlDataReader.get_Item(String name)
Error: at Octopus.Core.UpgradeScripts.Script0187ArtifactCleanup.<>c.b__0_1(IDataRecord record)
Error: at Octopus.Core.Initialization.DbCommandExecutor.d__51.MoveNext() Error: at System.Collections.Generic.List1…ctor(IEnumerable1 collection) Error: at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source)
Error: at Octopus.Core.UpgradeScripts.Script0187ArtifactCleanup.Execute(IDbCommandExecutor commandExecutor, IUpgradeLog log)
Error: at Octopus.Core.Initialization.DbCommandUpgradeScriptResolver.Execute(Func1 commandFactory, IUpgradeLog log) Error: at Octopus.Core.Initialization.DatabaseUpgrader.OctopusUpgradeScriptExecutor.<>c__DisplayClass6_0.<ExecuteDbCommandUpgradeScript>b__0(Func1 commandFactory)
Error: at DbUp.Engine.Transactions.SingleTrasactionStrategy.Execute(Action1 action) Error: at Octopus.Core.Initialization.DatabaseUpgrader.OctopusUpgradeScriptExecutor.ExecuteDbCommandUpgradeScript(IConnectionManager connectionManager, IJournal journal, IUpgradeLog log, IDbCommandUpgradeScript upgradeScript) Error: at Octopus.Core.Initialization.DatabaseUpgrader.OctopusUpgradeScriptExecutor.Execute(SqlScript script, IDictionary2 variables)
Error: at DbUp.Engine.UpgradeEngine.PerformUpgrade()

Affected versions

Octopus Server: v2019.4.4+

Workarounds

If you experience this issue, we suggest not upgrading at this time, and waiting for us to deliver a release to close this Issue.

Links

public: customer report
internal: original specification
internal: discussion

@JoelVanBeek

This comment has been minimized.

Copy link

commented Jul 25, 2019

What is the current status of this? We can't update Octopus anymore..

@TomPeters

This comment has been minimized.

Copy link

commented Jul 30, 2019

What is the current status of this? We can't update Octopus anymore..

We are looking into this right now, and should have a fix out soon for both the latest fast lane release, and the 2019.6 LTS.

@JoelVanBeek

This comment has been minimized.

Copy link

commented Jul 30, 2019

What is the current status of this? We can't update Octopus anymore..

We are looking into this right now, and should have a fix out soon for both the latest fast lane release, and the 2019.6 LTS.

That sounds great. I'll wait for the fix.

@TomPeters

This comment has been minimized.

Copy link

commented Aug 20, 2019

Release Note: Fixed a bug that prevent upgrades when malformed artifact documents exist

@TomPeters TomPeters added this to the 2019.7.12 milestone Aug 20, 2019
@droyad droyad modified the milestones: 2019.7.12, 2019.7.13 Aug 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.