Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[elasticache] [bug:28011] Fixes downgrade paths detection #28877

Closed
wants to merge 3 commits into from

Conversation

joseluis-fw
Copy link

@joseluis-fw joseluis-fw commented Jan 13, 2023

Description

Fixes recreation of aws_elasticache_replication_group when changing engine_version to upgraded versions.

There are at least some situations fixed by this PR:

  • Migration of 2.8.x to any version
  • Migration of 6.x ( 6.0.x in the real version ) to 6.0 or 6.2.
  • Whatever version that is X.[6-9] to any major version that is less than the previous minor version

Relations

Closes #28011

Output from Unit Testing

$ go test -v ./internal/service/elasticache/... -timeout=1m -run TestCustomizeDiffEngineVersionIsDowngrade
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/no_change
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/no_change
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.x_to_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.x_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.x_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/no_change
--- PASS: TestCustomizeDiffEngineVersionIsDowngrade (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0 (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.x_to_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/no_change (0.00s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/elasticache        4.340s

@github-actions
Copy link

Hey @joseluis-fw 👋 Thank you very much for your contribution! At times, our maintainers need to make direct edits to pull requests in order to help get it ready to be merged. Your current settings do not allow maintainers to make such edits. To help facilitate this, update your pull request to allow such edits as described in GitHub's Allowing changes to a pull request branch created from a fork documentation. (If you're using a fork owned by an organization, your organization may not allow you to change this setting. If that is the case, let us know.)

@github-actions
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added service/elasticache Issues and PRs that pertain to the elasticache service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. needs-triage Waiting for first response or review from a maintainer. size/S Managed by automation to categorize the size of a PR. labels Jan 13, 2023
@joseluis-fw joseluis-fw changed the title [bug:28011] Fixes downgrade paths detection [elasticache] [bug:28011] Fixes downgrade paths detection Jan 13, 2023
@justinretzolk justinretzolk added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Jan 13, 2023
@josacar
Copy link

josacar commented Jan 14, 2023

There's another path to go around this bug for 6.x engine_version, that is use engine_version_actual to compare it with engine_version, this is the PoC diff to solve the bug that way:

diff --git a/internal/service/elasticache/engine_version.go b/internal/service/elasticache/engine_version.go
index 3ece38f415..78a12bfce4 100644
--- a/internal/service/elasticache/engine_version.go
+++ b/internal/service/elasticache/engine_version.go
@@ -96,6 +96,12 @@ type getChangeDiffer interface {
 
 func engineVersionIsDowngrade(diff getChangeDiffer) (bool, error) {
 	o, n := diff.GetChange("engine_version")
+	oActual, _ := diff.GetChange("engine_version_actual")
+
+	if oActual != nil {
+		o = oActual
+	}
+
 	oVersion, err := normalizeEngineVersion(o.(string))
 	if err != nil {
 		return false, fmt.Errorf("error parsing old engine_version: %w", err)
@@ -105,15 +111,6 @@ func engineVersionIsDowngrade(diff getChangeDiffer) (bool, error) {
 		return false, fmt.Errorf("error parsing new engine_version: %w", err)
 	}
 
-	sixMaxVersion := fmt.Sprintf("%s.%d", "6", math.MaxInt)
-	sixMaxVersionComp, _ := gversion.NewVersion(sixMaxVersion)
-	sixMaxVersionCompString := sixMaxVersionComp.String()
-
-	// When both major are 6 and one of them is 6.x
-	if (sixVersionConstraint.Check(nVersion) && sixVersionConstraint.Check(oVersion)) && (oVersion.String() == sixMaxVersionCompString || nVersion.String() == sixMaxVersionCompString) {
-		return false, nil
-	}
-
 	return nVersion.LessThan(oVersion), nil
 }
 
diff --git a/internal/service/elasticache/engine_version_test.go b/internal/service/elasticache/engine_version_test.go
index d6f09ebd58..80dc25f6bc 100644
--- a/internal/service/elasticache/engine_version_test.go
+++ b/internal/service/elasticache/engine_version_test.go
@@ -296,7 +296,7 @@ func TestCustomizeDiffEngineVersionIsDowngrade(t *testing.T) {
 		},
 
 		"upgrade minor 6.x": {
-			old:      "6.x",
+			old:      "6.0",
 			new:      "6.2",
 			expected: false,
 		},
@@ -385,7 +385,6 @@ func (d *mockForceNewDiffer) HasChange(key string) bool {
 func (d *mockForceNewDiffer) GetChange(key string) (interface{}, interface{}) {
 	return d.old, d.new
 }
-
 func (d *mockForceNewDiffer) ForceNew(key string) error {
 	d.forceNew = true
 
@@ -431,17 +430,17 @@ func TestCustomizeDiffEngineVersionForceNewOnDowngrade(t *testing.T) {
 			expectForceNew: false,
 		},
 
-		// "upgrade major 6.x": {
-		// 	old:            "5.0.6",
-		// 	new:            "6.x",
-		// 	expectForceNew: false,
-		// },
+		"upgrade major 6.x": {
+			old:            "5.0.6",
+			new:            "6.x",
+			expectForceNew: false,
+		},
 
-		// "upgrade major 6.digit": {
-		// 	old:            "5.0.6",
-		// 	new:            "6.0",
-		// 	expectForceNew: false,
-		// },
+		"upgrade major 6.digit": {
+			old:            "5.0.6",
+			new:            "6.0",
+			expectForceNew: false,
+		},
 
 		"downgrade minor versions": {
 			old:            "1.3.5",
@@ -491,13 +490,18 @@ func TestCustomizeDiffEngineVersionForceNewOnDowngrade(t *testing.T) {
 		t.Run(name, func(t *testing.T) {
 			t.Parallel()
 
-			diff := &mockForceNewDiffer{}
-			if !testcase.isNew {
-				diff.id = "some id"
-				diff.old = testcase.old
-				diff.new = testcase.new
+			diff := &mockChangesDiffer{
+				values: map[string]mockDiff{
+					"engine_version_actual": {
+						old:       testcase.old,
+						new:       testcase.new,
+					},
+					"engine_version": {
+						old:       testcase.old,
+						new:       testcase.new,
+					},
+				},
 			}
-			diff.hasChange = testcase.hasChange
 
 			err := engineVersionForceNewOnDowngrade(diff)
 
@@ -625,6 +629,7 @@ func (d mockDiff) GetChange() (interface{}, interface{}) {
 type mockChangesDiffer struct {
 	id     string
 	values map[string]mockDiff
+	forceNew  bool
 }
 
 func (d *mockChangesDiffer) Id() string {
@@ -639,6 +644,12 @@ func (d *mockChangesDiffer) GetChange(key string) (interface{}, interface{}) {
 	return d.values[key].GetChange()
 }
 
+func (d *mockChangesDiffer) ForceNew(key string) error {
+	d.forceNew = true
+
+	return nil
+}
+
 func TestParamGroupNameRequiresMajorVersionUpgrade(t *testing.T) {
 	t.Parallel()
 

@josacar
Copy link

josacar commented Jan 19, 2023

Right now the downgrade detection is broken, can we also add a parameter to enforce this as this shouldn't be the most common case?

Should we have a flag as deletion_protection or taint at least to avoid the blast radius in the custom code that is designed to help and not the opposite?

@github-actions github-actions bot added size/XS Managed by automation to categorize the size of a PR. and removed size/S Managed by automation to categorize the size of a PR. labels Jul 14, 2023
@josacar
Copy link

josacar commented Jul 15, 2023

Rebased:

terraform-provider-aws  main took 1m23s ✦ ❯ go test -v ./internal/service/elasticache/... -timeout=1m -run TestCustomizeDiffEngineVersionIsDowngrade
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/no_change
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/no_change
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.digit
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.digit
=== RUN   TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== PAUSE TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/no_change
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions
=== CONT  TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.digit
--- PASS: TestCustomizeDiffEngineVersionIsDowngrade (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_3.0 (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/no_change (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/switch_major_6.digit_to_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.digit (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_minor_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_6.digit (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/upgrade_major_6.x (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_minor_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_7.digit_to_6.digit (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_major_versions (0.00s)
    --- PASS: TestCustomizeDiffEngineVersionIsDowngrade/downgrade_from_major_6.x (0.00s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/elasticache        7.996s

@gdavison gdavison self-assigned this Oct 13, 2023
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Oct 13, 2023
@gdavison
Copy link
Contributor

Thanks for the PR, @joseluis-fw. I've created PR #33954 using an approach similar to #28877 (comment). We'll be merging that PR instead, since it includes other fixes.

@gdavison gdavison closed this Oct 16, 2023
@josacar
Copy link

josacar commented Oct 17, 2023

@gdavison have we fixed the comparation of the minor vs the major that made 2.8.x upgrades to any version < 7 recreate the cluster ( there's a missing parenthesis globing the | in the regexp )

Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. service/elasticache Issues and PRs that pertain to the elasticache service. size/XS Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: aws_elasticache_replication_group recreates resource on engine_version upgrade
4 participants