Skip to content

ROX-13692: fix final snapshot creation#1012

Merged
johannes94 merged 7 commits intomainfrom
jmalsam/ROX-13692-fix-final-db-snapshot
May 9, 2023
Merged

ROX-13692: fix final snapshot creation#1012
johannes94 merged 7 commits intomainfrom
jmalsam/ROX-13692-fix-final-db-snapshot

Conversation

@johannes94
Copy link
Copy Markdown
Contributor

@johannes94 johannes94 commented May 4, 2023

The first PR  #994 was causing errors in the stage environment because the reconciler would proceed to run delete DB calls to AWS even though the deletion was already in progress. The ad-hoc fix for staging was to revert the PR. This PR is the new attempt to implement the feature without breaking reconciliation logic.

Description

Checklist (Definition of Done)

  • Unit and integration tests added
  • Added test description under Test manual
  • Documentation added if necessary (i.e. changes to dev setup, test execution, ...)
  • CI and all relevant tests are passing
  • Add the ticket number to the PR title if available, i.e. ROX-12345: ...
  • Discussed security and business related topics privately. Will move any security and business related topics that arise to private communication channel.
  • Add secret to app-interface Vault or Secrets Manager if necessary

Test manual

See the new file howto-e2e-test-rds.md.

Also running local RDS integration tests.

# To run tests locally run:
make db/teardown db/setup db/migrate
make ocm/setup OCM_OFFLINE_TOKEN=<ocm-offline-token> OCM_ENV=development
make verify lint binary test test/integration

@johannes94 johannes94 temporarily deployed to development May 4, 2023 12:18 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 12:18 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 12:18 — with GitHub Actions Inactive
@openshift-ci openshift-ci bot added the approved label May 4, 2023
require.NoError(t, err)
assert.True(t, clusterDeleted)

// Always attemt to delete the final snapshot if it exists
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Always attemt to delete the final snapshot if it exists
// Always attempt to delete the final snapshot if it exists

Copy link
Copy Markdown
Contributor Author

@johannes94 johannes94 May 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed it.

@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:10 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:10 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:10 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:10 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:11 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 4, 2023 14:11 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 11:26 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 11:26 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 11:26 — with GitHub Actions Inactive
@johannes94 johannes94 requested a review from vladbologa May 8, 2023 11:27
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:28 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:28 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:28 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:34 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:34 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 8, 2023 12:34 — with GitHub Actions Inactive
Copy link
Copy Markdown
Contributor

@vladbologa vladbologa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing the E2E howto!

- 'docs/**'
- 'pkg/api/openapi/docs/**'
- 'pkg/api/openapi/.openapi-generator-ignore'
- 'dp-terraform/**'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be useful. Some parameters used in terraform-cluster.sh (such as MANAGED_DB_SUBNET_GROUP) are also used in rds_test.go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK those parameters are not directly defined in terraform-cluster.sh but loaded from AWS parameterstore/secretsmanager so that shouldn't be a problem. I'm concerned about having to run this now even more time consuming test and not beeing able to skip it as "not admin" of this git repo during a release process.

_, err := r.rdsClient.DeleteDBCluster(newDeleteCentralDBClusterInput(clusterID, false))
if err != nil {
if awsErr, ok := err.(awserr.Error); ok {
// This assumes that if a final snapshot exists, a deletion for the RDS cluster was already trigger
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// This assumes that if a final snapshot exists, a deletion for the RDS cluster was already trigger
// This assumes that if a final snapshot exists, a deletion for the RDS cluster was already triggered

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

# At some point your central instance should become ready
```

1. Make sure DB state is available and 2 instances exist in state available
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: and Central is ready too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented May 8, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johannes94, vladbologa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [johannes94,vladbologa]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot removed the lgtm label May 9, 2023
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented May 9, 2023

New changes are detected. LGTM label has been removed.

@johannes94 johannes94 temporarily deployed to development May 9, 2023 06:43 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 9, 2023 06:43 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 9, 2023 06:43 — with GitHub Actions Inactive
@johannes94 johannes94 merged commit 5c60dfd into main May 9, 2023
@johannes94 johannes94 deleted the jmalsam/ROX-13692-fix-final-db-snapshot branch May 9, 2023 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants