Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(kv): add KV migration to repair DBRP mappings broken by schema change #20168

Merged
merged 3 commits into from
Nov 25, 2020

Conversation

danxmoran
Copy link
Contributor

Closes #20167

I tested the new command locally, happy to refactor and set up automated tests if the reviewers would like.

@danxmoran danxmoran marked this pull request as draft November 24, 2020 22:16
@danxmoran
Copy link
Contributor Author

Marking as draft while I convert the logic to a KV migration (thanks for the idea @stuartcarnie!)

@danxmoran danxmoran marked this pull request as ready for review November 25, 2020 15:35
// repair DBRP owner and bucket IDs
Migration0013_RepairDBRPOwnerAndBucketIDs,
// reindex DBRPs
Migration0014_ReindexDBRPs,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reindex here is required for the Find methods in the DBRP service to locate repaired DBRPs. Reordering the migrations so my new Migration0013 runs before the existing Migration0012 also works, but that felt like a potentially-dangerous idea.

"github.com/influxdata/influxdb/v2/kv"
)

var Migration0013_RepairDBRPOwnerAndBucketIDs = UpOnlyMigration(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic & structure of this file is largely copied from the 0012 migration.

@danxmoran
Copy link
Contributor Author

To test this change, I:

  1. Upgraded a local 1.8 DB using a v2.0.1 influxd upgrade
  2. Ran a v2.0.2 influxd pointing at the upgraded data
  3. Used a v2.0.2 influx v1 dbrp list to trigger the panic that occurs because of the data corruption
  4. Used influx v1 dbrp create to create a new DBRP mapping using the v2.0.2 schema
  5. Stopped the v2.0.2 influxd, and started a new one built from my branch
  6. Used the v2.0.2 influx v1 dbrp list to confirm the panic was fixed, and all DBRPs were returned

@danxmoran danxmoran changed the title feat(cmd/influxd): add CLI command to repair broken DBRP mappings fix(kv): add KV migration to repair DBRP mappings broken by schema change Nov 25, 2020
Copy link
Contributor

@GeorgeMac GeorgeMac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. Cheers @danxmoran !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate DBRPs from old KV schema to new on startup
2 participants