Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(migrations): implement validate command #7087

Merged
merged 8 commits into from
Feb 24, 2021

Conversation

vcrfxia
Copy link
Contributor

@vcrfxia vcrfxia commented Feb 24, 2021

Description

The validate command of the migrations tool:

  1. Finds the latest migrated version by querying the metadata table for the current version and following "previous" pointers until one with status MIGRATED is found. This should never involve more than one hop following pointers since the apply command should always use the latest MIGRATED version as "previous". Currently an error is thrown if this is not the case, but we can revisit when we introduce the abort command to get out of bad states due to race conditions.
  2. Continuing to follow "previous" pointers, validating each file's checksum (and that each version has status MIGRATED) along the way.
  3. The "previous" pointers eventually lead to the special NONE_VERSION, indicating the first migration version has been reached.

Once table scans are implemented, we can simplify the algorithm by simply doing a table scan and filtering for status MIGRATED.

Testing done

Lots of unit tests. Integration tests will be added with the apply implementation.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@vcrfxia vcrfxia requested a review from a team as a code owner February 24, 2021 05:18
@@ -47,4 +55,105 @@ public static String getCurrentVersion(final MigrationConfig config, final Clien
String.format("Could not query %s: %s", migrationTableName, e.getMessage()));
}
}

public static String getLatestMigratedVersion(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jzaralim this method will be useful for implementing apply, in order to get the latest version to start applying commands from.

/**
* @return true if validation passes, else false.
*/
static boolean validate(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is non-private since we'll want to call it from the apply command, in order to validate applied migrations prior to submitting any new ones ( cc @jzaralim )

Copy link
Contributor

@jzaralim jzaralim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is super helpful!

@vcrfxia vcrfxia merged commit d4cf400 into confluentinc:master Feb 24, 2021
@vcrfxia vcrfxia deleted the migrations-validate branch February 24, 2021 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants