Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds padding to keyrange comparison #8296

Merged
merged 3 commits into from
Jun 10, 2021

Conversation

rafael
Copy link
Member

@rafael rafael commented Jun 9, 2021

Description

  • While trying to reshard a keyspace from 256 to 512, we discovered a bug in KeyRange comparison operations. Keyspaces larger than 256 shards, require two bytes (e.g -0080 to ff80-).
  • In Vitess, a keyrange is implicitly a 8 byte integer. So the following are equivalent:
    -80 == 00-80 == 0000-8000 == 000000-800000
    
  • Existent comparisons do not return equal when comparing 0000-8000 == 000000-800000.
  • The following PR fixes that problem by making sure that keyranges are padded as 8 byte integers.

Checklist

  • Tests were added or are not required
  • Documentation was added or is not required

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
@rafael rafael changed the title Adds padding to keyrange comparison [DRAFT] Adds padding to keyrange comparison Jun 9, 2021
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
@rafael rafael changed the title [DRAFT] Adds padding to keyrange comparison Adds padding to keyrange comparison Jun 9, 2021
// This means that from a keyrange perspective -80 == 00-80 == 0000-8000 == 000000-800000
// If we don't add this padding, we could run into issues when transitioning from keyranges
// that use 2 bytes to 4 bytes.
func addPadding(kr []byte) []byte {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like it's converting a key from a "short" version into something normalized? It might make sense to expose this as a function on the KeyRange struct or as a (KeyRange) -> KeyRange then make that an exported function from the package.

I don't think this is a blocking comment; this PR seems safe/good to merge is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The (very reasonable) counterpoint here is that it's a bit late to start talking about a normalized keyrange format; i don't really know if it's worth trying to go down that road at this point unless this comes up again.

Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests.

@deepthi deepthi merged commit 36657df into vitessio:main Jun 10, 2021
rafael added a commit to tinyspeck/vitess that referenced this pull request Sep 23, 2021
* We ran into some issues related to padding. This is another form of the bug
fixed in here: vitessio#8296

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
ajm188 pushed a commit to tinyspeck/vitess that referenced this pull request Sep 28, 2021
…ge-comparison

Adds padding to keyrange comparison
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants