Skip to content

Fix bug in DefaultKeySizeConstraint#2025

Closed
milleruntime wants to merge 1 commit intoapache:mainfrom
milleruntime:defaultKeySizeConstraint-bug
Closed

Fix bug in DefaultKeySizeConstraint#2025
milleruntime wants to merge 1 commit intoapache:mainfrom
milleruntime:defaultKeySizeConstraint-bug

Conversation

@milleruntime
Copy link
Contributor

  • Stop DefaultKeySizeConstraint from allowing keys with a large rowId that
    did not have any column update mutations
  • Add a test to DefaultKeySizeConstraintTest for this condition

* Stop DefaultKeySizeConstraint from allowing keys with a large rowId that
did not have any column update mutations
* Add a test to DefaultKeySizeConstraintTest for this condition

int size = mutation.getRow().length;
for (ColumnUpdate cu : mutation.getUpdates()) {
int size = mutation.getRow().length;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limited documentation on the class says it limits the size of keys. A column update would correspond to a key, so the existing code may correct. One change would be that it could break out of the loop after adding the first violation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure but to me the row ID is an integral part of the Key and a user could very easily create a key larger than the constraint if they have a large row ID

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a user could very easily create a key larger than the constraint if they have a large row ID

The row length was being factored into the size of each column update which would equate to the eventual key size.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but if there aren't any Column updates the row length will be ignored. Adding this chunk of code to the test will cause it to fail now:

m = new Mutation(oversized);
assertEquals(Collections.singletonList(DefaultKeySizeConstraint.MAX__KEY_SIZE_EXCEEDED_VIOLATION), 
        constraint.check(null, m));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BatchWriter at least prevents this from every happening and will throw an exception:

throw new IllegalArgumentException("Can not add empty mutations");

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but if there aren't any Column updates the row length will be ignored.

Ok I see what you are thinking about. Hopefully that case does not matter. When a mutation has no column updates, the batchwriter should reject it. Also if a mutation w/ no column update were to make it through somehow, when its inserted into the in memory map its converted to keys/values. If column updates are empty, then no key values will be created.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current code seems to impose a limit on any total key size (4 dimensions: row + cf + cq + cv; doesn't include timestamp or internal delete flag dimensions). Your change seems to impose a limit on the total size of this mutation, and seems to turn it into a MutationSizeConstraint.

@milleruntime
Copy link
Contributor Author

I guess it doesn't really matter if it is a bug or not, the question should be whether or not this should be changed. Changing this Constraint could break existing tables with big Row IDs.

@milleruntime
Copy link
Contributor Author

Closing this since it looks like it there are checks to prevent this situation from occurring.

@milleruntime milleruntime deleted the defaultKeySizeConstraint-bug branch April 26, 2021 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants