Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating inconsistent locality groups causes compaction failures #819

Closed
ivakegg opened this issue Dec 11, 2018 · 1 comment

Comments

@ivakegg
Copy link
Contributor

commented Dec 11, 2018

If the set of locality groups on a table are not consistent (e.g. overlapping), then compactions will start to fail. Normally this is fine in the short term except that in 1.8, 1.9, and possibly 2.0 this will actually cause a situation that cannot be remedied until the tservers are restarted (documented in another ticket). This ticket is to suggest that we prevent creating inconsistent locality group configurations in the first place.

@keith-turner keith-turner self-assigned this Dec 14, 2018

@keith-turner

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2018

I am thinking of implementing the following solution.

First, If inconsistent locality groups are configured, then log a warning. The reason for this is that locality groups are spread over multiple properties. So current code (that works) may create a briefly inconsistent/incorrect config. I would not want to break this code. The warning could let them know it would be better to set the props in a different order.

Second, read the locality group config at the start of the minor compaction. If its bad then use no locality groups when creating the new in mem map and writing out the file AND log a warning. This allows the tablet server to free memory.

keith-turner added a commit to keith-turner/accumulo that referenced this issue Dec 18, 2018

fixes apache#819 Handle bad locality group config.
Two changes were made for bad locality group config.  First, when setting
properties a warning  is issued if the change results in bad LG config.
Second, minor compactions will log a warning and proceed with no LG config
if the config is bad.

WIP

WIP

keith-turner added a commit to keith-turner/accumulo that referenced this issue Dec 18, 2018

fixes apache#819 Handle bad locality group config.
Two changes were made for bad locality group config.  First, when setting
properties a warning  is issued if the change results in bad LG config.
Second, minor compactions will log a warning and proceed with no LG config
if the config is bad.

@ctubbsii ctubbsii added this to Done in 1.9.3 Jun 14, 2019

@ctubbsii ctubbsii added this to Done in 2.0.0 Jun 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
2 participants
You can’t perform that action at this time.