Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Creating inconsistent locality groups causes compaction failures #819
If the set of locality groups on a table are not consistent (e.g. overlapping), then compactions will start to fail. Normally this is fine in the short term except that in 1.8, 1.9, and possibly 2.0 this will actually cause a situation that cannot be remedied until the tservers are restarted (documented in another ticket). This ticket is to suggest that we prevent creating inconsistent locality group configurations in the first place.
I am thinking of implementing the following solution.
First, If inconsistent locality groups are configured, then log a warning. The reason for this is that locality groups are spread over multiple properties. So current code (that works) may create a briefly inconsistent/incorrect config. I would not want to break this code. The warning could let them know it would be better to set the props in a different order.
Second, read the locality group config at the start of the minor compaction. If its bad then use no locality groups when creating the new in mem map and writing out the file AND log a warning. This allows the tablet server to free memory.