Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compaction IT that verifies queues are cleared when tablets no longer need to compact. #4399

Closed
keith-turner opened this issue Mar 18, 2024 · 3 comments · Fixed by #4466
Assignees
Milestone

Comments

@keith-turner
Copy link
Contributor

This is follow on work for #3528 and #4394. Would be good to add an integration test for the changes in #4394. Unit test were added in #4394 that cover a narrow part of the code, but it would be good to verify things work end to end. Should be able to do something like the following in an IT.

  • Create a table w/ N tablets
  • Configure the table to use compaction group G1
  • Start no compactors processes for G1. This should cause compactions to queue but not run.
  • Add files and change compaction settings such that all tablets need to compact, could lower the compaction ratio and/or add more files per tablet than the compaction ratio
  • Verify N tablets are queued for G1 via inspecting metrics
  • Change the table config in such a way that no tablets need to compact, could raise the compaction ratio. Once the tablets no longer need to compact they should be cleared from the queue.
  • Verify zero tablets are queued for G1 via inspecting metrics
@keith-turner
Copy link
Contributor Author

It may be worthwhile to add this test to 2.1 and forward. The implementations are very different, but for this case hopefully the behavior is the same.

@DomGarguilo
Copy link
Member

I can start looking into this.

@DomGarguilo
Copy link
Member

Closed via #4466

@ctubbsii ctubbsii added this to the 4.0.0 milestone Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
3 participants