Skip to content

Commit

Permalink
Deflake unit test DBCompactionTest.CompactionLimiter (#12596)
Browse files Browse the repository at this point in the history
Summary:
The test has been flaky for a long time. A recent [failure](https://github.com/facebook/rocksdb/actions/runs/8820808355/job/24215219590?pr=12578) shows that there is still flush running when the assertion fails. I think this is because `WaitForFlushMemTable()` may return before the a flush schedules the next compaction.

Pull Request resolved: #12596

Test Plan: I could not repro the failure locally: `gtest-parallel --repeat=8000 --workers=100 ./db_compaction_test --gtest_filter="*CompactionLimiter*"`

Reviewed By: ajkr

Differential Revision: D56715874

Pulled By: cbi42

fbshipit-source-id: f5f64eb30fff7e115c19beedad2dc22afa06258d
  • Loading branch information
cbi42 authored and facebook-github-bot committed May 3, 2024
1 parent 2cd4346 commit e2ef349
Showing 1 changed file with 6 additions and 21 deletions.
27 changes: 6 additions & 21 deletions db/db_compaction_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -6204,29 +6204,14 @@ TEST_F(DBCompactionTest, CompactionLimiter) {
NumTableFilesAtLevel(0, 0));
}

// All CFs are pending compaction
// Wait until all CFs are pending compaction. WaitForFlushMemtable() can
// return before the next compaction is scheduled, so we need to do some
// waiting here.
unsigned int tp_len = env_->GetThreadPoolQueueLen(Env::LOW);
if (cf_count != tp_len) {
// The test is flaky and fails the assertion below.
// Print some debug information.
uint64_t num_running_flushes = 0;
if (db_->GetIntProperty(DB::Properties::kNumRunningFlushes,
&num_running_flushes)) {
fprintf(stdout, "Running flushes: %" PRIu64 "\n", num_running_flushes);
}
fprintf(stdout,
"%zu CF in compaction queue: ", pending_compaction_cfs.size());
for (const auto& cf_name : pending_compaction_cfs) {
fprintf(stdout, "%s, ", cf_name.c_str());
}
fprintf(stdout, "\n");

// print lsm
for (unsigned int cf = 0; cf < cf_count; cf++) {
fprintf(stdout, "%s: %s\n", cf_names[cf], FilesPerLevel(cf).c_str());
}
for (int i = 0; i < 10000 && tp_len < cf_count; i++) {
env_->SleepForMicroseconds(1000);
tp_len = env_->GetThreadPoolQueueLen(Env::LOW);
}

ASSERT_EQ(cf_count, tp_len);

// Unblock all compaction threads
Expand Down

0 comments on commit e2ef349

Please sign in to comment.