compaction: Disconsider active tables in the hourly compaction reeval…

…uation

This hourly reevaluation is there to help tablets that have very low
write activity, which can go a long time without flushing a memtable,
and it's important to reevaluate compaction as data can get expired.
Today it can happen that we reevaluate a table that is being compacted
actively, which is waste of cpu as the reevaluation will happen anyway
when there are changes to sstable set. This waste can be amplified with
a significant tablet count in a given shard.
Eventually, we could make the revaluation time per table based on
expiration histogram, but until we get there, let's avoid this waste
by only reevaluating tables that are compaction idle for more than 1h.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>

Closes #18280

compaction/compaction_manager.cc

@@ -976,8 +976,11 @@ void compaction_manager::enable() {
 
 std::function<void()> compaction_manager::compaction_submission_callback() {
     return [this] () mutable {
-        for (auto& e: _compaction_state) {
-            postpone_compaction_for_table(e.first);
+        auto now = gc_clock::now();
+        for (auto& [table, state] : _compaction_state) {
+            if (now - state.last_regular_compaction > periodic_compaction_submission_interval()) {
+                postpone_compaction_for_table(table);
+            }
         }
         reevaluate_postponed_compactions();
     };
@@ -1227,6 +1230,7 @@ class regular_compaction_task_executor : public compaction_task_executor, public
                 fmt::ptr(this), descriptor.sstables.size(), weight, t);
 
             setup_new_compaction(descriptor.run_identifier);
+            _compaction_state.last_regular_compaction = gc_clock::now();
             std::exception_ptr ex;
 
             try {

compaction/compaction_state.hh

@@ -15,6 +15,7 @@
 
 #include "compaction/compaction_fwd.hh"
 #include "compaction/compaction_backlog_manager.hh"
+#include "gc_clock.hh"
 
 namespace compaction {
 
@@ -37,6 +38,8 @@ struct compaction_state {
     std::unordered_set<sstables::shared_sstable> sstables_requiring_cleanup;
     compaction::owned_ranges_ptr owned_ranges_ptr;
 
+    gc_clock::time_point last_regular_compaction;
+
     explicit compaction_state(table_state& t);
     compaction_state(compaction_state&&) = delete;
     ~compaction_state();