replica/database: quiesce compaction before closing system tables dur…

…ing shutdown

During shutdown, as all system tables are closed in parallel, there is a
possibility of a race condition between compaction stoppage and the
closure of the compaction_history table. So, quiesce all the compaction
tasks before attempting to close the tables.

Fixes #15721

Signed-off-by: Lakshmi Narayanan Sreethar <lakshmi.sreethar@scylladb.com>

Closes #17218

(cherry picked from commit 3b7b315)

replica/database.cc

@@ -2396,6 +2396,10 @@ future<> database::shutdown() {
     co_await _stop_barrier.arrive_and_wait();
     b.cancel();
 
+    // stop compaction across all shards before closing tables
+    co_await _compaction_manager.drain();
+    co_await _stop_barrier.arrive_and_wait();
+
     // Closing a table can cause us to find a large partition. Since we want to record that, we have to close
     // system.large_partitions after the regular tables.
     co_await close_tables(database::table_kind::user);