[DocDB] tests using test fixture PgMiniLargeClockSkewTest cause memory corruption, which sometimes makes them fail

[mdbridge]

Jira Link: DB-5902

Description

Tests based on test fixture PgMiniLargeClockSkewTest have a heap overflow bug that corrupts the heap and sometimes causes them to spuriously fail. It is also unclear if they actually are testing large clock skew.

In particular, the following code used by this test fixture:

/home/mdbridge/code/yugabyte-db/src/yb/integration-tests/mini_cluster.cc:747:
server::SkewedClockDeltaChanger JumpClock(
    server::RpcServerBase* server, std::chrono::milliseconds delta) {
  auto* hybrid_clock = down_cast<server::HybridClock*>(server->clock());
  return server::SkewedClockDeltaChanger(
      delta, std::static_pointer_cast<server::SkewedClock>(hybrid_clock->physical_clock()));
}

Assumes that the clock the mini cluster has been initialized with is a SkewedClock, but the fixture set up does not specify this:

/home/mdbridge/code/yugabyte-db/src/yb/yql/pgwrapper/pg_mini-test.cc:615:
class PgMiniLargeClockSkewTest : public PgMiniTest {
 public:
  void SetUp() override {
    SetAtomicFlag(250000ULL, &FLAGS_max_clock_skew_usec);
    PgMiniTestBase::SetUp();
  }
};

Because of this bug, SkewedClockDeltaChanger attempts to change the delta of the underlying clock by mutating an unallocated location in the heap. :-(

To resolve this issue, the fixture needs to specify that time source and code should be added to fail explicitly if the correct time source is missing instead of just silently corrupting memory.

To reproduce this issue, you can run ASAN for the test PgMiniTest.ReadRestartSnapshot; note that ASAN is disabled for these tests so you will have to manually enable them by changing the source code first.

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.