Segfault During compaction using FIFO Compaction style for a single CF

[wmorgan6796]

Expected behavior

As we're writing data to the CF for RocksDB via write batches of approximately 1000 keys per batch. Over time using FIFO with compaction style with compaction enabled, with a constant number of writes, we would expect the L0 SST files to be created until we hit the level0_file_num_compaction_trigger. Once that gets hit, we would expect the L0 files to compact down to a single file and then Rocks would continue with the next "batch" of SST files.

Some notes:

1. We use Leveled Compaction for all other Column Families in the same DB, only 1 column family has FIFO compaction enabled
2. RocksDB configuration is attached. The relevant CF is CF-DATA RocksDB-Options.txt

Actual behavior

We segfault pretty quickly (under an hour of process runtime) with the following segfault:

 logging.cpp:20 - *** Aborted at 1712944389 (unix time) try "date -d @1712944389" if you are using GNU date ***
 logging.cpp:20 - PC: @           0xfd5da2 rocksdb::VersionSet::GetObsoleteFiles(std::vector<rocksdb::ObsoleteFileInfo, std::allocator<rocksdb::ObsoleteFileInfo> >*, std::vector<rocksdb::ObsoleteBlobFileInfo, std::allocator<rocksdb::ObsoleteBlobFileInfo> >*, std::vector<std::__cxx11::basic_string<cþ^D
 logging.cpp:20 - *** SIGSEGV (@0x198) received by PID 3394741 (TID 0x7f6ca789b700) from PID 408; stack trace: ***
 logging.cpp:20 -     @     0x7f6ea0f9e4df __pthread_once_slow
 logging.cpp:20 -     @          0x23df528 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
 logging.cpp:20 -     @     0x7f6ea0fa1420 (unknown)
 logging.cpp:20 -     @           0xfd5da2 rocksdb::VersionSet::GetObsoleteFiles(std::vector<rocksdb::ObsoleteFileInfo, std::allocator<rocksdb::ObsoleteFileInfo> >*, std::vector<rocksdb::ObsoleteBlobFileInfo, std::allocator<rocksdb::ObsoleteBlobFileInfo> >*, std::vector<std::__cxx11::basic_string<cþ^D
 logging.cpp:20 -     @           0xe9b498 rocksdb::DBImpl::FindObsoleteFiles(rocksdb::JobContext*, bool, bool)
 logging.cpp:20 -     @           0xe2bf5a rocksdb::(anonymous namespace)::CleanupSuperVersionHandle(void*, void*)
 logging.cpp:20 -     @          0x12428ce rocksdb::Cleanable::~Cleanable()
 logging.cpp:20 -     @           0xd66dce rocksdb::DBIter::~DBIter()
 logging.cpp:20 -     @           0xd67521 rocksdb::ArenaWrappedDBIter::~ArenaWrappedDBIter()
 logging.cpp:20 -     @           0xcbf712 RocksDBStorage::RocksDBStorageIterator::~RocksDBStorageIterator()
 logging.cpp:20 -     @           0xcbf729 RocksDBStorage::RocksDBStorageIterator::~RocksDBStorageIterator()
 logging.cpp:20 -     @           0xcdbad0 ... REDACTED ...
 logging.cpp:20 -     @          0x27cf2f3 execute_native_thread_routine
 logging.cpp:20 -     @     0x7f6ea0f95609 start_thread

When we're not segfaulting, we also see that the L0 compaction doesn't catch all the files in the L0 and we end up with a quite a few long-lived (> 2 hours old), small SST files that never get compacted when they should.

Steps to reproduce the behavior

Run FIFO compaction with L0 compaction enabled and a sufficiently high enough write and read throughput.