Note. Some stacks below are more likely to be leaks than others. On the other hand all of them are hard to believe to be leaks but something is leaking. Accumulated ~19Gb ================== ntdll!RtlpAllocateHeap+1DE9 (d:\rs1\minkernel\ntos\rtl\heap.c, 6734) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTableIterator::InitDataBlock+2DE (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2103) RocksDBStore!rocksdb::BlockBasedTableIterator::FindKeyForward+D8 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2125) RocksDBStore!::operator()+169 (d:\vsts\agent_1\_work\1\s\db\compaction_job.cc, 647) RocksDBStore!rocksdb::CompactionJob::Run+660 (d:\vsts\agent_1\_work\1\s\db\compaction_job.cc, 664) RocksDBStore!rocksdb::DBImpl::BackgroundCompaction+1250 (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1895) RocksDBStore!rocksdb::DBImpl::BackgroundCallCompaction+D8 (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1504) RocksDBStore!rocksdb::DBImpl::BGWorkCompaction+5F (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1319) ~1GB Listed separate from the above for some reason ======= ntdll!RtlpAllocateHeap+20A5 (d:\rs1\minkernel\ntos\rtl\heap.c, 6882) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTableIterator::InitDataBlock+2DE (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2103) RocksDBStore!rocksdb::BlockBasedTableIterator::FindKeyForward+D8 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2125) RocksDBStore!::operator()+169 (d:\vsts\agent_1\_work\1\s\db\compaction_job.cc, 647) RocksDBStore!rocksdb::CompactionJob::Run+660 (d:\vsts\agent_1\_work\1\s\db\compaction_job.cc, 664) RocksDBStore!rocksdb::DBImpl::BackgroundCompaction+1250 (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1895) RocksDBStore!rocksdb::DBImpl::BackgroundCallCompaction+D8 (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1504) RocksDBStore!rocksdb::DBImpl::BGWorkCompaction+5F (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1319) Accumulated 7GB (MultiGet) ================ ntdll!RtlpAllocateHeap+1DE9 (d:\rs1\minkernel\ntos\rtl\heap.c, 6734) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+CD (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1600) RocksDBStore!rocksdb::BlockBasedTable::Get+34F (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2318) RocksDBStore!rocksdb::TableCache::Get+5ED (d:\vsts\agent_1\_work\1\s\db\table_cache.cc, 413) RocksDBStore!rocksdb::Version::Get+2ED (d:\vsts\agent_1\_work\1\s\db\version_set.cc, 1200) RocksDBStore!rocksdb::DBImpl::MultiGet+81B (d:\vsts\agent_1\_work\1\s\db\db_impl.cc, 1259) RocksDBStore!rocksdb::StackableDB::MultiGet+33 (d:\vsts\agent_1\_work\1\s\include\rocksdb\utilities\stackable_db.h, 99) 0.5 Gb (MultiGet) == ntdll!RtlpAllocateHeap+20A5 (d:\rs1\minkernel\ntos\rtl\heap.c, 6882) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+CD (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1600) RocksDBStore!rocksdb::BlockBasedTable::Get+34F (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2318) RocksDBStore!rocksdb::TableCache::Get+5ED (d:\vsts\agent_1\_work\1\s\db\table_cache.cc, 413) RocksDBStore!rocksdb::Version::Get+2ED (d:\vsts\agent_1\_work\1\s\db\version_set.cc, 1200) RocksDBStore!rocksdb::DBImpl::MultiGet+81B (d:\vsts\agent_1\_work\1\s\db\db_impl.cc, 1259) RocksDBStore!rocksdb::StackableDB::MultiGet+33 (d:\vsts\agent_1\_work\1\s\include\rocksdb\utilities\stackable_db.h, 99) > 1.2 Gb =========== ntdll!RtlpAllocateHeap+1DE9 (d:\rs1\minkernel\ntos\rtl\heap.c, 6734) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTableIterator::InitDataBlock+2DE (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2103) RocksDBStore!rocksdb::BlockBasedTableIterator::FindKeyForward+D8 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2125) RocksDBStore!rocksdb::`anonymous namespace'::LevelIterator::Next+16 (d:\vsts\agent_1\_work\1\s\db\version_set.cc, 617) RocksDBStore!rocksdb::MergingIterator::Next+3A (d:\vsts\agent_1\_work\1\s\table\merging_iterator.cc, 202) RocksDBStore!rocksdb::DBIter::Next+9C (d:\vsts\agent_1\_work\1\s\db\db_iter.cc, 389) > 3Gb ===== ntdll!RtlpAllocateHeap+1DE9 (d:\rs1\minkernel\ntos\rtl\heap.c, 6734) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::ZSTD_Uncompress+65 (d:\vsts\agent_1\_work\1\s\util\compression.h, 1044) RocksDBStore!rocksdb::UncompressBlockContentsForCompressionType+27D (d:\vsts\agent_1\_work\1\s\table\format.cc, 354) RocksDBStore!rocksdb::UncompressBlockContents+3B (d:\vsts\agent_1\_work\1\s\table\format.cc, 390) RocksDBStore!rocksdb::BlockFetcher::ReadBlockContents+552 (d:\vsts\agent_1\_work\1\s\table\block_fetcher.cc, 230) RocksDBStore!rocksdb::`anonymous namespace'::ReadBlockFromFile+12D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 87) RocksDBStore!rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache+3A9 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1748) RocksDBStore!rocksdb::BlockBasedTable::NewDataBlockIterator+19D (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 1624) RocksDBStore!rocksdb::BlockBasedTableIterator::InitDataBlock+2DE (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2103) RocksDBStore!rocksdb::BlockBasedTableIterator::FindKeyForward+D8 (d:\vsts\agent_1\_work\1\s\table\block_based_table_reader.cc, 2125) RocksDBStore!rocksdb::BuildTable+DA9 (d:\vsts\agent_1\_work\1\s\db\builder.cc, 216) RocksDBStore!rocksdb::FlushJob::WriteLevel0Table+A4F (d:\vsts\agent_1\_work\1\s\db\flush_job.cc, 351) RocksDBStore!rocksdb::FlushJob::Run+115 (d:\vsts\agent_1\_work\1\s\db\flush_job.cc, 217) RocksDBStore!rocksdb::DBImpl::FlushMemTableToOutputFile+371 (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 157) RocksDBStore!rocksdb::DBImpl::BackgroundFlush+2CD (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1400) RocksDBStore!rocksdb::DBImpl::BackgroundCallFlush+BB (d:\vsts\agent_1\_work\1\s\db\db_impl_compaction_flush.cc, 1425) > 3Gb =========== ntdll!RtlpAllocateHeap+20A5 (d:\rs1\minkernel\ntos\rtl\heap.c, 6882) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::Arena::AllocateNewBlock+50 (d:\vsts\agent_1\_work\1\s\util\arena.cc, 232) RocksDBStore!rocksdb::Arena::AllocateAligned+A4 (d:\vsts\agent_1\_work\1\s\util\arena.cc, 201) RocksDBStore!rocksdb::ConcurrentArena::AllocateImpl< >+322 (d:\vsts\agent_1\_work\1\s\util\concurrent_arena.h, 145) RocksDBStore!rocksdb::ConcurrentArena::AllocateAligned+33 (d:\vsts\agent_1\_work\1\s\util\concurrent_arena.h, 64) RocksDBStore!rocksdb::`anonymous namespace'::SkipListRep::Allocate+50 (d:\vsts\agent_1\_work\1\s\memtable\skiplistrep.cc, 33) RocksDBStore!rocksdb::MemTable::Add+C5 (d:\vsts\agent_1\_work\1\s\db\memtable.cc, 466) RocksDBStore!rocksdb::MemTableInserter::PutCFImpl+1E9 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1245) RocksDBStore!rocksdb::MemTableInserter::PutCF+25 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1318) RocksDBStore!rocksdb::WriteBatch::Iterate+2FF (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 445) RocksDBStore!rocksdb::WriteBatchInternal::InsertInto+149 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1769) RocksDBStore!rocksdb::DBImpl::WriteImpl+11A7 (d:\vsts\agent_1\_work\1\s\db\db_impl_write.cc, 311) RocksDBStore!rocksdb::DBImpl::Write+36 (d:\vsts\agent_1\_work\1\s\db\db_impl_write.cc, 55) RocksDBStore!rocksdb::WriteCommittedTxn::CommitBatchInternal+29 (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction.cc, 314) RocksDBStore!rocksdb::PessimisticTransaction::CommitBatch+2AD (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction.cc, 153) RocksDBStore!rocksdb::PessimisticTransactionDB::Write+8D (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction_db.cc, 511) > 1.5 Gb ============= ntdll!RtlpAllocateHeap+20A5 (d:\rs1\minkernel\ntos\rtl\heap.c, 6882) ntdll!RtlpAllocateHeapInternal+727 (d:\rs1\minkernel\ntos\rtl\heap.c, 2021) ucrtbase!_malloc_base+36 (d:\rs1\minkernel\crts\ucrt\src\appcrt\heap\malloc_base.cpp, 34) RocksDBStore!operator new+31 (f:\dd\vctools\crt\vcstartup\src\heap\new_scalar.cpp, 19) RocksDBStore!rocksdb::Arena::AllocateNewBlock+50 (d:\vsts\agent_1\_work\1\s\util\arena.cc, 232) RocksDBStore!rocksdb::Arena::AllocateAligned+A4 (d:\vsts\agent_1\_work\1\s\util\arena.cc, 201) RocksDBStore!rocksdb::ConcurrentArena::AllocateImpl< >+322 (d:\vsts\agent_1\_work\1\s\util\concurrent_arena.h, 145) RocksDBStore!rocksdb::ConcurrentArena::AllocateAligned+33 (d:\vsts\agent_1\_work\1\s\util\concurrent_arena.h, 64) RocksDBStore!rocksdb::`anonymous namespace'::SkipListRep::Allocate+50 (d:\vsts\agent_1\_work\1\s\memtable\skiplistrep.cc, 33) RocksDBStore!rocksdb::MemTable::Add+C5 (d:\vsts\agent_1\_work\1\s\db\memtable.cc, 466) RocksDBStore!rocksdb::MemTableInserter::PutCFImpl+1E9 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1245) RocksDBStore!rocksdb::MemTableInserter::PutCF+25 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1318) RocksDBStore!rocksdb::WriteBatch::Iterate+2FF (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 445) RocksDBStore!rocksdb::WriteBatchInternal::InsertInto+149 (d:\vsts\agent_1\_work\1\s\db\write_batch.cc, 1769) RocksDBStore!rocksdb::DBImpl::WriteImpl+11A7 (d:\vsts\agent_1\_work\1\s\db\db_impl_write.cc, 311) RocksDBStore!rocksdb::DBImpl::Write+36 (d:\vsts\agent_1\_work\1\s\db\db_impl_write.cc, 55) RocksDBStore!rocksdb::WriteCommittedTxn::CommitBatchInternal+29 (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction.cc, 314) RocksDBStore!rocksdb::PessimisticTransaction::CommitBatch+2AD (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction.cc, 153) RocksDBStore!rocksdb::PessimisticTransactionDB::Write+8D (d:\vsts\agent_1\_work\1\s\utilities\transactions\pessimistic_transaction_db.cc, 511)