Error writing data: Corruption: bad entry in block

[toktarev]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

Expected behavior

No such problem

Actual behavior

Error writing data: Corruption: bad entry in block

Steps to reproduce the behavior

I have difficult business code which uses a about 4 column families, Merge operator and execute frequent writes.

After about 200_000_000 records has been written I see that application hunging and after that the only one message I see is:

Error writing data: Corruption: bad entry in block

Was not able to reproduce this bug in separate C++ script.

Can't attach here my business code.

[toktarev]

Configuration parameters:

  auto blockOpts = rocksdb::BlockBasedTableOptions{};
  blockOpts.index_type = rocksdb::BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch;
  blockOpts.partition_filters = false;
  blockOpts.metadata_block_size = 4096;
  blockOpts.block_size = kernel_opts.block_size;
  blockOpts.block_cache = block_cache;
  blockOpts.cache_index_and_filter_blocks = true;
  blockOpts.cache_index_and_filter_blocks_with_high_priority = true;
  blockOpts.filter_policy = nullptr;

[toktarev]

stack trace:
  1   librocksdb.5.dylib                  0x0000000107ff5e75 _ZN7rocksdb23BlockBasedTableIterator4NextEv + 21
  2   librocksdb.5.dylib                  0x0000000108056793 _ZN7rocksdb15MergingIterator4NextEv + 51
  3   librocksdb.5.dylib                  0x0000000107e693dc _ZN7rocksdb11MergeHelper10MergeUntilEPNS_16InternalIteratorEPNS_18RangeDelAggregatorEyb + 1564
  4   librocksdb.5.dylib                  0x0000000107d358af _ZN7rocksdb18CompactionIterator13NextFromInputEv + 1951
  5   librocksdb.5.dylib                  0x0000000107d361e2 _ZN7rocksdb18CompactionIterator4NextEv + 386
  6   librocksdb.5.dylib                  0x0000000107d3edd0 _ZN7rocksdb13CompactionJob25ProcessKeyValueCompactionEPNS0_18SubcompactionStateE + 3792
  7   librocksdb.5.dylib                  0x0000000107d3cd43 _ZN7rocksdb13CompactionJob3RunEv + 419
  8   librocksdb.5.dylib                  0x0000000107dab329 _ZN7rocksdb6DBImpl20BackgroundCompactionEPbPNS_10JobContextEPNS_9LogBufferEPNS0_19PrepickedCompactionE + 4921
  9   librocksdb.5.dylib                  0x0000000107da97c0 _ZN7rocksdb6DBImpl24BackgroundCallCompactionEPNS0_19PrepickedCompactionENS_3Env8PriorityE + 192
  10  librocksdb.5.dylib                  0x0000000107da84e8 _ZN7rocksdb6DBImpl16BGWorkCompactionEPv + 56
  11  librocksdb.5.dylib                  0x00000001081575a7 _ZN7rocksdb14ThreadPoolImpl4Impl8BGThreadEm + 487
  12  librocksdb.5.dylib                  0x0000000108157851 _ZN7rocksdb14ThreadPoolImpl4Impl15BGThreadWrapperEPv + 97
  13  librocksdb.5.dylib                  0x000000010815925c _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEPFPvS7_EPN7rocksdb16BGThreadMetadataEEEEEES7_S7_ + 44
  14  libsystem_pthread.dylib             0x00007fff5e0fc33d _pthread_body + 126
  15  libsystem_pthread.dylib             0x00007fff5e0ff2a7 _pthread_start + 70
  16  libsystem_pthread.dylib             0x00007fff5e0fb425 thread_start + 13

[Rafat001]

Similar issue but not exactly same https://stackoverflow.com/questions/70300584/getting-org-rocksdb-rocksdbexception-bad-entry-in-block