Mempurge garbage bytes ticker does not consider some value types or data written to blob files

[CkTD]

rocksdb.memtable.garbage.bytes.at.flush is outdated bytes of data present on memtable at flush time.

It is calculated by total_payload_bytes - total_payload_bytes_written.

rocksdb/db/builder.cc

Line 275 in 6c3bf83

*memtable_garbage_bytes =

total_payload_bytes_written is the bytes writes to SSTs. When BlobDB is enabled , However, the values write to blob file will also be considered as garbage.

[ltamasi]

Seems to me there's more to this than the stats being off: mempurge jobs are not supposed to write blob files in the first place (in other words, large values are supposed to be extracted only by actual flush jobs).

[ltamasi]

Actually, I misread the code: there is no BlobFileBuilder in use in mempurge jobs, and the ticker @CkTD mentioned is incremented only in real flush jobs. So the underlying issue is related to the statistics, as originally reported. (There is actually a comment in the code mentioning this limitation: https://github.com/facebook/rocksdb/blob/main/db/builder.cc#L271-L272.) Sorry about the noise...