Skip to content

i1.38.2

i1.38.2: fix: handle incompressible data in HuffmanCheckTask (#7251)
@shitaoxai shitaoxai tagged this 03 May 11:38 
HuffmanCheckTask crashes the process with SIGABRT when key data is
incompressible (e.g. random hashes, UUIDs, encrypted blobs):

  F huff_coder.cc:108] Check failed: !HUF_isError(size)

The task runs as an idle task on shard 0 once key memory exceeds 50 MB.
It builds a Huffman frequency table from sampled key data, then
unconditionally calls HuffmanEncoder::Export() to serialize the table
for logging. When the data is high-entropy (compression ratio = 1.0),
HUF_writeCTable_wksp returns an error for the degenerate table and the
CHECK kills the process. Since the task re-triggers on every restart
after snapshot recovery, this results in a crash loop.

Note: --compression_mode does not prevent this. That flag controls
snapshot serialization compression (LZ4/ZSTD), not the background
HuffmanCheckTask.

Fix:
- Export() returns std::optional<std::string> instead of crashing via
  CHECK. Logs a warning on failure.
- HuffmanCheckTask::Run() skips Export() when compression ratio >= 1.0,
  since there is no useful table to log.
- All callers (debugcmd.cc, tests) updated to handle the optional.
Assets 2
Loading