Skip to content

perf(encoder): strategy-aware literal compression gates G4+G5 (skipLitCheck + minGain) #207

@polaz

Description

@polaz

Context

Follow-up to #126 (G3 whole-block bail-out, merged). Donor
ZSTD_compressLiterals has two more strategy-dependent gates we don't
yet honour:

G4 — skipLitCheck: when the strategy emits a synthetic
single-literal RLE-like block, donor skips the HUF compression attempt
entirely (zstd_compress_literals.c huf_compress_skipLitCheck path).

G5 — minGain: even after computing the HUF-compressed size, donor
requires the gain to exceed a strategy-specific minimum threshold
before emitting compressed literals (HUF_compress_internalminGain
formula varies per strategy: fast/dfast use a tighter threshold than
greedy/lazy/btultra).

Our path emits compressed literals whenever HUF reduces size by any
amount — produces sub-optimal frames vs donor on Fast/dfast levels.

Acceptance criteria

  • Port skipLitCheck short-circuit for synthetic RLE-like blocks
  • Port strategy-aware minGain formula matching donor per-strategy
    thresholds (cite donor file:line for each strategy)
  • compare_ffi REPORT: no ratio regression; Fast/dfast bytes within
    ±0.5% of donor on decodecorpus-* corpus
  • Unit tests for the gate decisions per strategy
  • No speed regression on encode hot path

Files involved

  • zstd/src/encoding/blocks/compressed.rs (where G3 currently lives)
  • zstd/src/huff0/huff0_encoder.rs (size estimator if needed)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2-mediumMedium priority — important improvementenhancementNew feature or requestperformancePerformance optimization

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions