"unexpected EOF" from some salza2 deflate data

[3b]

Following has a reasonable chance (30%+) of triggering EOF error from chipz:

(ql:quickload '(chipz salza2))

(let* ((d (map-into (make-array (expt 2 8) :element-type '(unsigned-byte 8))
                    (lambda () (random 255))))
       (v (salza2:compress-data d 'salza2:deflate-compressor)))
  (with-simple-restart (continue "continue")
    (let ((x (time (chipz:decompress nil 'chipz:deflate v))))
      (assert (equalp d x)))))

Not completely sure it is chipz rather than salza2, but other decompressors don't seem to object to the same data.

[3b]

smaller/non-random version

(let* ((d (coerce #(161 16 164 216 206 26 13 97 172 65 152 52 116 77 19 10) '(simple-array (unsigned-byte 8) 1)))
       (v (salza2:compress-data d 'salza2:deflate-compressor)))
  (with-simple-restart (continue "continue")
    (let ((x (time (chipz:decompress nil 'chipz:deflate v))))
      (assert (equalp d x)))))

[se-mz]

This is an issue with chipz. decode-value ensures as many bits as are needed for the longest code, which can read bits past the end of the input if the code for the end-of-block marker isn't at least as long as all other codes. An encoder will usually not produce such a tree since EOB is by definition the least frequently used symbol - except when the last block uses the fixed Huffman code. Of course, a stupid/malicious encoder could also trigger this in a dynamic block.

Since this issue only affects the rare raw Deflate (zlib & gzip have trailing checksums that end up protecting against this) and repeatedly calling ensure-bits in the hottest part of the code is probably a bad idea, it might be worth it to write a separate version of decode-value for raw Deflate.