Handling 0-length input (always output frame+block)

[klauspost]

There is no strict clause in RFC 8878 that says that a 0-length input is invalid.

In upstream a 0-length input is output as 0 bytes, with an option to always write a Frame + EOF block.

The approach to this should be decided. Mainly on the encoder side, but also whether the decoder should return an error if it encounters EOF straight away.

[Jorropo]

What does other libraries in the ecosystem like the facebook C implementation does about it ?

[klauspost]

The commandline rejects it, but actually not sure if it is the library or the commandline tool.

I guess the safest option is to write the frame+block - 9 bytes. And make the Reader return io.ErrUnexpectedEOF if no frame is found. Then the implementer can decide if they want to do a pre-check for 0-length.

[klauspost]

#12 would be the code to not accept []byte{} <-> []byte{}.

[terrelln]

From the RFC:

3.1. Frames
Zstandard compressed data is made up of one or more frames. Each frame is independent and can be decompressed independently of other frames. The decompressed content of multiple concatenated frames is the concatenation of each frame's decompressed content.

However, APIs in the library that accept multiple frames generally also accept zero frames, for example ZSTD_decompress(). In my opinion:

• Standard compatible compressors MUST emit an empty frame for empty inputs (frame header + EOF block). Though a flag could enable the non-standard behavior of emitting zero compressed frames for empty inputs.
• Standard decoders MUST accept the empty frame + EOF block.
• Decoders MAY accept the empty compressed data to produce an empty output. This is technically non-standard behavior, but decoders are allowed to accept more than the spec allows.

[klauspost]

Seems like consensus is to disallow 0 length.

Merging PR for now. Keeping topic open for discussion.