Decompression fails where no content size is included in the frame (e.g. streaming)

[starrify]

Examples for reproducing the issue:

$ man zstd > tmpmanual
$ zstd tmpmanual --content-size -o with-size.zst
tmpmanual            : 34.74%   ( 26286 =>   9133 bytes, with-size.zst)        
$ zstd tmpmanual --no-content-size -o no-size.zst
tmpmanual            : 34.74%   ( 26286 =>   9132 bytes, no-size.zst)          
$ man zstd | zstd -o no-size-2.zst
/*stdin*\            : 34.70%   ( 26286 =>   9120 bytes, no-size-2.zst)        
$ python -c 'import zstd; zstd.uncompress(open("with-size.zst", "rb").read())'
<string>:1: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
$ python -c 'import zstd; zstd.uncompress(open("no-size.zst", "rb").read())'
<string>:1: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
Traceback (most recent call last):
  File "<string>", line 1, in <module>
zstd.Error: Input data invalid or missing content size in frame header.
$ python -c 'import zstd; zstd.uncompress(open("no-size-2.zst", "rb").read())'
<string>:1: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
Traceback (most recent call last):
  File "<string>", line 1, in <module>
zstd.Error: Input data invalid or missing content size in frame header.

Test environment:

$ uname -a
Linux GLaDOS 5.8.10-arch1-1 #1 SMP PREEMPT Thu, 17 Sep 2020 18:01:06 +0000 x86_64 GNU/Linux
$ python --version
Python 3.8.5
$ python -c 'import zstd; print(zstd.version())'
1.4.5.1
$ zstd --version
*** zstd command line interface 64-bits v1.4.5, by Yann Collet ***

Proposal: Enhance the API so that it may work also for data without the content size field embedded.

By the way, the other Python-binding library python-zstandard also fails for the same reason when using its simple decompression API (zstandard.ZstdDecompressor().decompress()), but its other APIs (e.g. streaming API) may handle properly the case when there is no content size.

[sergey-dryabzhinsky]

Yes, this module is simple and dumb.
It never meant to support streaming compression.
And I'll keep it this way.