Legacy v0.5 streaming decompression produces different bytes than one-shot API

[simonresch]

Describe the bug
A legacy v0.5 frame (magic 0xFD2FB525) decodes to different outputs depending on whether it is processed via ZSTD_decompress or ZSTD_decompressStream. Both APIs report success and produce the same output length, but the contents differ.

To Reproduce

• Place the attached reproducer legacy_stream_mismatch.c in tests/reproducers
• Add the following to build/cmake/tests/CMakeLists.txt

add_executable(legacy_stream_mismatch ${TESTS_DIR}/reproducers/legacy_stream_mismatch.c)
target_link_libraries(legacy_stream_mismatch libzstd_static)

Build and run:

cmake -S . -B build/repro -DZSTD_BUILD_TESTS=ON -DZSTD_LEGACY_SUPPORT=5
cmake --build build/repro --config Release --target legacy_stream_mismatch
./build/repro/build/cmake/tests/legacy_stream_mismatch

Output:

Mismatch: one_shot_size=283 stream_size=283 first_diff=14 (one_shot=0x4e, stream=0x68)

one_shot text:
NNNNNNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNNNNhNNNNNhhhhhhhhhhhh
streamed text:
NNNNNNNNNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNNNNhhhhNNhhhhhhhhhhhh

Expected behavior
Decoded bytes should be the same using both APIs.

Desktop (please complete the following information):

• OS: Linux
• Version: current working tree (1.5.8 at time of report)
• Compiler: gcc & clang

[Cyan4973]

It's a troubling issue.
If it was on the main decompression path, this would be high priority.

That being said, legacy v0.5 format is on its way out.
It's no longer produced since > 9 years (one needs to download install and use a v0.5 version of zstd from early 2016) and support for its decoder (where this bug happens) will be removed in an upcoming release.

Therefore, I'm wondering if it's worth making a lot of efforts to fix that one.
If it happens to be simple, ok.
But if it takes multiple hours or even days, that sounds like wasted time.

[Cyan4973]

The bug is fixed in #4555.

Additionally, we'll remove support of v0.5 by default in upcoming release (#4554).