Upgrade DataDog/zstd to v1.5.0

[dferstay]

Motivation

This updates zstd to 1.5.0 which supports some major performance
improvements. For details see:
https://github.com/facebook/zstd/releases/tag/v1.5.0

Results from pulsar/internal/compression/compression_bench_test.go show the speedup.

name                                          old time/op    new time/op     delta
CompressionParallel/zstd-cgo-level-fastest      1.30ms ±13%     0.95ms ± 0%  -27.09%  (p=0.000 n=9+8)
CompressionParallel/zstd-cgo-level-fastest-2     672µs ± 2%      517µs ± 4%  -23.12%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-4     421µs ± 6%      323µs ± 3%  -23.10%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-8     241µs ± 1%      187µs ± 1%  -22.25%  (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-default      1.34ms ±15%     1.00ms ± 2%  -25.02%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-2     676µs ± 1%      525µs ± 1%  -22.30%  (p=0.000 n=10+9)
CompressionParallel/zstd-cgo-level-default-4     389µs ± 1%      317µs ± 5%  -18.39%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-8     245µs ±11%      210µs ± 2%  -14.36%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best         1.29ms ± 2%     0.97ms ± 0%  -24.63%  (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-best-2        684µs ± 1%      528µs ± 2%  -22.75%  (p=0.000 n=8+10)
CompressionParallel/zstd-cgo-level-best-4        434µs ± 4%      323µs ± 2%  -25.64%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best-8        260µs ± 7%      211µs ± 1%  -19.05%  (p=0.000 n=10+9)

name                                          old speed      new speed       delta
CompressionParallel/zstd-cgo-level-fastest    77.4MB/s ±12%  105.9MB/s ± 0%  +36.85%  (p=0.000 n=9+8)
CompressionParallel/zstd-cgo-level-fastest-2   150MB/s ± 2%    195MB/s ± 4%  +30.11%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-4   240MB/s ± 6%    311MB/s ± 3%  +29.93%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-8   418MB/s ± 1%    537MB/s ± 1%  +28.62%  (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-default    75.6MB/s ±14%  100.4MB/s ± 2%  +32.82%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-2   149MB/s ± 1%    191MB/s ± 1%  +28.71%  (p=0.000 n=10+9)
CompressionParallel/zstd-cgo-level-default-4   259MB/s ± 1%    318MB/s ± 5%  +22.66%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-8   411MB/s ±10%    479MB/s ± 2%  +16.39%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best       77.8MB/s ± 2%  103.2MB/s ± 0%  +32.66%  (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-best-2      147MB/s ± 1%    190MB/s ± 2%  +29.47%  (p=0.000 n=8+10)
CompressionParallel/zstd-cgo-level-best-4      232MB/s ± 4%    312MB/s ± 2%  +34.44%  (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best-8      388MB/s ± 8%    478MB/s ± 1%  +23.27%  (p=0.000 n=10+9)

name                                  old time/op    new time/op     delta
Compression/zstd-cgo-level-fastest       369µs ± 0%      371µs ± 0%   +0.65%  (p=0.000 n=8+9)
Compression/zstd-cgo-level-fastest-2     369µs ± 0%      370µs ± 1%     ~     (p=0.796 n=9+9)
Compression/zstd-cgo-level-fastest-4     369µs ± 0%      373µs ± 3%     ~     (p=0.101 n=8+10)
Compression/zstd-cgo-level-fastest-8     371µs ± 2%      374µs ± 2%     ~     (p=0.053 n=9+10)
Compression/zstd-cgo-level-default      1.28ms ± 0%     0.96ms ± 2%  -25.24%  (p=0.000 n=9+10)
Compression/zstd-cgo-level-default-2    1.29ms ± 2%     0.96ms ± 2%  -26.05%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-4    1.30ms ± 2%     0.96ms ± 1%  -25.88%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-8    1.28ms ± 3%     0.96ms ± 1%  -24.63%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-best         3.90ms ± 0%     2.32ms ± 1%  -40.39%  (p=0.000 n=10+9)
Compression/zstd-cgo-level-best-2       3.90ms ± 0%     2.36ms ± 0%  -39.63%  (p=0.000 n=9+8)
Compression/zstd-cgo-level-best-4       3.94ms ± 2%     2.33ms ± 1%  -40.88%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-best-8       3.97ms ± 2%     2.36ms ± 0%  -40.59%  (p=0.000 n=10+9)

name                                  old speed      new speed       delta
Compression/zstd-cgo-level-fastest     273MB/s ± 0%    271MB/s ± 0%   -0.64%  (p=0.000 n=8+9)
Compression/zstd-cgo-level-fastest-2   272MB/s ± 0%    272MB/s ± 1%     ~     (p=0.779 n=9+9)
Compression/zstd-cgo-level-fastest-4   272MB/s ± 0%    270MB/s ± 3%     ~     (p=0.101 n=8+10)
Compression/zstd-cgo-level-fastest-8   271MB/s ± 2%    269MB/s ± 2%     ~     (p=0.053 n=9+10)
Compression/zstd-cgo-level-default    78.7MB/s ± 0%  105.2MB/s ± 2%  +33.77%  (p=0.000 n=9+10)
Compression/zstd-cgo-level-default-2  77.8MB/s ± 2%  105.1MB/s ± 2%  +35.21%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-4  77.7MB/s ± 2%  104.8MB/s ± 1%  +34.91%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-8  78.9MB/s ± 3%  104.7MB/s ± 1%  +32.66%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-best       25.8MB/s ± 0%   43.3MB/s ± 1%  +67.77%  (p=0.000 n=10+9)
Compression/zstd-cgo-level-best-2     25.8MB/s ± 0%   42.7MB/s ± 0%  +65.63%  (p=0.000 n=9+8)
Compression/zstd-cgo-level-best-4     25.5MB/s ± 2%   43.1MB/s ± 1%  +69.12%  (p=0.000 n=10+10)
Compression/zstd-cgo-level-best-8     25.4MB/s ± 2%   42.7MB/s ± 0%  +68.30%  (p=0.000 n=10+9)

name                                    old time/op   new time/op    delta
Decompression/zstd-cgo-level-fastest      121µs ± 1%     111µs ±20%     ~     (p=0.133 n=9+10)
Decompression/zstd-cgo-level-fastest-2    119µs ±10%      98µs ± 6%  -17.77%  (p=0.000 n=9+10)
Decompression/zstd-cgo-level-fastest-4    108µs ± 2%      96µs ± 6%  -10.93%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-fastest-8    113µs ± 5%     103µs ± 4%   -9.03%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default      155µs ± 7%     135µs ± 4%  -12.99%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default-2    203µs ±51%     119µs ± 0%  -41.63%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-4    138µs ± 2%     122µs ± 2%  -11.63%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-8    143µs ± 1%     127µs ± 2%  -11.16%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-best         137µs ± 1%     125µs ± 0%   -8.98%  (p=0.000 n=9+8)
Decompression/zstd-cgo-level-best-2       139µs ±15%     122µs ±13%  -12.42%  (p=0.034 n=8+10)
Decompression/zstd-cgo-level-best-4       124µs ± 1%     110µs ± 2%  -11.14%  (p=0.000 n=9+10)
Decompression/zstd-cgo-level-best-8       129µs ± 3%     114µs ± 1%  -11.17%  (p=0.000 n=10+8)

name                                    old speed     new speed      delta
Decompression/zstd-cgo-level-fastest    832MB/s ± 1%   914MB/s ±17%     ~     (p=0.133 n=9+10)
Decompression/zstd-cgo-level-fastest-2  848MB/s ±11%  1029MB/s ± 6%  +21.29%  (p=0.000 n=9+10)
Decompression/zstd-cgo-level-fastest-4  930MB/s ± 2%  1044MB/s ± 6%  +12.36%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-fastest-8  889MB/s ± 4%   977MB/s ± 4%   +9.91%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default    652MB/s ± 7%   748MB/s ± 5%  +14.83%  (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default-2  557MB/s ±41%   849MB/s ± 0%  +52.42%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-4  730MB/s ± 2%   826MB/s ± 2%  +13.16%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-8  705MB/s ± 1%   794MB/s ± 2%  +12.56%  (p=0.000 n=10+9)
Decompression/zstd-cgo-level-best       734MB/s ± 1%   806MB/s ± 0%   +9.87%  (p=0.000 n=9+8)
Decompression/zstd-cgo-level-best-2     660MB/s ±45%   815MB/s ±12%  +23.45%  (p=0.017 n=10+9)
Decompression/zstd-cgo-level-best-4     811MB/s ± 1%   912MB/s ± 2%  +12.54%  (p=0.000 n=9+10)
Decompression/zstd-cgo-level-best-8     782MB/s ± 3%   880MB/s ± 1%  +12.56%  (p=0.000 n=10+8)

Modifications

The version of DataDog/zstd has been updated to tag v1.5.0

Verifying this change

This change is already covered by existing tests, such as:.

• pulsar/internal/compression/compression_test.go
• pulsar/internal/compression/compression_bench_test.go

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): Yes, it moves to a newer version of github.com/DataDog/zstd
• The public API: No
• The schema: No
• The default values of configurations: No
• The wire protocol: No

Documentation

• Does this pull request introduce a new feature? No

[wolfstudy]

Nice work