Skip to content

Commit

Permalink
zstd: Add better compression mode (#240)
Browse files Browse the repository at this point in the history
Add compression mode that is better than default.

Approximately 50% of the throughput with ~5% overall improvement. Roughly the same as zstd level 7-8.

Some comparisons (best of 3 runs):

```
file	out	level	insize	outsize	millis	mb/s
enwik9	zskp	2	1000000000	317517832	7452	127.96
enwik9	zskp	3	1000000000	295162043	13341	71.48

nyc-taxi-data-10M.csv	zskp	2	3325605752	595200765	14299	221.80
nyc-taxi-data-10M.csv	zskp	3	3325605752	548223444	27764	114.23

github-june-2days-2019.json	zskp	2	6273951764	640407088	14517	412.15
github-june-2days-2019.json	zskp	3	6273951764	604145629	35487	168.60

rawstudio-mint14.tar	zskp	2	8558382592	3379874208	42571	191.72
rawstudio-mint14.tar	zskp	3	8558382592	3287441065	90137	90.55

silesia.tar	zskp	2	211947520	67542426	1254	161.15
silesia.tar	zskp	3	211947520	64657881	2135	94.65

gob-stream	zskp	2	1911399616	207819726	4573	398.57
gob-stream	zskp	3	1911399616	193227707	8663	210.39
```

Enabled as `zstd.SpeedBetterCompression` level.
  • Loading branch information
klauspost committed Mar 9, 2020
1 parent 4e45121 commit 56999ed
Show file tree
Hide file tree
Showing 6 changed files with 559 additions and 93 deletions.
Loading

0 comments on commit 56999ed

Please sign in to comment.