Skip to content

Conversation

@jcadduono
Copy link

While it might be very early in the stages of Zstandard's development, I'm stuck at work with nothing to do and wanted to learn C (everything is guesswork based on grep -rHni 'lz4' . and a few peeks at zstd libraries). :)
Perhaps merging could wait a while due to the early stage development of Zstandard, but I do think at this point the Zstd library functions used are unlikely to be changed. I did some testing on my uncompressed initrd (~10 MB) and some small files (~1300 b) and figured the results from the current revision of Zstd were very impressive for realtime packet compression. I might build a bigger testing environment for individual packet compression later on to better confirm my results, but for now, I give you this:

initrd:

unc:    : -         @ 10395136 b
lz4-1   : 0.026s    @  6299540 b
snappy  : 0.031s    @  6287148 b
lz4-9   : 0.270s    @  5261604 b
gz1     : 0.156s    @  4978336 b
zstd    : 0.037s    @  4969289 b
gz9     : 1.600s    @  4590729 b
xz1     : 0.706s    @  3693364 b
xz9e    : 3.851s    @  3248256 b

packet:

unc     : -         @ 1032 b
lz4-1   : 0.001s    @  609 b
lz4-9   : 0.001s    @  586 b
snappy  : 0.000s    @  579 b
xz1     : 0.001s    @  520 b
xz9e    : 0.014s    @  512 b
zstd    : 0.001s    @  506 b
gz1     : 0.001s    @  485 b
gz9     : 0.001s    @  462 b

As for the negative impact of this patch, the FSE library throws 384 extra bytes into the buffer space or so I was able to interpret from fse.c, then Zstd throws another 9 extra bytes on top of that. Yikes!
Perhaps it would be better to more dynamically get COMP_EXTRA_BUFFER(len) based on current compression method. :/ In any case, we're talking bytes so it's probably not too bad.
ex, in comp.h:

 * Zstandard: len + len/128 + 512 + 12  (ZSTD_compressBound(len))
 */
#define COMP_EXTRA_BUFFER(len) ((len)/8 + 512 + 12 + COMP_PREFIX_LEN)

@cron2
Copy link
Contributor

cron2 commented Feb 15, 2016

Hi,

after a bit of discussion in the OpenVPN team, we have decided that we do not want to support a larger number of compression algorithms, and have removed snappy as well - so, for the time being, we only support LZO (for compatibility) and LZ4 (for modern deployments).

gert

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants