Add support for Zstandard compression #35
                
     Closed
            
            
          
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
While it might be very early in the stages of Zstandard's development, I'm stuck at work with nothing to do and wanted to learn C (everything is guesswork based on
grep -rHni 'lz4' .and a few peeks at zstd libraries). :)Perhaps merging could wait a while due to the early stage development of Zstandard, but I do think at this point the Zstd library functions used are unlikely to be changed. I did some testing on my uncompressed initrd (~10 MB) and some small files (~1300 b) and figured the results from the current revision of Zstd were very impressive for realtime packet compression. I might build a bigger testing environment for individual packet compression later on to better confirm my results, but for now, I give you this:
initrd:
packet:
As for the negative impact of this patch, the FSE library throws 384 extra bytes into the buffer space or so I was able to interpret from fse.c, then Zstd throws another 9 extra bytes on top of that. Yikes!
Perhaps it would be better to more dynamically get COMP_EXTRA_BUFFER(len) based on current compression method. :/ In any case, we're talking bytes so it's probably not too bad.
ex, in comp.h: