Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for LZ4 to System.Compression #17714

Closed
redknightlois opened this issue Jun 27, 2016 · 4 comments
Closed

Add support for LZ4 to System.Compression #17714

redknightlois opened this issue Jun 27, 2016 · 4 comments
Labels
area-System.IO.Compression enhancement Product code improvement that does NOT require public API changes/additions help wanted [up-for-grabs] Good issue for external contributors
Milestone

Comments

@redknightlois
Copy link

As per requested in https://github.com/dotnet/corefx/issues/9657 I am adding a new one.

LZ4 is by far the fastest compression algorithm out there (at the expense of compression ratio that-is). It makes sense to have such an algorithm available on the baseline release.

We can also provide our port of our managed LZ4 implementation of the r131 (which includes acceleration) if that makes the official inclusion of an algorithm we heavily rely upon easier. We are still working on the perf front but it is currently faster than the one that we had https://github.com/Corvalius/ravendb/blob/lz4-131/src/Sparrow/Compression/LZ4.cs

@ianhays ianhays self-assigned this Jun 27, 2016
@GSPP
Copy link

GSPP commented Aug 11, 2016

From what I could quickly find on the web it seems that indeed LZ4 is the fastest and has the best compression in it's speed category.

@redknightlois
Copy link
Author

@GSPP yes and the latest build also allows you to tune that with the acceleration parameter. You can try the port we did of the latest one from this code: https://github.com/Corvalius/ravendb/blob/v4.0/src/Sparrow/Compression/LZ4.cs

It is perf tuned but not that much... On Gen2 i5 and 1333Mhz we are memory bound. I still have a pass to be done in a few months on an Skylake DDR4, 3200Mhz memory to see if there is place for improvement. But as I mentioned before, we would gladly donate the implementation if anyone is interested into do the work to include that here.

@ianhays ianhays removed their assignment Oct 10, 2016
@ianhays
Copy link
Contributor

ianhays commented Oct 10, 2016

Next step here is to provide an API suggestion to get discussion moving. LZ4 support is a reasonable suggestion and something we would benefit from having.

@karelz
Copy link
Member

karelz commented Dec 16, 2019

Triage: This is sitting here for a long time, but no interest. It seems it is not worth tracking it at this moment.
Looks like larger work anyway.

@karelz karelz closed this as completed Dec 16, 2019
@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 5.0 milestone Jan 31, 2020
@dotnet dotnet locked as resolved and limited conversation to collaborators Dec 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.IO.Compression enhancement Product code improvement that does NOT require public API changes/additions help wanted [up-for-grabs] Good issue for external contributors
Projects
None yet
Development

No branches or pull requests

5 participants