-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question regarding Lz4net archived repo #83
Comments
So this is touching relatively common problem. Let me start with some analogy. Imagine two systems which need to exchange a lot of numbers. The problem is one is producing a text file with numbers separated by commas and the other one is expecting them to be separated by tabs. There is a BLOCK format and STREAM format. A STREAM contains a lot of BLOCKs. STREAM has stream header, block header, BLOCK, block header, BLOCK, block header, BLOCK, etc... Both stream and block headers are quite small (let's say 8 bytes), while block carry data and are large (64K - 4MB). Make sense?
Stream header carries information what is the overall length of the stream, what compression method was used, what was default block size, etc. Block header say something about block, how many bytes it actually had before compression, and how many bytes after compression, etc. And then there is a BLOCK of compressed LZ4 data. BLOCK format in lz4net is absolutely the same any any other implementation of LZ4, but STREAM format is not. It is custom wrapping of LZ4 blocks. You did not give me enough information about how lz4net is used, and lz4net had already two APIs: BLOCK and (custom) STREAM. If it expecting just BLOCK then it is almost trivial. If it expects STREAM you would need to implement your own code for headers (~8 bytes) but you can copy blocks without any modifications. You will need to implement this file in Rust: (If you need to implement more it means you are doing something wrong) Also it will be even less if Rust just need to write or just read, as you need to implement only half of this custom stream handling. |
I assume there are no further questions. |
Anyway, it should be simple and not too much code. If you case about performance of C# Looking at this: it seems you need: DO NOT use: |
Hi, thank you so much for your help!
Thank you very much for your time and effort! Best Regards |
Hi,
some legacy software I try to stay compatible with uses the archived version of lz4net.
Now I am trying to write a rust program which compresses data with the lz4 crate which will be then decompressed with that legacy software.
There is no way to update the depencency on lz4net so I have to adapt my rust code accordingly.
Is there documentation what exact settings lz4net uses for compression and decompression so that I can adapt to that?
Best regards and thanks in advance
Tarcontar
The text was updated successfully, but these errors were encountered: