Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LZ4 Legacy support? #99

Closed
Caesurus opened this issue Oct 19, 2020 · 7 comments
Closed

LZ4 Legacy support? #99

Caesurus opened this issue Oct 19, 2020 · 7 comments

Comments

@Caesurus
Copy link

I noticed that the package doesn't have legacy support
There is this legacy magic value here: https://github.com/lz4/lz4/blob/dev/programs/lz4io.c#L80

Is there any plan to support this?

@pierrec
Copy link
Owner

pierrec commented Oct 20, 2020

There is no plan to support the legacy format at this moment.
Feel free to research it and send a PR if you need it though :).

@Caesurus
Copy link
Author

Thanks :) I'm trying to use this project to extract a compressed linux kernel. Turns out that various samples I have are compressed with legacy LZ4.

$ hexdump -n 32 vmlinux_LZ4_19377.lz4
0000000 02 21 4c 18 2d 80 39 00 84 7f 45 4c 46 02 01 01
0000010 00 01 00 43 02 00 3e 00 0e 00 01 07 00 13 40 18

Not knowing anything about lz4 I started digging and it seems that even the lz4 repo has support for it in a standalone application but not in the library itself. https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#legacy-frame

If I do get some spare cycles I may try to figure out how to add support and provide a PR.
Thanks again.

@Caesurus
Copy link
Author

Sorry for commenting/reopening a closed issue.

Had some time this morning to hack away at some stuff. Got it implemented here:
https://github.com/Caesurus/lz4/blob/master/reader_legacy.go
at the moment there is still a unit test failing (partial read after seek): https://github.com/Caesurus/lz4/blob/master/reader_legacy_test.go#L118

It's extracting my legacy sample kernel correctly. I'll do clean up and do some more testing, then I'll send a PR your way (if you're still interested).

I decided against trying to integrate it into the reader as is, I didn't want to impact performance with unneeded operations and making the code more complex. I also don't have a writer for it since I'm not interested in creating legacy lz4 compressed files. Any comments or recommendations are always welcome

@Caesurus Caesurus reopened this Oct 21, 2020
@pierrec
Copy link
Owner

pierrec commented Oct 21, 2020

Thanks. Definitely interested. Once you are done, I will add it to the v4 branch as well, this time using a functional option for it.

@Caesurus
Copy link
Author

Yay, I have the PR open. I love go... but don't love what I need to do to correct imports when merging back from a forked project :(

Thanks for the excellent work on this project BTW, and making it available! Much much appreciated 👍

@pierrec
Copy link
Owner

pierrec commented Nov 1, 2020

PR merged and legacy support also added on v4.

@pierrec pierrec closed this as completed Nov 1, 2020
@anatol
Copy link
Contributor

anatol commented Mar 16, 2021

And I am trying to write an initramfs in lz4 format anatol/booster#34

As it was mentioned earlier kernel need the legacy format. Does anybody have a plan to implement the legacy writer?

anatol added a commit to anatol/lz4 that referenced this issue Mar 17, 2021
Legacy lz4 format is still used by Linux kernel.

Issue pierrec#99
anatol added a commit to anatol/lz4 that referenced this issue Mar 17, 2021
Legacy lz4 format is still used by Linux kernel.

See the format description here
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md

Issue pierrec#99
anatol added a commit to anatol/lz4 that referenced this issue Mar 17, 2021
Legacy lz4 format is still used by Linux kernel.

See the format description here
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md

Issue pierrec#99
anatol added a commit to anatol/lz4 that referenced this issue Mar 18, 2021
Legacy lz4 format is still used by Linux kernel.

See the format description here
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md

Issue pierrec#99
anatol added a commit to anatol/lz4 that referenced this issue Mar 18, 2021
Legacy lz4 format is still used by Linux kernel.

See the format description here
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md

Issue pierrec#99
anatol added a commit to anatol/lz4 that referenced this issue Mar 18, 2021
See the format description here
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md

Legacy lz4 format is still used by Linux kernel. I was able to verify
this legacy writer functionality by creating an lz4 compressed initramfs
and booting it at 5.11.7 Linux kernel.

Issue pierrec#99
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants