Replace lz4 with small-lz4

[trufae]

The official lz4 implementation is unnecessarily complex and hard to read, not to mention it segfaults when compiled with tinycc, probably because it depends on lots of tricky ifdefs. The algorithm is pretty simple, so it would be better if r2 ships a smaller, more readable and maintainable version of it. and the only option out there is this one:

https://create.stephan-brumme.com/smallz4/#git1

We can keep supporting system-wide lz4 dependency, or even expose the same public api, but right now, the tcc testsuite is disabled because the lz4 impl overflows

[trufae]

http://fastcompression.blogspot.com/2011/05/lz4-explained.html

[MewtR]

Is this issue still relevant? The problem with smallz4 is that it assumes that your input is a full LZ4 Frame. For example, if I try using it on the NSO binary from the testbins repository, it fails because it can't find the magic bytes.

Also, I was able to compile things with tcc and didn't face any issues. Can you provide a specific example of something that fails when compiling with tcc?

[trufae]

i would say that is probably better to ship our own implementation of lz4, the current one is about 2500LOC which is not a lot, and most of it is wrongly indented and ifdefed code that we can probably cleanup. this code also contains asserts which are against the r2 developer guidelines. But small LZ4 is about 400 LOC and it probably fits better in r2 for our custom implementation. the less lines to maintain the better imho

If its just the magic header check we can probably skip the check and see if it doesnt breaks anything. not sure if we have many tests for that nso file 🤔

[trufae]

I think we are good to get rid of the non small implementation @MewtR do you want to go finish the task? Thanks!

[MewtR]

Apologies for the lengthy absence. I'm on it.

[trufae]

we can close this now :) thank you!! @MewtR