Please implement a streaming interface for lz4-java

[mooreb]

I suggest this interface:
https://github.com/mooreb/lz4-java-stream

which implements LZ4InputStream, LZ4OutputStream and a test that verifies the streams work.

[jpountz]

I agree this would be a good addition. However, the LZ4 format is not appropriate for streaming. For example, because the literal length must be encoded before the raw bytes, an uncompressible stream would require to buffer everything into memory before writing the length and flushing.

See http://code.google.com/p/lz4/issues/detail?id=42&can=1 for more information. Eventually, there should be a streaming format with support for error detection, but I'm waiting for the original LZ4 implementation to implement this to keep API and format compatible.

[mooreb]

Well, appropriate or not, it's implemented; please see:
https://github.com/mooreb/lz4-java-stream

You can take it or leave it. :-)

[jpountz]

Actually, I'm almost sold to add convenience classes to help people compress and decompress streams of data. I'm just a little worried LZ4 will eventually come out with a streaming format which doesn't match the one which you implemented and that might confuse users. I need to think a little more about the best way to do it. :-)

[jpountz]

Brian, I just committed LZ4BlockOutputStream and LZ4BlockInputStream in net.jpountz.lz4. Does it work for you?

[mooreb]

Hi Adrien,Wow, thanks for the quick turnaround. I'll take a look at this on Monday. I found a bug in my implementation with back-to-back flush()es. It looks like your implementation is immune to that particular bug, but I haven't looked too closely yet. Hoping this finds you well,b

-------- Original Message --------
Subject: Re: [lz4-java] Please implement a streaming interface for
lz4-java (#10)
From: Adrien Grand notifications@github.com
Date: Sat, January 26, 2013 5:16 pm
To: jpountz/lz4-java lz4-java@noreply.github.com
Cc: Brian Moore mooreb@mooreb.comBrian, I just committed LZ4BlockOutputStream and LZ4BlockInputStream in net.jpountz.lz4. Does it work for you? — Reply to this email directly or view it on GitHub.

[mooreb]

Hi Adrien,

I've (visually) reviewed the code and it looks good to me. Unfortunately I can't test it by just dropping it into the currently released jar in maven. Would you consider another release?

Thanks in advance,

b

[jpountz]

Thanks for the review! Yes, I'm working on another release, I would just like to fix #8 first.