proposal: support for snappy

[matthax]

I've done some digging through the docs, and while I've had better luck with this library than a few others but one thing I've noticed it seems to be lacking is support for snappy.

Most of the Kafka clusters I work with are using snappy by default for compression, so the ability to decompress messages coming from the reader and compress outgoing messages from the writer directly through this library would be great.

I'm just getting started with Go, so I'm not sure I'd be the best person to implement it, but I know the c headers are pretty readily available so it probably should not be terribly difficult...

Thoughts on this?

[achille-roussel]

This is a very good point, and it would be a nice feature to add.

Would you have the cycles to work on this?

[matthax]

I can give it a shot

[errnoh]

I've been using https://github.com/eapache/go-xerial-snappy Decode() for decompressing Message.Value without running into any problems.

[matthax]

@errnoh When I decode using that library I still seem to get some sort of header in the value. Any idea what might cause that or have you not experienced that problem?

[matthax]

I noticed these messages also have no Key the nbytes being read for the key is -1. Perhaps related? Not sure.

[Pryz]

Hey @matthax sorry for the late response here. See my last comment here : #100 (comment)

[achille-roussel]

@matthax snappy compression is fully supported in this package and we are using it in production. I'm going to close this issue, feel free to re-open if you have any issues getting snappy compression to work.