Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brotli: implement encoder #42

Open
jpillora opened this issue Feb 5, 2016 · 13 comments

Comments

@jpillora
Copy link

commented Feb 5, 2016

Please close this if/when brotli encoder is implemented. Nice work btw!

@dsnet dsnet added the enhancement label Feb 5, 2016

@dsnet dsnet self-assigned this Feb 5, 2016

@dsnet

This comment has been minimized.

Copy link
Owner

commented Feb 5, 2016

Thanks for the issue. I'll definitely try to get a working brotli encoder released soon. I'll give fair warning that the first cut might not perform much better than DEFLATE in terms of it's compression ratio. I plan on doing a iterative approach to implementing brotli, where each iteration improves it's compression performance.

@jpillora

This comment has been minimized.

Copy link
Author

commented Feb 5, 2016

Cool that sounds fine to me +1
On Sat, 6 Feb 2016 at 7:05 AM Joe Tsai notifications@github.com wrote:

Thanks for the issue. I'll definitely try to get a working brotli encoder
released soon. I'll give fair warning that the first cut might not perform
much better than DEFLATE in terms of it's compression ratio. I plan on
doing a iterative approach to implementing brotli, where each iteration
improves it's compression performance.


Reply to this email directly or view it on GitHub
#42 (comment).

@dsnet

This comment has been minimized.

Copy link
Owner

commented Feb 7, 2016

Placing this here for reference: caddyserver/caddy#525

@akyoto

This comment has been minimized.

Copy link

commented Nov 26, 2017

Has this been abandoned or is there any chance somebody is still working on a native implementation?

@dsnet

This comment has been minimized.

Copy link
Owner

commented Nov 29, 2017

I wouldn't say "abandoned", but I'm not actively working on it at the present moment. I have a number of other projects currently occupying my time.

@akyoto

This comment has been minimized.

Copy link

commented Nov 29, 2017

I see, thank you for the reply!
Would have loved to see a native implementation in Go but I realize it's a very time-consuming task.

@dmitshur

This comment has been minimized.

Copy link

commented Jan 18, 2018

@dsnet In caddyserver/caddy#525 (comment), you said:

I'm actually currently working on finishing my own implementation of a RFC 1951 DEFLATE compressor. The components built to implement DEFLATE will make implementing Brotli significantly easier.

I had a question about that. Given that a pure Go DEFLATE decompressor already exists in Go standard library (compress/flate), what's the reason you're building your own? Is it because you have different objectives compared to the std lib one, or another reason?

Can the components of compress/flate compressor be used to help implement Brotli compressor, or is there a reason why compress/flate isn't helpful for that?

Just looking to understand this better, thanks.

@dsnet

This comment has been minimized.

Copy link
Owner

commented Jan 18, 2018

Obviously, the lack of traffic on this repo implies haven't done much in that regard 😞

The reason are the following:

  • I already implemented huffman encoding (since it's the same for bzip2 and brotli), so the only other component left for DEFLATE is just LZ77. Implementing LZ77 would allow me to experiment with different algorithms and understand it better.
  • There are low level functionality that I would want to be able to do on DEFLATE that I don't think they standard library should ever support. See the zlib manual for what features they provide.
@akyoto

This comment has been minimized.

Copy link

commented Mar 21, 2018

@dsnet I know absolutely nothing about brotli internals but I have a decent knowledge of Go. Is there a chance for someone like me to help you?

@dsnet

This comment has been minimized.

Copy link
Owner

commented Mar 21, 2018

@blitzprog, I appreciate your enthusiasm, but implementation compression algorithms really does require specialized knowledge. You're welcome to do research on how LZ77 works and various approaches to compute it.

@faradaytrs

This comment has been minimized.

Copy link

commented Jun 12, 2019

Any progress on this?

@dmitshur

This comment has been minimized.

Copy link

commented Aug 12, 2019

@dsnet FYI, I was made aware (thanks to @mappu) that there is a pure Go implementation of a Brotli encoder that was created using a c2go-based approach by @andybalholm:

https://groups.google.com/forum/#!msg/golang-nuts/J5MwJnmDpOw/yQacKG2pBwAJ

I haven't had a chance to try it yet, but I wanted to share it with you (and others here).

@dsnet

This comment has been minimized.

Copy link
Owner

commented Aug 19, 2019

Thanks for the update @dmitshur! I've been too busy with my actual work to work on this implementation. I'm glad someone else is providing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.