New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confused explanation of buffer.Reader #38

tv42 opened this Issue Oct 23, 2018 · 2 comments


None yet
2 participants

tv42 commented Oct 23, 2018

Reader is a wrapper around a []byte that implements the io.Reader interface. It is a much thinner layer than bytes.Buffer provides and is therefore faster.

But bytes.Buffer does so much more. Your comparison should be to bytes.Reader!

The only meaningful difference between your implementation and bytes.Reader is that your implementation has a Bytes method. The bodies of Read methods are close enough to identical that there's likely no reason to not use the stdlib one. The extra methods on bytes.Reader shouldn't hurt.

bytes.Reader doesn't have Bytes, but you can extract the bytes from it by with WriteTo.

You could just switch to bytes.Reader, or at least fix the text to not talk about bytes.Buffer.

@tdewolff tdewolff closed this in 5472461 Oct 23, 2018


This comment has been minimized.


tdewolff commented Oct 23, 2018

Yes, the problem was mainly memory allocation, as bytes.Buffer is relatively large (though bytes.Reader is much better). It is actually a relic from the old buffered lexer and I think this reader can be trimmed down even further. Specifically, the parse and minify libraries accept io.Reader interfaces, but in all internal cases we really only want to pass a byte slice and not allocate a separate reader for that.

So either we switch to NewLexerBytes at some places, or this reader is amended to be something along these lines:

type Reader []byte

func (r Reader) Read(b []byte) (n int, err error) {
    panic("not implemented")

func (r Reader) Bytes() []byte {
    return r

This way the reader is allocated on the stack, with only the Bytes method available. Need to check if parse and minify really don't use Read, Len or Reset.

@tdewolff tdewolff reopened this Oct 23, 2018


This comment has been minimized.


tdewolff commented Oct 24, 2018

What I proposed cannot be done without changing the interfaces in minify to accept []byte instead of io.Reader. This would actually be a good step, since the first thing all minifiers do is convert the reader to a byte slice. Additionally, memory allocations performed by creating readers from byte slices (to pass to minifiers) are quite significant, so there is a fairly good win on performance as well.

@tdewolff tdewolff referenced this issue Nov 8, 2018


Planned improvements #44

0 of 8 tasks complete

@tdewolff tdewolff closed this Nov 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment