Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release entryBufferPool once #5324

Merged
merged 2 commits into from
Feb 4, 2022

Conversation

owen-d
Copy link
Member

@owen-d owen-d commented Feb 4, 2022

While investigating an issue, I found a bug where we can put the *entryBuffer back into the pool twice. This PR ensures we only release it once.

This PR also improves efficiency a bit by ensuring we don't nil the underlying slice unnecessarily, instead letting the pool reuse it.

This can happen when:
First, we put it into the pool here:

func (i *reverseEntryIterator) Next() bool {
	i.load()
	if len(i.buf.entries) == 0 {
		entryBufferPool.Put(i.buf)
		i.buf.entries = nil
		return false
	}
	i.cur, i.buf.entries = i.buf.entries[len(i.buf.entries)-1], i.buf.entries[:len(i.buf.entries)-1]
	return true
}

then it gets Get()'d by another goroutine, has it's entries appended, then the original goroutine puts it into the pool a second time here:

func (i *reverseEntryIterator) Close() error {
	if i.buf.entries != nil {
		i.buf.entries = i.buf.entries[:0]
		entryBufferPool.Put(i.buf)
		i.buf.entries = nil
	}
	if !i.loaded {
		return i.iter.Close()
	}
	return nil
}

@owen-d owen-d requested a review from a team as a code owner February 4, 2022 21:12
if i.buf.entries != nil {
// preserve the underlying slice before releasing to pool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is a bit weird. Should we just reslice when we get ?

Copy link
Member Author

@owen-d owen-d Feb 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do this after Get'ing, yeah -- I believe that has equivalent results. The important part is that we don't nil it here because the reason we're using a pool is to preserve the underlying slice so we don't have to reallocate it. Does that make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I would just re-slice entries are never nil.

Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cyriltovena
Copy link
Contributor

Good catch btw.

We definitively lacking a test on close, after iteration finished.

@owen-d owen-d merged commit f598484 into grafana:main Feb 4, 2022
dannykopping pushed a commit to dannykopping/loki that referenced this pull request Feb 7, 2022
* release entryBufferPool once

* reverseEntryIterator.Next is safe to call past expiry
dannykopping pushed a commit that referenced this pull request Feb 7, 2022
* release entryBufferPool once

* reverseEntryIterator.Next is safe to call past expiry

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants