Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow pre-allocating buffers to reduce frequent GCs during growth #18686

Merged
merged 2 commits into from
Dec 21, 2023

Conversation

harshavardhana
Copy link
Member

@harshavardhana harshavardhana commented Dec 20, 2023

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

allow pre-allocating buffers to reduce frequent GCs during growth

Motivation and Context

This PR also increases per node bpool memory from 1024 entries
to 2048 entries; along with that, it also moves the byte pool
centrally instead of being per pool.

How to test this PR?

Nothing special this is not enabled by default

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Unit tests added/updated
  • Internal documentation updated
  • Create a documentation update request here

@klauspost
Copy link
Contributor

Usually garbage collecting is the expensive part. But pre-allocating will make it seem more "stable" in terms of memory usage, so that is a reasonable bonus.

Maybe keep 1024 buffers for SNSD?

cmd/server-main.go Outdated Show resolved Hide resolved
internal/bpool/bpool.go Outdated Show resolved Hide resolved
@harshavardhana harshavardhana force-pushed the pre-allocate branch 3 times, most recently from a511a9f to 2c8c38c Compare December 20, 2023 19:37
This PR also increase per node bpool memory from 1024 entries
to 2048 entries, along with that it also moves byte pool
centrally instead of being per pool.
@harshavardhana harshavardhana changed the title allow pre-allocating buffers to avoid memory allocation overheads allow pre-allocating buffers to reduce frequent GCs during growth Dec 21, 2023
@harshavardhana harshavardhana marked this pull request as ready for review December 21, 2023 00:56
Copy link
Contributor

@klauspost klauspost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@harshavardhana harshavardhana merged commit 7c948ad into minio:master Dec 21, 2023
19 checks passed
@harshavardhana harshavardhana deleted the pre-allocate branch December 21, 2023 16:59
moting9 pushed a commit to moting9/minio that referenced this pull request Dec 25, 2023
…nio#18686)

This PR also increases per node bpool memory from 1024 entries
to 2048 entries; along with that, it also moves the byte pool
centrally instead of being per pool.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants