Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cluster: Avoid oversize allocs for topic creation and configuration #16982

Merged
merged 5 commits into from
Mar 11, 2024
Merged

cluster: Avoid oversize allocs for topic creation and configuration #16982

merged 5 commits into from
Mar 11, 2024

Conversation

BenPope
Copy link
Member

@BenPope BenPope commented Mar 8, 2024 •
edited
Loading

Convert some types from std::vector to chunked_vector to avoid oversize allocations

  • topic_configuration (392 bytes)
  • custom_assignable_topic_configuration (416 bytes)
  • topic_properties_update (456 bytes)

Fixes #16758

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x

Release Notes

Improvements

  • cluster: Avoid oversize allocs for topic creation and configuration

@BenPope BenPope requested a review from ballard26 March 8, 2024 20:28
@BenPope BenPope self-assigned this Mar 8, 2024
@BenPope
cluster/utils: Refactor make_error_topic_results
f524df7 
A future PR will use `chunked_vector` in addition to `std::vector`

Signed-off-by: Ben Pope <ben@redpanda.com>
@BenPope
cluster: Drive-by optimisation
12daecc 
Signed-off-by: Ben Pope <ben@redpanda.com>
@BenPope
cluster: Add some using declarations for vectors of types
99247c5 
Signed-off-by: Ben Pope <ben@redpanda.com>
@BenPope
cluster: Avoid oversize alloc for topic configuration
d58c9fa 
`topic_configuration` is 392 bytes
`custom_assignable_topic_configuration`	is 416 bytes

Switch vectors of them to `chunked_vector`

Fixes redpanda-data#16758

Signed-off-by: Ben Pope <ben@redpanda.com>
@BenPope
cluster: Avoid oversize alloc for topic properties update`
9191f39 
`topic_properties_update` is 456 bytes.

Switch it to from `std::vector` to `chunked_vector`.

Signed-off-by: Ben Pope <ben@redpanda.com>
@vbotbuildovich
Copy link
Collaborator

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/45907#018e2043-af83-41ad-b5e3-1e0a57f7c7f4

@piyushredpanda piyushredpanda merged commit a488dbe into redpanda-data:dev Mar 11, 2024
17 checks passed
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

@vbotbuildovich
Copy link
Collaborator

Failed to create a backport PR to v23.3.x branch. I tried:

git remote add upstream https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-16982-v23.3.x-145 remotes/upstream/v23.3.x
git cherry-pick -x f524df7dc255f65e3bb0e9d4d2805fea25d1d76a 12daecc409f885d046c7a47ed85b3a5b81907e8f 99247c562d9b0f263debb1678496902c996d845e d58c9fa751f676a8fa969f88040b9af95ff90530 9191f391b09909a32bb679abbc1a803a0d54de18

Workflow run logs.

@vbotbuildovich vbotbuildovich mentioned this pull request Mar 11, 2024
Closed
@BenPope BenPope mentioned this pull request Mar 12, 2024
Merged
@BenPope BenPope modified the milestone: v23.3.8 Mar 15, 2024
@BenPope BenPope mentioned this pull request Mar 15, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@StephanDollberg StephanDollberg StephanDollberg approved these changes

@ballard26 ballard26 Awaiting requested review from ballard26

Assignees

@BenPope BenPope

Labels
area/oversized-allocation area/redpanda
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Oversized allocation: 409600 bytes in kafka::create_topics_api
4 participants
@BenPope @vbotbuildovich @StephanDollberg @piyushredpanda