Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use chunked vector as batches cache in raft::replicate_batcher #17425

Merged
merged 2 commits into from
Mar 28, 2024
Merged

use chunked vector as batches cache in raft::replicate_batcher #17425

merged 2 commits into from
Mar 28, 2024

Conversation

mmaslankaprv
Copy link
Member

@mmaslankaprv mmaslankaprv commented Mar 27, 2024 •
edited

Using fragmented_vector to store batches in raft::replicate_batcher to avoid large allocations.

Fixes: #16944
Fixes: #16945

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x

Release Notes

  • none

rockwotj
rockwotj previously approved these changes Mar 27, 2024
View reviewed changes
src/v/raft/replicate_batcher.cc Outdated Show resolved Hide resolved
@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Mar 27, 2024 •
edited

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46873#018e7fd3-329f-464a-985e-388cdb85f863

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46891#018e80b3-c3ae-48e6-bd18-7161dcb60714

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46891#018e80b8-0dd0-4b63-965a-bc5f8580ce36

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46974#018e8441-a532-4185-9243-90513daa6cf9

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46974#018e8441-a529-4657-8bc2-356d7eadb897

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46980#018e84c9-4cf0-4737-af0a-ad22d3893964

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/46980#018e84c9-4cf3-4942-a05a-5e0484f516d4

rockwotj
rockwotj previously approved these changes Mar 27, 2024
View reviewed changes
rockwotj
rockwotj previously approved these changes Mar 28, 2024
View reviewed changes
src/v/model/record_batch_reader.h Outdated Show resolved Hide resolved
@mmaslankaprv
model/record_batch_reader: made fragmented memory reader generic
9212eed 
Made all fragmented memory reader helper functions generic so that they
can be used with fragmented vector of different fragment sizes.

Signed-off-by: Michał Maślanka <michal@redpanda.com>
@mmaslankaprv
r/replicate_batcher: use chunked vector as batches cache
6b880ec 
Using `fragmented_vector` to store batches in `raft::replicate_batcher`
to avoid large allocations.

Fixes: redpanda-data#16944
Fixes: redpanda-data#16945

Signed-off-by: Michał Maślanka <michal@redpanda.com>
@mmaslankaprv
Copy link
Member Author

ci failure: #12897

@mmaslankaprv mmaslankaprv merged commit 952b8c9 into redpanda-data:dev Mar 28, 2024
17 checks passed
@mmaslankaprv mmaslankaprv deleted the fix-batcher-large-alloc branch March 28, 2024 20:06
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

@vbotbuildovich
Copy link
Collaborator

Failed to create a backport PR to v23.3.x branch. I tried:

git remote add upstream https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-17425-v23.3.x-124 remotes/upstream/v23.3.x
git cherry-pick -x 9212eed8c8b903fef3f115bfce1575443b402103 6b880ec89e9097cefc69a755ffe0f41611c5db51

Workflow run logs.

@vbotbuildovich vbotbuildovich mentioned this pull request Mar 28, 2024
Closed
@mmaslankaprv mmaslankaprv mentioned this pull request Apr 3, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rockwotj rockwotj rockwotj approved these changes

@bharathv bharathv Awaiting requested review from bharathv

@ztlpn ztlpn Awaiting requested review from ztlpn

@StephanDollberg StephanDollberg Awaiting requested review from StephanDollberg

@dotnwat dotnwat Awaiting requested review from dotnwat

Assignees
No one assigned
Labels
area/redpanda
Projects
None yet
Milestone
No milestone
4 participants
@mmaslankaprv @vbotbuildovich @dotnwat @rockwotj