Topic-aware replica placement in partition_allocator::reallocate_partition
#17704
Conversation
ztlpn
force-pushed
the
partition-balancer-topic-aware-followup
branch
from
851ad6d
to
3747e15
Compare
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47546#018ebffd-c59d-402d-ba91-ad227dcd20a3 ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47546#018ebffd-c5a0-4927-8f56-44babc7d73f8 ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47546#018ec006-b139-4a99-92eb-b91904460cd7 ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47546#018ec006-b13b-4daa-9b92-e38f576c75da |
Do all reallocations in a single invoke_on call instead of doing a cross-shard call for each partition. No semantic change.
This is better as the whole topic is updated by one controller command, preventing the situation where different partitions of a topic have different replication factors. Also, we get topic awareness of new replica placement for free.
ztlpn
force-pushed
the
partition-balancer-topic-aware-followup
branch
from
3747e15
to
2868a6c
Compare
| @@ -321,7 +321,8 @@ result<allocated_partition> partition_allocator::reallocate_partition( | |||
| partition_constraints p_constraints, | |||
| const partition_assignment& current_assignment, | |||
| const partition_allocation_domain domain, | |||
| const std::vector<model::node_id>& replicas_to_reallocate) { | |||
| const std::vector<model::node_id>& replicas_to_reallocate, | |||
| node2count_t* existing_replica_counts) { | |||
There was a problem hiding this comment.
nit: I was debating pointer vs an optional<reference_wrapper<node2count_t>>.. the latter looks really ugly I guess, sigh
There was a problem hiding this comment.
Yeah, I always feel a bit guilty about using raw pointers... OTOH I am always drawn to it by its simplicity and versatility :)
There was a problem hiding this comment.
@bharathv @ztlpn https://github.com/redpanda-data/redpanda/blob/dev/src/v/net/transport.h#L111 oh I saw this pattern being discussed on slack and tried it. passed code review :P
Add topic awareness to
partition_allocator::reallocate_partitionand enable it in the usage sites of this API. Additionally, updatehealth_managerto usetopics_frontend::do_update_topic_propertiesinstead of dispatching partition reconfigurations manually.Backports Required
Release Notes