Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c/partition_balancer: don't copy topic metadata on each partition visit #17413

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

ztlpn
Copy link
Contributor

@ztlpn ztlpn commented Mar 26, 2024

Previously we were copying the whole metadata (including partition assignments) just to query the topic revision.

partition_balancer_bench results before:

test                                                   iterations      median         mad         min         max      allocs       tasks        inst
partition_balancer_planner_fixture.unavailable_nodes           27    24.481ms   161.139us    24.320ms    35.441ms 1235710.104      93.852 425407989.4
partition_balancer_planner_fixture.counts_rebalancing           1      1.857s     7.421ms      1.846s      1.888s 118912546.400    7424.600 33427470288.6

and after:

test                                                   iterations      median         mad         min         max      allocs       tasks        inst
partition_balancer_planner_fixture.unavailable_nodes           91     8.695ms     1.429ms     5.078ms    10.124ms   68660.031      11.301 100617054.8
partition_balancer_planner_fixture.counts_rebalancing          19    54.006ms    56.923us    53.942ms    55.873ms 2172589.168     208.305 972473623.6

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x

Release Notes

  • none

Previously we were copying the whole metadata (including partition
assignments) just to query the topic revision.
Copy link
Member

@dotnwat dotnwat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1s vs 55ms is not bad!

Copy link
Contributor

@bharathv bharathv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yikes, thanks for fixing it, I believe I added that snippet of code.

@vbotbuildovich
Copy link
Collaborator

@ztlpn ztlpn merged commit 764752b into redpanda-data:dev Mar 27, 2024
18 of 19 checks passed
@ztlpn ztlpn deleted the partition-balancer-md-ref branch March 27, 2024 01:20
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants