Tests: KCL to retry reassignment request when clashing with partition balancer #26149
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It's in the header, so it was probably intended to be implemented. With it implemented we can print any command key which is helpful for adding a debug print of all commands going through controller with their types and keys. We're probably miles away from being able to print all values.
We already classify cluster::errc::no_update_in_progress as error_code::no_reassignment_in_progress. This is to do similar for cluster::errc::update_in_progress. This error code is used in node joining, but it is handled there specifically.
We already have 3 similar handlers and looking to add one more. Separate repeated code.
partition balancer may kick in at any time, so this error should be handled gracefully
Collaborator
CI test resultstest results on build#66014
|
Contributor
Author
|
this should fix https://redpandadata.atlassian.net/browse/CORE-9958 |
bharathv
reviewed
May 21, 2025
Contributor
bharathv
left a comment
bharathv
left a comment
There was a problem hiding this comment.
on "improvements" in release notes of this PR, I thought that is only reserved for user facing improvements? This is a test only change though.
| return False | ||
|
|
||
| # A concurrent reassignment may be triggered by partition_balancer | ||
| if has_partition_err(l, "REASSIGNMENT_IN_PROGRESS"): |
Contributor
There was a problem hiding this comment.
I'm wondering if the caller should handle the retry in this case. Reassignments are typically submitted with the assumption that the original assignment is still valid. However, the presence of a REASSIGNMENT_IN_PROGRESS status suggests that the original assignment may no longer apply, so it might be more appropriate for the caller to decide how to proceed.
Contributor
Author
There was a problem hiding this comment.
I thought reassignment specifies desired state, and if a concurrent reassignment has changed something the desired state still applies. WDYT?
Contributor
Author
No, we actually change the behaviour of Redpanda by returning a different error code. I think users should be prepared to it. |
Contributor
duh! I totally missed that commit. 🙈 |
bharathv
approved these changes
Jun 2, 2025
Contributor
Author
|
/backport v25.1.x v24.3.x v24.2.x |
Contributor
Author
|
/backport v25.1.x |
Contributor
Author
|
/backport v24.3.x |
Contributor
Author
|
/backport v24.2.x |
This was referenced Jun 4, 2025
Closed
Merged
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sometimes explicitly requested partition reassignment will clash with a reassignment triggered by Partition Balancer. Retry in this case.
No backport as behaviour changes (more specific error code).
Backports Required
Release Notes
Improvements