Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add topic replication factor change #9483

Merged
merged 19 commits into from
Mar 11, 2024
Merged

Conversation

fvaleri
Copy link
Contributor

@fvaleri fvaleri commented Dec 21, 2023

This change implements the proposal about topic replication factor change.
https://github.com/strimzi/proposals/blob/main/066-topic-replication-factor-change.md

Documentation will be added in a separate PR.

To avoid making the review of new components harder, there is a small amount of duplication related to Cruise Control client. This will be addressed in a follow up PR, where I plan to move the shared part between Rebalance and Topic operator in the operator-common module, and to switch from Vertx HTTP client to Java HTTP client.

Unit tests are included, but I also performed a number of manual tests on Minikube and OpenShift that could help in testing the feature, and creating system tests:

  • Deleting secret $CLUSTER_NAME-cruise-control-api should roll CC
  • Deleting secret $CLUSTER_NAME-entity-topic-operator-cc-api should roll both EO and CC
  • Setting more replicas than available brokers should report an error in .status.replicasChange.message and reverting the change should clear that status
  • Setting .spec.config."min.insync.replicas" < .spec.replicas should work logging a warning
  • Bulk replicas change should take a reasonable amount of time (tested with 100 topics with 100MB each, and it took about 5-8 minutes on my local test environment)
  • An ongoing replicas change should be recovered after TO crash
  • An ongoing replicas change should be recovered after CC crash

@fvaleri fvaleri marked this pull request as draft December 21, 2023 17:04
@scholzj scholzj removed their request for review December 21, 2023 17:46
@fvaleri fvaleri force-pushed the topic-rf-change branch 2 times, most recently from ecbc282 to 916f3c3 Compare December 21, 2023 18:08
@fvaleri fvaleri changed the title Add topic replication factor change feature Add topic replication factor change Dec 23, 2023
@fvaleri fvaleri force-pushed the topic-rf-change branch 2 times, most recently from 1bff12a to 5f6b083 Compare December 29, 2023 08:44
@fvaleri fvaleri force-pushed the topic-rf-change branch 7 times, most recently from d5f205b to c0c23ae Compare January 15, 2024 18:14
@fvaleri fvaleri force-pushed the topic-rf-change branch 5 times, most recently from 5bd15c0 to 4bcccd1 Compare January 22, 2024 15:17
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

CHANGELOG.md Outdated Show resolved Hide resolved
This change implements the proposal about topic replication factor change.

Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
@scholzj scholzj merged commit f7e7789 into strimzi:main Mar 11, 2024
13 checks passed
@fvaleri fvaleri deleted the topic-rf-change branch March 11, 2024 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants