Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kafka: Limit Scaling to Partitions w/Lag #5060

Merged
merged 13 commits into from
Oct 11, 2023
Merged

Kafka: Limit Scaling to Partitions w/Lag #5060

merged 13 commits into from
Oct 11, 2023

Conversation

BojanZelic
Copy link
Contributor

@BojanZelic BojanZelic commented Oct 6, 2023 •
edited

Added ability to configure both Kafka & Apache Kafka (experimental) scalers to limit the max number of replicas to the number of partitions with lag;

Checklist

Fixes #3997

Relates to #4453

DmitrySenin and others added 8 commits April 13, 2023 11:37
@DmitrySenin
Limit the max replicas to the number of partitions with lag
e5d8427 
Signed-off-by: Dmitrii Senin <white.lord.mane@gmail.com>
@DmitrySenin
Extra tests for parsing limitToPartitionsWithLag
12d8203 
Signed-off-by: Dmitrii Senin <white.lord.mane@gmail.com>
@DmitrySenin
Check for topic when the limit is enabled
b78e7d4 
Signed-off-by: Dmitrii Senin <white.lord.mane@gmail.com>
@DmitrySenin
Changelog updated
48ba6ae 
Signed-off-by: Dmitrii Senin <white.lord.mane@gmail.com>
@BojanZelic
Merge tag 'v2.12.0' of https://github.com/kedacore/keda into kafka_ac…
1bcb2ac 
…tive_partitions
@BojanZelic
Kafka: add e2e tests
29e6329 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@BojanZelic
Apache Kafka: add limitToPartitionsWithLag param
9aed2f0 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@BojanZelic
update changelog
4dfc08b 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@BojanZelic BojanZelic requested a review from a team as a code owner October 6, 2023 21:55
@github-actions
Copy link

github-actions bot commented Oct 6, 2023

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

Learn more about:

@BojanZelic
linting
4f9e1c9 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@BojanZelic
Merge branch 'main' of https://github.com/kedacore/keda into kafka_ac…
657ea5a 
…tive_partitions
@BojanZelic
add to changelog
b2b721c 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
zroubalik
zroubalik reviewed Oct 10, 2023
View reviewed changes
Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

CHANGELOG.md Outdated Show resolved Hide resolved
@zroubalik
Copy link
Member

zroubalik commented Oct 10, 2023 •
edited by github-actions bot

/run-e2e kafka
Update: You can check the progress here

@BojanZelic
rm Experimental changelog for kafka
aef7c8d 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@BojanZelic
Merge branch 'main' into kafka_active_partitions
ff8ef97 
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
@zroubalik zroubalik enabled auto-merge (squash) October 10, 2023 16:58
@zroubalik
Copy link
Member

zroubalik commented Oct 10, 2023 •
edited by github-actions bot

/run-e2e kafka
Update: You can check the progress here

@zroubalik
Copy link
Member

@BojanZelic please fix the documentation PR as well 🙏

@BojanZelic
Copy link
Contributor Author

Thanks, should be good now 🤞

@zroubalik
Copy link
Member

Thanks for the contribution!

@zroubalik zroubalik merged commit 9a49dff into kedacore:main Oct 11, 2023
19 checks passed
@zroubalik zroubalik mentioned this pull request Dec 21, 2023
Closed
4 tasks
toniiiik pushed a commit to toniiiik/keda that referenced this pull request Jan 15, 2024
@BojanZelic @DmitrySenin @toniiiik
Kafka: Limit Scaling to Partitions w/Lag (kedacore#5060)
ab2a76e 
Signed-off-by: Dmitrii Senin <white.lord.mane@gmail.com>
Signed-off-by: Bojan Zelic <bnzelic@gmail.com>
Co-authored-by: Dmitrii Senin <white.lord.mane@gmail.com>
Signed-off-by: anton.lysina <alysina@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zroubalik zroubalik zroubalik approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Kafka scaler: limit number of replicas to the number of partitions with lag.
3 participants
@BojanZelic @zroubalik @DmitrySenin