Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjusting the scale doesn't take the shortest path #1188

Open
tnozicka opened this issue Mar 17, 2023 · 2 comments
Open

Adjusting the scale doesn't take the shortest path #1188

tnozicka opened this issue Mar 17, 2023 · 2 comments
Assignees
@zimnx
Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@tnozicka
Copy link
Member

Is this a bug report or feature request?

  • Feature Request

What should the feature do:
The operator should stop scaling up if a new desired state has less members.

What is use case behind this feature:
Say we have a scylla cluster with 3 nodes and want to adjust members to 5 but accidentally put 55 there. The controller will create 55 services which is fine but when we correct ourselves and change the members to 5, the controller will first scale up to 55 nodes and only then goes back to 5. In the worst case your infra won't be big enough to scale to 55 instances and you'll get stuck.

Additional Information:
We are creating the services ahead of time which is fine but we need to make sure we stop scaling up as soon as we can and somehow differentiate the services that never had a bootstrapped node (which may be hard).
@tnozicka tnozicka added kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Mar 17, 2023
zimnx added a commit to zimnx/scylla-operator that referenced this issue Jun 16, 2023
@zimnx
Support scaling changes during ongoing scaling
593ed6b 
Decommissioning label reconcilation was moved from StatefulSet
controller into Service controller.
StatefulSet controller can now react dynamically to scaling events
occuring while scaling operations are still ongoing.

Fixes scylladb#1188
@zimnx zimnx mentioned this issue Jun 16, 2023
Closed
zimnx added a commit to zimnx/scylla-operator that referenced this issue Jun 16, 2023
@zimnx
Support scaling changes during ongoing scaling
cf5178c 
Decommissioning label reconcilation was moved from StatefulSet
controller into Service controller.
StatefulSet controller can now react dynamically to scaling events
occuring while scaling operations are still ongoing.

Fixes scylladb#1188
zimnx added a commit to zimnx/scylla-operator that referenced this issue Jun 27, 2023
@zimnx
Support scaling changes during ongoing scaling
431e7cd 
Decommissioning label reconcilation was moved from StatefulSet
controller into Service controller.
StatefulSet controller can now react dynamically to scaling events
occuring while scaling operations are still ongoing.

Fixes scylladb#1188
@scylla-operator-bot scylla-operator-bot
Copy link
Contributor

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 30d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out

/lifecycle stale

@scylla-operator-bot scylla-operator-bot bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 24, 2024
@tnozicka
Copy link
Member Author

/remove-lifecycle stale
/triage accepted

@scylla-operator-bot scylla-operator-bot bot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zimnx zimnx

Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support scaling changes during ongoing scaling zimnx/scylla-operator
3 participants
@zimnx @tnozicka and others