Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ACTION-TRACKING] auto-scaling of self-managed ha clusters #251

Open
dipankardas011 opened this issue Dec 5, 2023 · 1 comment · Fixed by #304
Open

[ACTION-TRACKING] auto-scaling of self-managed ha clusters #251

dipankardas011 opened this issue Dec 5, 2023 · 1 comment · Fixed by #304
Assignees
@dipankardas011 @prateek041
Labels
kind/feature triage/accepted
Milestone
ksctl event drive...

Comments

@dipankardas011
Copy link
Member

dipankardas011 commented Dec 5, 2023 •
edited
Loading

Is your feature request related to a problem?📖

it lacks auto-scaling of nodes based on resources usage. ksctl provides manual way to scale the cluster up or down

Functionality Refer
Scale up https://github.com/kubesimplify/ksctl/blob/a89d76382ed576725cd23979520f958028ed0df8/pkg/controllers/manager.go#L452
Scale Down https://github.com/kubesimplify/ksctl/blob/a89d76382ed576725cd23979520f958028ed0df8/pkg/controllers/manager.go#L510

Describe the solution you'd like 👨‍💻👩‍💻

  1. Add support for the helm and k8s client
  2. determine what all state info needs to be available for the ksctl scaller service running in the cluster
  3. create a custom app which helps k8s cluster runningksctl -scaller to talk with the ksctl controller server running on a different environment
  4. develop the ksctl scaller app
  5. k8s manifest development
  6. determine according to which metrics we can tell when to scale up and down the workload

Sub-Goals

Rough architecture diagram

Note
not to scale

image

some new discussion
image

Provider in question

  • Civo
  • Local
  • Azure
  • K3s
  • Kubeadm

Additional context

Check Contribution's guidelines
@dipankardas011
Copy link
Member Author

some of the reference for working on the custom controller

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
https://www.kubecost.com/kubernetes-autoscaling/kubernetes-cluster-autoscaler/
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

@dipankardas011 dipankardas011 mentioned this issue Feb 12, 2024
Closed
This was referenced Mar 9, 2024
Merged
Merged
@dipankardas011 dipankardas011 linked a pull request Mar 13, 2024 that will close this issue
feature: support for ksctl controllers and ksctl agent #304
Merged
3 tasks
@dipankardas011 dipankardas011 mentioned this issue Apr 1, 2024
Merged
16 tasks
@dipankardas011 dipankardas011 mentioned this issue Apr 6, 2024
Merged
3 tasks
@dipankardas011 dipankardas011 changed the title feature: auto-scaling of self-managed ha clusters [FEATURE-TRACKING] auto-scaling of self-managed ha clusters Apr 11, 2024
@dipankardas011 dipankardas011 changed the title [FEATURE-TRACKING] auto-scaling of self-managed ha clusters [ACTION-TRACKING] auto-scaling of self-managed ha clusters Apr 11, 2024
@dipankardas011 dipankardas011 mentioned this issue Apr 11, 2024
Open
4 tasks
@dipankardas011 dipankardas011 added triage/accepted and removed needs-triage help wanted Extra attention is needed labels Apr 11, 2024
dipankardas011 added a commit that referenced this issue Apr 29, 2024
@dipankardas011
sub-feature: ksctl agent (#322)
243e649 
# Tasks description

This hosts as a parent pr for:
- #325
- #327

It also added implementation for

| Kind | Group | Version |
|--------|--------|--------|
| importStates | storage.ksctl.com | v1alpha1 |
| stacks | application.ksctl.com | v1alpha1 | 

Ksctl agent for the handle of requests from various controllers

Using this you can achieve the application creation, deletion, version
changes seemlessly with only the use of crd (currently single app
support)

Improved the Makefile and the testing of the core via the adding
solution for #316

## Issues
### Completed Issue(s)

### Related Issue(s)
- #251
- #304 (base-feature)
- Closes #316


# Solution

### Sub-Tasks
- [x] Ksctl agent planning
- [x] Document all the things in
ksctl/enhancements#1
- [x] ksctl grpc proto file
- [x] Ksctl storage export
- [x] Ksctl storage import
- [x] Test cases
- [x] controller for the storage export to the newly/already created
ksctl or non-ksctl(Currently not supported) cluster
- [x] controller for the application
  - [x] creation
  - [x] deletion
  - [x] store the application information in stateDocuemnt
  - [x] updating the version based on current state of the crd
- [x] improve the logging for the kind and helm client
- [x] Add support for the incluster kubernetes environment (kube go
client, helm, storageDriver)


# Note to reviewers

> [!CAUTION]
> make sure you are fixing the URL for the controllers present to more
appropriate url. Make sure it is stable and not just main branch

- [x] Ran Tests locally
- [x] Checked [Contribution's
guidelines](https://docs.ksctl.com/docs/contribution-guidelines/)
@dipankardas011 dipankardas011 mentioned this issue Apr 29, 2024
Closed
4 tasks
@dipankardas011 dipankardas011 reopened this May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dipankardas011 dipankardas011

@prateek041 prateek041

Labels
kind/feature triage/accepted
Projects
None yet
Milestone
ksctl event driven cluster configuration
Development

Successfully merging a pull request may close this issue.

feature: support for ksctl controllers and ksctl agent ksctl/ksctl
2 participants
@dipankardas011 @prateek041