Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: autoScalerController implementation #1

Merged
merged 6 commits into from
May 1, 2024
Merged

feature: autoScalerController implementation #1

merged 6 commits into from
May 1, 2024

Conversation

dipankardas011
Copy link
Member

@dipankardas011 dipankardas011 commented Mar 9, 2024 •
edited
Loading

Tasks description

This PR will add all the design decisions here
and all the implementations related to it will be done on ksctl/ksctl/main/ksctl-components folder

Issues

Related Issue(s)

Solution

Sub-Tasks

Note to reviewers

@dipankardas011 dipankardas011 requested review from prateek041, a team, saiyam1814 and Horiodino and removed request for a team, prateek041 and saiyam1814 March 9, 2024 06:41
@dipankardas011 dipankardas011 mentioned this pull request Mar 9, 2024
Closed
3 tasks
@dipankardas011 dipankardas011 changed the title Add autoScalerController implementation feature: autoScalerController implementation Mar 9, 2024
@dipankardas011
Copy link
Member Author

Note

date: 9-3-2024

image

@dipankardas011
Copy link
Member Author

Updates

  • I have updated the location of the idea.md to seperate and new document
  • also added a design document which needs to be reviewed!

@prateek041 @dipankardas011
Add autoScalerController implementation
d78315b 
Signed-off-by: prateek041 <prateeksingh9741@gmail.com>

feat: added the new design documents and updated diagram

Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
@dipankardas011
Copy link
Member Author

Also I am replacing this PR #301 in ksctl/ksctl to a new PR so that we can use the ksctl branch instead of folk

@dipankardas011
Copy link
Member Author

also do add a note or warning that we should not use a api key of user instead a restricted apikey just for security reasons

@dipankardas011 dipankardas011 mentioned this pull request Mar 13, 2024
Merged
3 tasks
@dipankardas011
Copy link
Member Author

dipankardas011 commented Mar 13, 2024 •
edited
Loading

Question ❓

Should we make the ksctl agent to do k8s client call to fetch the configmap or we can tell the controller 2 for geting the data from the configmap and create a grpc call to the ksctl agent

But I am worrying about that controller has to wait for the response of the ksctl agent when using the second method

and if we are having the agent to use configmap then whenever it completes it can update the info

what do you say about that

@dipankardas011
Copy link
Member Author

dipankardas011 commented Mar 15, 2024 •
edited
Loading

Question ❓

Should we make the ksctl agent to do k8s client call to fetch the configmap or we can tell the controller 2 for geting the data from the configmap and create a grpc call to the ksctl agent

But I am worrying about that controller has to wait for the response of the ksctl agent when using the second method

and if we are having the agent to use configmap then whenever it completes it can update the info

what do you say about that

currently I have implemented this method of operation for ksctl agent
the ksctl agent will be a deployment with serviceAccount which has all RBAC access for configmaps for it to fetch the state of the ksctl and write as well and the tokens for cloud provider will be passed by the environment variable same goes for the storage driver

Note

by this controller 2 only has to tell the ksctl agent what is the desired workernodes and it will attach and write that back to the state.json configmap
this making the roles of the ksctlagent clearnly seperatble

@dipankardas011
Copy link
Member Author

dipankardas011 commented Mar 15, 2024 •
edited
Loading

@Horiodino
Copy link

Question ❓

Should we make the ksctl agent to do k8s client call to fetch the configmap or we can tell the controller 2 for geting the data from the configmap and create a grpc call to the ksctl agent
But I am worrying about that controller has to wait for the response of the ksctl agent when using the second method
and if we are having the agent to use configmap then whenever it completes it can update the info
what do you say about that

currently I have implemented this method of operation for ksctl agent the ksctl agent will be a deployment with serviceAccount which has all RBAC access for configmaps for it to fetch the state of the ksctl and write as well and the tokens for cloud provider will be passed by the environment variable same goes for the storage driver

Note

by this controller 2 only has to tell the ksctl agent what is the desired workernodes and it will attach and write that back to the state.json configmap this making the roles of the ksctlagent clearnly seperatble

the first option is more reliable as the agent is doing all the work, ksctl agent can autonomously manage its own configuration data!

@dipankardas011
added ksctl agent iter0 design
e4134c6 
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
@dipankardas011 dipankardas011 mentioned this pull request Apr 11, 2024
Closed
@dipankardas011
updated the poc
fce0e86 
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
@dipankardas011 dipankardas011 mentioned this pull request Apr 28, 2024
Merged
16 tasks
@dipankardas011
added for now need to review the documentation
58ca369 
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
dipankardas011 added a commit to ksctl/ksctl that referenced this pull request Apr 29, 2024
@dipankardas011
sub-feature: ksctl agent (#322)
243e649 
# Tasks description

This hosts as a parent pr for:
- #325
- #327

It also added implementation for

| Kind | Group | Version |
|--------|--------|--------|
| importStates | storage.ksctl.com | v1alpha1 |
| stacks | application.ksctl.com | v1alpha1 | 

Ksctl agent for the handle of requests from various controllers

Using this you can achieve the application creation, deletion, version
changes seemlessly with only the use of crd (currently single app
support)

Improved the Makefile and the testing of the core via the adding
solution for #316

## Issues
### Completed Issue(s)

### Related Issue(s)
- #251
- #304 (base-feature)
- Closes #316


# Solution

### Sub-Tasks
- [x] Ksctl agent planning
- [x] Document all the things in
ksctl/enhancements#1
- [x] ksctl grpc proto file
- [x] Ksctl storage export
- [x] Ksctl storage import
- [x] Test cases
- [x] controller for the storage export to the newly/already created
ksctl or non-ksctl(Currently not supported) cluster
- [x] controller for the application
  - [x] creation
  - [x] deletion
  - [x] store the application information in stateDocuemnt
  - [x] updating the version based on current state of the crd
- [x] improve the logging for the kind and helm client
- [x] Add support for the incluster kubernetes environment (kube go
client, helm, storageDriver)


# Note to reviewers

> [!CAUTION]
> make sure you are fixing the URL for the controllers present to more
appropriate url. Make sure it is stable and not just main branch

- [x] Ran Tests locally
- [x] Checked [Contribution's
guidelines](https://docs.ksctl.com/docs/contribution-guidelines/)
@dipankardas011 dipankardas011 mentioned this pull request Apr 29, 2024
Closed
4 tasks
@dipankardas011
feat: improved the documentation
e794e46 
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
@dipankardas011
feat: completely updated the docs
3229957 
Signed-off-by: Dipankar Das <65275144+dipankardas011@users.noreply.github.com>
@dipankardas011
Copy link
Member Author

Thanks everyone @Horiodino @prateek041 I am merging this PR

@dipankardas011 dipankardas011 merged commit 3ee8d4a into main May 1, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@dipankardas011 dipankardas011

@prateek041 prateek041

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dipankardas011 @Horiodino @prateek041