Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plugin planner for k8s #4819

Merged
merged 13 commits into from
Mar 15, 2024
Merged

Add plugin planner for k8s #4819

merged 13 commits into from
Mar 15, 2024

Conversation

ffjlabo
Copy link
Member

@ffjlabo ffjlabo commented Mar 12, 2024

What this PR does / why we need it:

This is an experimental PR for a plugin.
As a first step, I'm trying to convert the k8s planner into the k8s plugin.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

  • How are users affected by this change:
  • Is this breaking change:
  • How to migrate (if breaking change):

@ffjlabo
Copy link
Member Author

ffjlabo commented Mar 12, 2024

TODO

  • how to create gitClient
  • how to create secretDecrypter
  • fix to get lastSuccessfulConfigFilename from request parameter
  • fix to get cfg.Input.HelmChart.Insecur from request parameter
  • set version to response

@ffjlabo
Copy link
Member Author

ffjlabo commented Mar 12, 2024

We use piped config passed via grpc request. ref: #4821
Use it to do below ↓

how to create gitClient
how to create secretDecrypter
fix to get lastSuccessfulConfigFilename from request parameter
fix to get cfg.Input.HelmChart.Insecur from request parameter
#4819 (comment)

Also, the set version to response option is not fixed because the version will be deprecated.
https://github.com/pipe-cd/pipecd/blob/master/pkg/model/deployment.pb.go#L257-L258

Copy link

codecov bot commented Mar 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 28.91%. Comparing base (930487a) to head (78a070a).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4819      +/-   ##
==========================================
+ Coverage   28.88%   28.91%   +0.02%     
==========================================
  Files         317      317              
  Lines       40366    40366              
==========================================
+ Hits        11658    11670      +12     
+ Misses      27779    27767      -12     
  Partials      929      929              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ffjlabo
Copy link
Member Author

ffjlabo commented Mar 12, 2024

Current progress: Startup PlannerService as gRPC server.

% cd pkg/app/pipedv1/pluggin/applicationkind/kubernetes
% go run main.go server.go server                                                             
grpc server will be run without tls
grpc server is running on [::]:10000
handled an unary gRPC request: /grpc.pluggin.applicationkind.PlannerService/BuildPlan	{"code": "InvalidArgument", "error": "rpc error: code = InvalidArgument desc = invalid request: invalid BuildPlanRequest.WorkingDir: value length must be at least 1 runes", "duration": 0.000009417}
handled an unary gRPC request: /grpc.pluggin.applicationkind.PlannerService/BuildPlan	{"code": "InvalidArgument", "error": "rpc error: code = InvalidArgument desc = invalid request: invalid BuildPlanRequest.PipedConfig: value length must be at least 1 bytes", "duration": 0.000065375}

@ffjlabo ffjlabo force-pushed the add-plugin-planner branch 2 times, most recently from 963afa3 to d2af85b Compare March 13, 2024 15:04
@ffjlabo ffjlabo changed the title [WIP] Add plugin planner for k8s Add plugin planner for k8s Mar 14, 2024
khanhtc1202
khanhtc1202 previously approved these changes Mar 14, 2024
t-kikuc
t-kikuc previously approved these changes Mar 14, 2024
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
@ffjlabo ffjlabo merged commit ca5ef54 into master Mar 15, 2024
14 checks passed
@ffjlabo ffjlabo deleted the add-plugin-planner branch March 15, 2024 02:44
@github-actions github-actions bot mentioned this pull request Apr 8, 2024
t-kikuc pushed a commit that referenced this pull request Apr 8, 2024
* [WIP] Add planner

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Not to use out.Version

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use last_successful_commit_hash and last_successful_config_file_name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.WorkingDir

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.PipedConfig

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create git client

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create secret encryptor

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add startup server implementation

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Fix for relocation of proto api

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add roughly implementation for planner plugin

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Rename pkg name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add licence

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Comment out for the testing code

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

---------

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
t-kikuc pushed a commit that referenced this pull request Apr 8, 2024
* [WIP] Add planner

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Not to use out.Version

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use last_successful_commit_hash and last_successful_config_file_name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.WorkingDir

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.PipedConfig

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create git client

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create secret encryptor

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add startup server implementation

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Fix for relocation of proto api

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add roughly implementation for planner plugin

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Rename pkg name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add licence

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Comment out for the testing code

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

---------

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
t-kikuc added a commit that referenced this pull request Apr 8, 2024
* BUGFIX: Added cancel method to fix context leak (#4767)

Signed-off-by: fazledyn-or <ataf@openrefactory.com>

* Define piped pluggin api (#4815)

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Update BuldPlan API for piped pluggin (#4821)

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Relocate plugin proto (#4826)

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Update controller to use new planner logic (#4825)

* Update controller to use new planner logic

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Update proto path

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Fix typo

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Fix typo

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Update planner logic to call proto instead of self executing

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

---------

Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>

* Update plugin proto for ExecutorService and add piped pluginservice (#4834)

* Add plugin planner for k8s (#4819)

* [WIP] Add planner

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Not to use out.Version

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use last_successful_commit_hash and last_successful_config_file_name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.WorkingDir

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.PipedConfig

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create git client

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create secret encryptor

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add startup server implementation

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Fix for relocation of proto api

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add roughly implementation for planner plugin

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Rename pkg name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add licence

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Comment out for the testing code

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

---------

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Truncate `deploymentStatus` metrics after reporting stats (#4857)

* Truncate deploymentStatus metrics after reporting to avoid excess message size

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

* Rename func to Flush() for clarity

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

* Add comment of what's included in statsreporter's body

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

* Fix indent in the comment

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

* Copy change of metrics.go to pipedv1

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

* Copy change of reporter.go to pipedv1

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

---------

Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>

---------

Signed-off-by: fazledyn-or <ataf@openrefactory.com>
Signed-off-by: khanhtc1202 <khanhtc1202@gmail.com>
Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Signed-off-by: t-kikuc <tkikuchi07f@gmail.com>
Co-authored-by: Ataf Fazledin Ahamed <ataf@openrefactory.com>
Co-authored-by: Khanh Tran <32532742+khanhtc1202@users.noreply.github.com>
Co-authored-by: Yoshiki Fujikane <40124947+ffjlabo@users.noreply.github.com>
@t-kikuc t-kikuc mentioned this pull request Apr 9, 2024
sZma5a pushed a commit to sZma5a/pipecd that referenced this pull request Apr 18, 2024
* [WIP] Add planner

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Not to use out.Version

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use last_successful_commit_hash and last_successful_config_file_name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.WorkingDir

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Use in.PipedConfig

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create git client

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Create secret encryptor

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add startup server implementation

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Fix for relocation of proto api

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add roughly implementation for planner plugin

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Rename pkg name

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Add licence

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

* Comment out for the testing code

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>

---------

Signed-off-by: Yoshiki Fujikane <ffjlabo@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants