feature: support helm chart #668

jrkeen · 2021-08-27T09:09:41Z

What type of PR is this?
/kind feature

What this PR does / why we need it:
Support deploy by helm. #652

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

karmada-bot · 2021-08-27T09:09:50Z

Welcome @jrkeen! It looks like this is your first PR to karmada-io/karmada 🎉

pidb · 2021-08-27T09:30:45Z

cc @RainbowMango

RainbowMango

Thanks!!! I'll review and test it ASAP!
Also, cc @lfbear @mrlihanbo who might interest in it.

artifacts/chart/Chart.yaml

pigletfly

The karmada installation using helm should support two scenarios: with karmada-apiserver and without.And also should take cert-manager for CA injection into consideration.

artifacts/chart/_crds/cluster.karmada.io_clusters.yaml

pigletfly · 2021-08-28T14:44:28Z

artifacts/chart/templates/_karmada_webhook_configuration.tpl

+apiVersion: admissionregistration.k8s.io/v1
+kind: MutatingWebhookConfiguration
+metadata:
+  name: mutating-config


should we provide a option if user uses cert-manager for CA injection ?

More options will be provided in the future.

pigletfly · 2021-08-28T14:45:22Z

artifacts/chart/templates/etcd.yaml

@@ -0,0 +1,127 @@
+{{- if and (eq .Values.etcd.mode "internal") (eq .Values.installMode "host")}}
+apiVersion: apps/v1


if users are not using karmada-apiserver,then etcd should be optional.

artifacts/chart/templates/karmada_agent.yaml

artifacts/chart/templates/karmada_apiserver.yaml

RainbowMango · 2021-08-30T03:51:52Z

@jrkeen
Can you give some descriptions about how to use the chart? (For other reviewers to try it on).

[edit]:
When I tried to install the chart on my kind cluster, I met an error Error: Chart.yaml file is missing

# helm install karmada -n karmada-system --create-namespace .
Error: Chart.yaml file is missing
-bash-4.2# pwd
/root/go/src/github.com/karmada-io/karmada
-bash-4.2# kubectl config get-contexts 
CURRENT   NAME            CLUSTER         AUTHINFO        NAMESPACE
*         kind-helmdemo   kind-helmdemo   kind-helmdemo   
-bash-4.2#

Did I miss something?

jrkeen · 2021-08-30T07:29:15Z

@jrkeen
Can you give some descriptions about how to use the chart? (For other reviewers to try it on).

[edit]:
When I tried to install the chart on my kind cluster, I met an error Error: Chart.yaml file is missing
# helm install karmada -n karmada-system --create-namespace .
Error: Chart.yaml file is missing
-bash-4.2# pwd
/root/go/src/github.com/karmada-io/karmada
-bash-4.2# kubectl config get-contexts 
CURRENT   NAME            CLUSTER         AUTHINFO        NAMESPACE
*         kind-helmdemo   kind-helmdemo   kind-helmdemo   
-bash-4.2# 
Did I miss something?

First of all, edit values as your needs.

## use host mode.
installMode: "host"
certs:
  ## @param certs.mode "auto" and "custom" are provided
  ## "auto" means auto generate certificate
  ## "custom" means use user certificate
  mode: auto
  auto:
    ## @param certs.auto.expiry expiry of the certificate
    expiry: 43800h
    ## @param certs.auto.hosts hosts of the certificate
    hosts: [
      "kubernetes.default.svc",
      "*.etcd.karmada-system.svc.cluster.local",
      "*.karmada-system.svc.cluster.local",
      "*.karmada-system.svc",
      "localhost",
      "127.0.0.1"
    ]
  custom:
    ## server-ca.crt
    caCrt: |
      -----BEGIN CERTIFICATE-----
      xxxxx
      -----END CERTIFICATE-----
    ## karmada.crt
    crt: |
      -----BEGIN CERTIFICATE-----
      xxxxx
      -----END CERTIFICATE-----
    ## karmada.key
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      xxxxx
      -----END RSA PRIVATE KEY-----
-----------------------------------------------------------------
## use agent mode.
installMode: "agent"
agent:
  clusterName: "cluster name for your member cluster"
  kubeconfig:
    ## server-ca.crt
    caCrt: |
      -----BEGIN CERTIFICATE-----
      xxxxx
      -----END CERTIFICATE-----
    ## karmada.crt
    crt: |
      -----BEGIN CERTIFICATE-----
      xxxxx
      -----END CERTIFICATE-----
    ## karmada.key
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      xxxxx
      -----END RSA PRIVATE KEY-----
    ## karmada apiserver
    server: "https://karmada-apiserver.karmada-system.svc.cluster.local"

Then, execute the installation command.

# cd /root/go/src/github.com/karmada-io/karmada/artifacts/chart
# helm install karmada/karmada-agent -n karmada-system --create-namespace .

Tip: Manually generate certificates.

# openssl req -x509 -sha256 -new -nodes -days 365 -newkey rsa:2048 -keyout "server-ca.key" -out "server-ca.crt" -subj "/C=xx/ST=x/L=x/O=x/OU=x/CN=ca/emailAddress=x/"
# echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","client auth","server auth"]}}}' > "server-ca-config.json"
# echo '{"CN":"system:admin","hosts":["kubernetes.default.svc","*.etcd.karmada-system.svc.cluster.local","*.karmada-system.svc.cluster.local","*.karmada-system.svc","localhost","127.0.0.1"],"names":[{"O":"system:masters"}],"key":{"algo":"rsa","size":2048}}' | cfssl gencert -ca=server-ca.crt -ca-key=server-ca.key -config=server-ca-config.json - | cfssljson -bare karmada
# mv "karmada-key.pem" "karmada.key"
# mv "karmada.pem" "karmada.crt"

artifacts/chart/Chart.yaml

RainbowMango · 2021-09-06T06:56:33Z

@jrkeen What's the progress now?
Can we install the chart by helm install karmada -n karmada-system --create-namespace . now?

jrkeen · 2021-09-06T07:01:59Z

@jrkeen What's the progress now?
Can we install the chart by helm install karmada -n karmada-system --create-namespace . now?

Yes, try it. #668 (comment)

RainbowMango · 2021-09-06T08:00:32Z

Does this chart support a kind cluster? Or should I install it under a sub-directory of the repo?

# kubectl config get-contexts 
CURRENT   NAME       CLUSTER         AUTHINFO        NAMESPACE
*         helmdemo   kind-helmdemo   kind-helmdemo
# pwd
/root/go/src/github.com/karmada-io/karmada
# helm install karmada -n karmada-system --create-namespace .
Error: Chart.yaml file is missing
# helm version
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}

jrkeen · 2021-09-06T08:08:06Z

Does this chart support a kind cluster? Or should I install it under a sub-directory of the repo?

# kubectl config get-contexts 
CURRENT   NAME       CLUSTER         AUTHINFO        NAMESPACE
*         helmdemo   kind-helmdemo   kind-helmdemo
# pwd
/root/go/src/github.com/karmada-io/karmada
# helm install karmada -n karmada-system --create-namespace .
Error: Chart.yaml file is missing
# helm version
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}

# cd /root/go/src/github.com/karmada-io/karmada/artifacts/chart
# helm install karmada -n karmada-system --create-namespace .

or

# helm install karmada -n karmada-system --create-namespace /root/go/src/github.com/karmada-io/karmada/artifacts/chart

jrkeen · 2021-09-06T09:52:23Z

cc @RainbowMango

RainbowMango

Generally LGTM. Only some nits.

In addition:

Please sync the crd from master again, since we just updated a little bit.
Please tidy your commits.

Thanks~~

RainbowMango · 2021-09-07T02:31:26Z

charts/README.md

+$ helm install karmada -n karmada-system --create-namespace .
+```
+
+If the certificate is `automatically generated`, need to get kubeconfig from the cluster:


What do you mean If? I don't see any description of customizing the certificate.

It's my mistake, both modes can be used to obtain kubeconfig in this way, I need to modify the doc. By the way, users can edit values as needs.

RainbowMango · 2021-09-07T02:38:30Z

charts/README.md

+To install the chart with the release name `karmada` in namespace `karmada-system`:
+
+```console
+$ helm install karmada -n karmada-system --create-namespace .


Suggested change

$ helm install karmada -n karmada-system --create-namespace .

$ helm install karmada -n karmada-system --create-namespace ./charts

Does this make sense?

charts/README.md

Signed-off-by: jrkeen <jrkeen@hotmail.com>

jrkeen · 2021-09-07T03:47:10Z

All done @RainbowMango

RainbowMango · 2021-09-07T13:23:43Z

LGTM. @mrlihanbo @lfbear Would you like to take a look?

lfbear · 2021-09-08T03:03:15Z

I am not good at the helm, I just wonder how to keep in sync the files in charts/_crds/ and charts/templates/ with files in artifacts/deploy, that can not be processed by manual.

RainbowMango · 2021-09-08T03:10:17Z

I am not good at the helm, I just wonder how to keep in sync the files in charts/_crds/ and charts/templates/ with files in artifacts/deploy, that can not be processed by manual.

Good question. I'll be done by following PR.
The CRDs should be generated and verified automatically.

RainbowMango · 2021-09-08T09:25:16Z

/lgtm
/approve

RainbowMango · 2021-09-08T09:27:54Z

Many thanks to @algebra2k @jrkeen and all of the guys who are reviewing this PR.

The chart means a lot to the project and It might be not so perfect yet, but it will be a good start to iterate it.

karmada-bot · 2021-09-08T09:28:09Z

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

karmada-bot added the kind/feature Categorizes issue or PR as related to a new feature. label Aug 27, 2021

karmada-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 27, 2021

jrkeen force-pushed the feat/support_helm_chart branch from 174cf09 to dcb49f1 Compare August 27, 2021 09:31

RainbowMango reviewed Aug 27, 2021

View reviewed changes

artifacts/chart/Chart.yaml Outdated Show resolved Hide resolved

pigletfly suggested changes Aug 28, 2021

View reviewed changes

RainbowMango reviewed Aug 30, 2021

View reviewed changes

artifacts/chart/Chart.yaml Outdated Show resolved Hide resolved

RainbowMango reviewed Sep 7, 2021

View reviewed changes

support helm chart

f679846

Signed-off-by: jrkeen <jrkeen@hotmail.com>

jrkeen force-pushed the feat/support_helm_chart branch from e082a6c to f679846 Compare September 7, 2021 03:41

karmada-bot assigned RainbowMango Sep 8, 2021

karmada-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 8, 2021

RainbowMango added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 8, 2021

karmada-bot merged commit 552f957 into karmada-io:master Sep 8, 2021

RainbowMango mentioned this pull request Sep 9, 2021

makes sure crds in chart always up to date and re-org the crds. #712

Merged

RainbowMango mentioned this pull request Sep 14, 2021

support for karmada TODO list installed by helm #652

Closed

12 tasks

lfbear mentioned this pull request Oct 26, 2021

karmada will support distribution of helm afterwards？ #861

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: support helm chart #668

feature: support helm chart #668

jrkeen commented Aug 27, 2021

karmada-bot commented Aug 27, 2021

pidb commented Aug 27, 2021

RainbowMango left a comment

pigletfly left a comment

pigletfly Aug 28, 2021

jrkeen Aug 30, 2021

pigletfly Aug 28, 2021

RainbowMango commented Aug 30, 2021 •

edited

jrkeen commented Aug 30, 2021 •

edited

RainbowMango commented Sep 6, 2021

jrkeen commented Sep 6, 2021

RainbowMango commented Sep 6, 2021

jrkeen commented Sep 6, 2021

jrkeen commented Sep 6, 2021

RainbowMango left a comment

RainbowMango Sep 7, 2021

jrkeen Sep 7, 2021

RainbowMango Sep 7, 2021

jrkeen commented Sep 7, 2021

RainbowMango commented Sep 7, 2021

lfbear commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

karmada-bot commented Sep 8, 2021

		@@ -0,0 +1,127 @@
		{{- if and (eq .Values.etcd.mode "internal") (eq .Values.installMode "host")}}
		apiVersion: apps/v1

	$ helm install karmada -n karmada-system --create-namespace .
	$ helm install karmada -n karmada-system --create-namespace ./charts

feature: support helm chart #668

feature: support helm chart #668

Conversation

jrkeen commented Aug 27, 2021

karmada-bot commented Aug 27, 2021

pidb commented Aug 27, 2021

RainbowMango left a comment

Choose a reason for hiding this comment

pigletfly left a comment

Choose a reason for hiding this comment

pigletfly Aug 28, 2021

Choose a reason for hiding this comment

jrkeen Aug 30, 2021

Choose a reason for hiding this comment

pigletfly Aug 28, 2021

Choose a reason for hiding this comment

RainbowMango commented Aug 30, 2021 • edited

jrkeen commented Aug 30, 2021 • edited

RainbowMango commented Sep 6, 2021

jrkeen commented Sep 6, 2021

RainbowMango commented Sep 6, 2021

jrkeen commented Sep 6, 2021

jrkeen commented Sep 6, 2021

RainbowMango left a comment

Choose a reason for hiding this comment

RainbowMango Sep 7, 2021

Choose a reason for hiding this comment

jrkeen Sep 7, 2021

Choose a reason for hiding this comment

RainbowMango Sep 7, 2021

Choose a reason for hiding this comment

jrkeen commented Sep 7, 2021

RainbowMango commented Sep 7, 2021

lfbear commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

RainbowMango commented Sep 8, 2021

karmada-bot commented Sep 8, 2021

RainbowMango commented Aug 30, 2021 •

edited

jrkeen commented Aug 30, 2021 •

edited