karmada will support distribution of helm afterwards？ #861

learner0810 · 2021-10-25T08:23:30Z

What would you like to be added:

Why is this needed:
karmada is doing application distribution, currently most applications are delivered via helm charts, does karmada have any plans to support distribution charts?

OCM supports installing applications as Helm Chart and loading remote Chart repositories.

lfbear · 2021-10-26T07:37:45Z

Sure, I knew this is in plan and the helm chart has done.

lfbear · 2021-10-26T07:40:07Z

Related issue #323

learner0810 · 2021-10-26T09:43:45Z

@lfbear sorry,Maybe my previous expression is not accurate enough. I meant to distribute the helm application to the Member cluster through Karmada, not install Karmada through helm.

lfbear · 2021-10-26T10:18:07Z

Oh, I see. That seems a good demand, @RainbowMango What do you think

wawa0210 · 2021-10-27T01:36:04Z

In general scenarios, users can deploy applications in the following three ways

kubernetes yaml
helm charts
CRD bundle

Kubernetes yaml is usually used for the distribution of a single resource, but in real customer scenarios, applications usually include not only one workload (deployment, daemonset, etc.), but also RBAC, configmap, secret, and multiple deployments, etc.
At this time, users generally use helm charts to describe the entire application (similar to crd bundle). Does karmada have any special attention to this scenario?

At present, the open source community already has some open source projects similar to helm CRD,such as
https://github.com/fluxcd/helm-operator. We can think about how to integrate efficiently

RainbowMango · 2021-10-27T01:42:34Z

Yes. Karmada will support helm chart propagating, but no clear solution yet.
If anyone could help give a proposal would be appreciated.

learner0810 · 2021-10-28T11:06:31Z

Use an external component, such as helm-Opertaor and install the helm-Opertaor based on the parameters during cluster installation.

The following is an example of karmada distribution of Helm Charts by helm-operator

Install helm-operator in the member cluster

$ helm install helm-operator fluxcd/helm-operator --namespace flux --set helm.versions=v3

Distribute helmreleases.helm.fluxcd.io to member cluster

apiVersion: policy.karmada.io/v1alpha1
kind: ClusterPropagationPolicy
metadata:
  name: crd-propagation
spec:
  resourceSelectors:
    - apiVersion: apiextensions.k8s.io/v1
      kind: CustomResourceDefinition
      name: helmreleases.helm.fluxcd.io
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Distribute this yaml to member cluster

kind: HelmRelease
metadata:
  name: jaeger-operator
  namespace: default
spec:
  chart:
    repository: $url
    name: jaeger-operator
    version: 2.12.3

Then you can see the jaeger-operator deployed to the Member cluster

Karmada built-in support for helm Charts distribution

Which one do you prefer? @RainbowMango

pidb · 2021-10-29T01:18:50Z

Can we consider this third-party application package management tool (such as helm) for a unified abstraction, because I know that in addition to helm, there are other management tools such as kustomzie. We should minimize the CRD definition inside karmada, They should be included in a generic CRD.

learner0810 · 2021-12-21T07:34:19Z

tools：flux
Download the flux CLI:

 curl -s https://fluxcd.io/install.sh | sudo bash

Install the toolkit controllers in the flux-system namespace:

flux install

tips:

The Flux tool is installed on each cluster,Using Flux Install requires scientific surfing

If Flux is successfully installed, you can see the following POD

[root@10-6-201-150 ~]# kubectl get pod -n flux-system
NAME                                       READY   STATUS    RESTARTS   AGE
helm-controller-55896d6ccf-dlf8b           1/1     Running   0          15d
kustomize-controller-76795877c9-mbrsk      1/1     Running   0          15d
notification-controller-7ccfbfbb98-lpgjl   1/1     Running   0          15d
source-controller-6b8d9cb5cc-7dbcb         1/1     Running   0          15d

helm

Define a HelmRepository source

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: podinfo
spec:
  interval: 1m
  url: https://stefanprodan.github.io/podinfo     
--- 
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-repo
spec:
  resourceSelectors:
    - apiVersion: source.toolkit.fluxcd.io/v1beta1
      kind: HelmRepository
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Define a HelmRelease source

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: podinfo
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      version: 5.0.3
      sourceRef:
        kind: HelmRepository
        name: podinfo
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-release
spec:
  resourceSelectors:
    - apiVersion: helm.toolkit.fluxcd.io/v2beta1
      kind: HelmRelease
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Apply those YAML to karma-apiserver

[root@10-6-201-150 helm]# kubectl apply -f ../helm/
helmrelease.helm.toolkit.fluxcd.io/podinfo created
helmrepository.source.toolkit.fluxcd.io/podinfo created
propagationpolicy.policy.karmada.io/helm-release created
propagationpolicy.policy.karmada.io/helm-repo created

Switch to the distributed cluster

[root@10-6-201-150 ~]# kubectl config use-context member2
Switched to context "member2".
[root@10-6-201-150 ~]# kubectl get pod
NAME                      READY   STATUS    RESTARTS   AGE
podinfo-78c475b77-94x54   1/1     Running   0          104s
[root@10-6-201-150 ~]# helm list -A
NAME   	NAMESPACE	REVISION	UPDATED                               	STATUS  	CHART        	APP VERSION
podinfo	default  	1       	2021-12-21 07:14:50.73460681 +0000 UTC	deployed	podinfo-5.0.3	5.0.3
[root@10-6-201-150 ~]#

kustomize

Define a Git repository source

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
  name: podinfo
spec:
  interval: 1m
  url: https://github.com/stefanprodan/podinfo
  ref:
    branch: master
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: kust-git
spec:
  resourceSelectors:
    - apiVersion: source.toolkit.fluxcd.io/v1beta1
      kind: GitRepository
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Define a kustomization

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: podinfo-dev
spec:
  interval: 5m
  path: "./deploy/overlays/dev/"
  prune: true
  sourceRef:
    kind: GitRepository
    name: podinfo
  validation: client
  timeout: 80s
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: kust-release
spec:
  resourceSelectors:
    - apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
      kind: Kustomization
      name: podinfo-dev
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Apply those YAML to karma-apiserver

[root@10-6-201-150 flux]# kubectl apply -f kust/
gitrepository.source.toolkit.fluxcd.io/podinfo created
kustomization.kustomize.toolkit.fluxcd.io/podinfo-dev created
propagationpolicy.policy.karmada.io/kust-git created
propagationpolicy.policy.karmada.io/kust-release created

Switch to the distributed cluster

[root@10-6-201-150 ~]# kubectl get pod -n dev
NAME                        READY   STATUS    RESTARTS   AGE
backend-69c7655cb-rbtrq     1/1     Running   0          15s
cache-bdff5c8dc-mmnbm       1/1     Running   0          15s
frontend-7f98bf6f85-dw4vq   1/1     Running   0          15s

learner0810 · 2021-12-21T07:43:20Z

@RainbowMango Please Take A Look.This demo is just basic functionality, I can prepare more demos if you need to learn more about Flux's functionality.
Please let me know if you have any questions.

RainbowMango · 2021-12-21T07:54:07Z

Pretty good after a quick look!!!
This is more like an integration solution with fluxcd.io. I like it.

RainbowMango · 2021-12-21T07:59:15Z

I'll look into it and try it on my side and get back to you. Many thanks.

RainbowMango · 2021-12-24T08:38:38Z

I think we can add a document at docs/work-with-flux.md. The structure would be:

Overview: description about flux
Prerequisites: installing flux.
Propagating Helm by Karmada
Propagating Kustomize by Karmada

How do you think @learner0810 ? And would like to do it?

learner0810 · 2021-12-24T09:54:53Z

I think we can add a document at docs/work-with-flux.md. The structure would be:

Overview: description about flux

Prerequisites: installing flux.

Propagating Helm by Karmada

Propagating Kustomize by Karmada

How do you think @learner0810 ? And would like to do it?

I agree with the idea. I would be more than happy to complete this document
/assign

RainbowMango · 2021-12-24T10:11:55Z

Thanks, hard to say this is the only way we support helm chart propagation, but it's a very very good start!

RainbowMango · 2021-12-29T13:32:22Z

Hi @learner0810
Since we are going to cut the new release by the end of this week, so I'm asking when this document will be ready?

learner0810 · 2021-12-30T03:10:41Z

Hi @learner0810 Since we are going to cut the new release by the end of this week, so I'm asking when this document will be ready?

I am very sorry. I've been very busy lately, so the document is not ready yet, can you put it in the next version?

Poor12 · 2022-05-10T09:44:17Z

Really interested with flux. Since @learner0810 is busy, I will help with this workguide.

learner0810 · 2022-05-10T12:12:10Z

Really interested with flux. Since @learner0810 is busy, I will help with this workguide.

I am very sorry that I am writing this work guide. You can look at other issues and say sorry again

Poor12 · 2022-05-11T01:11:45Z

It doesn't matter. Looking Forward to your contribution.

Poor12 · 2022-05-16T03:23:41Z

kindly ping @learner0810

RainbowMango · 2022-05-25T07:25:47Z

Hi @learner0810 Are you still working on this?

learner0810 · 2022-05-25T07:40:36Z

Hi @learner0810 Are you still working on this?

#1881

I see that someone has mentioned the distribution of helm PR, this document is not needed, I closed this issue, what do you think?

RainbowMango · 2022-05-25T08:06:11Z

#1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.

This issue demonstrates how to integrate flux with Karmada, that's a useful option too.
So, we still need a tutorial guide based on your demo above. Feel free to let me know if you don't have enough time to do it.

learner0810 · 2022-05-26T04:22:36Z

#1881 is trying an experimental approach that saves the efforts to deploy flux components onto member clusters.

This issue demonstrates how to integrate flux with Karmada, that's a useful option too. So, we still need a tutorial guide based on your demo above. Feel free to let me know if you don't have enough time to do it.

I see, thanks. I will finish the document by the weekend.

RainbowMango · 2022-05-26T08:15:46Z

Thanks
/assign @learner0810

We are going to cut a new release by the end of this week(probably this Saturday), hope this document could be included in this release.

learner0810 added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 25, 2021

karmada-bot assigned learner0810 May 26, 2022

learner0810 mentioned this issue May 27, 2022

docs/work-with-flux #1900

Merged

karmada-bot closed this as completed in #1900 May 28, 2022

learner0810 mentioned this issue May 26, 2023

REQUEST: New membership for learner0810 karmada-io/community#32

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

karmada will support distribution of helm afterwards？ #861

karmada will support distribution of helm afterwards？ #861

learner0810 commented Oct 25, 2021 •

edited

Loading

lfbear commented Oct 26, 2021

lfbear commented Oct 26, 2021

learner0810 commented Oct 26, 2021

lfbear commented Oct 26, 2021

wawa0210 commented Oct 27, 2021 •

edited

Loading

RainbowMango commented Oct 27, 2021

learner0810 commented Oct 28, 2021 •

edited

Loading

pidb commented Oct 29, 2021

learner0810 commented Dec 21, 2021

learner0810 commented Dec 21, 2021

RainbowMango commented Dec 21, 2021

RainbowMango commented Dec 21, 2021

RainbowMango commented Dec 24, 2021

learner0810 commented Dec 24, 2021 •

edited

Loading

RainbowMango commented Dec 24, 2021

RainbowMango commented Dec 29, 2021

learner0810 commented Dec 30, 2021

Poor12 commented May 10, 2022

learner0810 commented May 10, 2022

Poor12 commented May 11, 2022

Poor12 commented May 16, 2022

RainbowMango commented May 25, 2022

learner0810 commented May 25, 2022

RainbowMango commented May 25, 2022

learner0810 commented May 26, 2022 •

edited

Loading

RainbowMango commented May 26, 2022

karmada will support distribution of helm afterwards？ #861

karmada will support distribution of helm afterwards？ #861

Comments

learner0810 commented Oct 25, 2021 • edited Loading

lfbear commented Oct 26, 2021

lfbear commented Oct 26, 2021

learner0810 commented Oct 26, 2021

lfbear commented Oct 26, 2021

wawa0210 commented Oct 27, 2021 • edited Loading

RainbowMango commented Oct 27, 2021

learner0810 commented Oct 28, 2021 • edited Loading

pidb commented Oct 29, 2021

learner0810 commented Dec 21, 2021

learner0810 commented Dec 21, 2021

RainbowMango commented Dec 21, 2021

RainbowMango commented Dec 21, 2021

RainbowMango commented Dec 24, 2021

learner0810 commented Dec 24, 2021 • edited Loading

RainbowMango commented Dec 24, 2021

RainbowMango commented Dec 29, 2021

learner0810 commented Dec 30, 2021

Poor12 commented May 10, 2022

learner0810 commented May 10, 2022

Poor12 commented May 11, 2022

Poor12 commented May 16, 2022

RainbowMango commented May 25, 2022

learner0810 commented May 25, 2022

RainbowMango commented May 25, 2022

learner0810 commented May 26, 2022 • edited Loading

RainbowMango commented May 26, 2022

learner0810 commented Oct 25, 2021 •

edited

Loading

wawa0210 commented Oct 27, 2021 •

edited

Loading

learner0810 commented Oct 28, 2021 •

edited

Loading

learner0810 commented Dec 24, 2021 •

edited

Loading

learner0810 commented May 26, 2022 •

edited

Loading