Auto gen of AWS CNI, metrics helper and calico artifacts through helm #1271

jayanthvn · 2020-10-27T21:05:31Z

Thanks to Brandon Wagner (@bwagner5) for the scripts. Modified it to create CNI/CNI metrics and calico manifests for all the supported regions/account id and copy as assets.

What type of PR is this?
feature

Which issue does this PR fix:
#758

What does this PR do / Why do we need it:
Jay Pipes comment -
"We will be replacing the config/ directory and its contents with a single charts/ directory containing a Helm chart that installs the equivalent of :latest Docker image tag with a set of reasonable production default configuration values. We will write a script that runs helm template to generate static manifests and attach those static manifests as artifacts to Github releases of the source repository's release. Finally, we'll be adding a script that automates the process of copying the Helm chart from this repository's charts/ directory into the github.com/aws/eks-charts repository's stable/amazon-vpc-cni-k8s Chart when releases are cut."

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:

Testing done on this change:

Yes

dev-dsk-varavaj-2b-72f02457 % helm template charts
---
# Source: aws-vpc-cni/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: aws-node
  labels:
    app.kubernetes.io/name: aws-node
    helm.sh/chart: aws-vpc-cni-1.1.0
    app.kubernetes.io/instance: RELEASE-NAME
    k8s-app: aws-node
    app.kubernetes.io/version: "v1.7.3"
    app.kubernetes.io/managed-by: Helm
---
# Source: aws-vpc-cni/templates/customresourcedefinition.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: eniconfigs.crd.k8s.amazonaws.com
  labels:
    app.kubernetes.io/name: aws-node
    helm.sh/chart: aws-vpc-cni-1.1.0
    app.kubernetes.io/instance: RELEASE-NAME
    k8s-app: aws-node
    app.kubernetes.io/version: "v1.7.3"
    app.kubernetes.io/managed-by: Helm
spec:
  scope: Cluster
  group: crd.k8s.amazonaws.com
  versions:
    - name: v1alpha1
      served: true
      storage: true
  names:
    plural: eniconfigs
    singular: eniconfig
    kind: ENIConfig
---
# Source: aws-vpc-cni/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: aws-node
  labels:
    app.kubernetes.io/name: aws-node
    helm.sh/chart: aws-vpc-cni-1.1.0
    app.kubernetes.io/instance: RELEASE-NAME
    k8s-app: aws-node
    app.kubernetes.io/version: "v1.7.3"
    app.kubernetes.io/managed-by: Helm
rules:
  - apiGroups:
      - crd.k8s.amazonaws.com
    resources:
      - eniconfigs
    verbs: ["list", "watch", "get"]
  - apiGroups: [""]
    resources:
      - pods
      - namespaces
    verbs: ["list", "watch", "get"]
  - apiGroups: [""]
    resources:
      - nodes
    verbs: ["list", "watch", "get", "update"]
  - apiGroups: ["extensions"]
    resources:
      - '*'
    verbs: ["list", "watch"]
---
# Source: aws-vpc-cni/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: aws-node
  labels:
    app.kubernetes.io/name: aws-node
    helm.sh/chart: aws-vpc-cni-1.1.0
    app.kubernetes.io/instance: RELEASE-NAME
    k8s-app: aws-node
    app.kubernetes.io/version: "v1.7.3"
    app.kubernetes.io/managed-by: Helm
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: aws-node
subjects:
  - kind: ServiceAccount
    name: aws-node
    namespace: default
---
# Source: aws-vpc-cni/templates/daemonset.yaml
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: aws-node
  labels:
    app.kubernetes.io/name: aws-node
    helm.sh/chart: aws-vpc-cni-1.1.0
    app.kubernetes.io/instance: RELEASE-NAME
    k8s-app: aws-node
    app.kubernetes.io/version: "v1.7.3"
    app.kubernetes.io/managed-by: Helm
spec:
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 10%
    type: RollingUpdate
  selector:
    matchLabels:
      app.kubernetes.io/name: aws-node
      app.kubernetes.io/instance: RELEASE-NAME
  template:
    metadata:
      labels:
        app.kubernetes.io/name: aws-node
        app.kubernetes.io/instance: RELEASE-NAME
        k8s-app: aws-node
    spec:
      priorityClassName: "system-node-critical"
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: "beta.kubernetes.io/os"
                    operator: In
                    values:
                      - linux
                  - key: "beta.kubernetes.io/arch"
                    operator: In
                    values:
                      - amd64
                      - arm64
                  - key: "eks.amazonaws.com/compute-type"
                    operator: NotIn
                    values:
                      - fargate
              - matchExpressions:
                  - key: "kubernetes.io/os"
                    operator: In
                    values:
                      - linux
                  - key: "kubernetes.io/arch"
                    operator: In
                    values:
                      - amd64
                      - arm64
                  - key: "eks.amazonaws.com/compute-type"
                    operator: NotIn
                    values:
                      - fargate
      serviceAccountName: aws-node
      hostNetwork: true
      initContainers:
      - name: aws-vpc-cni-init
        image: "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni-init:v1.7.3"
        imagePullPolicy: Always
        env:
          - name: DISABLE_TCP_EARLY_DEMUX
            value: "false"
        securityContext:
            privileged: true
        volumeMounts:
          - mountPath: /host/opt/cni/bin
            name: cni-bin-dir
      terminationGracePeriodSeconds: 10
      tolerations:
        - operator: Exists
      securityContext:
        {}
      containers:
        - name: aws-node
          image: "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon-k8s-cni:v1.7.3"
          imagePullPolicy: Always
          ports:
            - containerPort: 61678
              name: metrics
          livenessProbe:
            exec:
              command:
              - /app/grpc-health-probe
              - -addr=:50051
            initialDelaySeconds: 60
          readinessProbe:
            exec:
              command:
              - /app/grpc-health-probe
              - -addr=:50051
            initialDelaySeconds: 1
          env:
            - name: ADDITIONAL_ENI_TAGS
              value: "{}"
            - name: AWS_VPC_CNI_NODE_PORT_SUPPORT
              value: "true"
            - name: AWS_VPC_ENI_MTU
              value: "9001"
            - name: AWS_VPC_K8S_CNI_CONFIGURE_RPFILTER
              value: "false"
            - name: AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG
              value: "false"
            - name: AWS_VPC_K8S_CNI_EXTERNALSNAT
              value: "false"
            - name: AWS_VPC_K8S_CNI_LOGLEVEL
              value: "DEBUG"
            - name: AWS_VPC_K8S_CNI_LOG_FILE
              value: "/host/var/log/aws-routed-eni/ipamd.log"
            - name: AWS_VPC_K8S_CNI_RANDOMIZESNAT
              value: "prng"
            - name: AWS_VPC_K8S_CNI_VETHPREFIX
              value: "eni"
            - name: AWS_VPC_K8S_PLUGIN_LOG_FILE
              value: "/var/log/aws-routed-eni/plugin.log"
            - name: AWS_VPC_K8S_PLUGIN_LOG_LEVEL
              value: "DEBUG"
            - name: DISABLE_INTROSPECTION
              value: "false"
            - name: DISABLE_METRICS
              value: "false"
            - name: ENABLE_POD_ENI
              value: "false"
            - name: WARM_ENI_TARGET
              value: "1"
            - name: MY_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          resources:
            requests:
              cpu: 10m
          securityContext:
            capabilities:
              add:
              - NET_ADMIN
          volumeMounts:
          - mountPath: /host/opt/cni/bin
            name: cni-bin-dir
          - mountPath: /host/etc/cni/net.d
            name: cni-net-dir
          - mountPath: /host/var/log/aws-routed-eni
            name: log-dir
          - mountPath: /var/run/dockershim.sock
            name: dockershim
          - mountPath: /var/run/aws-node
            name: run-dir
          - mountPath: /run/xtables.lock
            name: xtables-lock
      volumes:
      - name: cni-bin-dir
        hostPath:
          path: /opt/cni/bin
      - name: cni-net-dir
        hostPath:
          path: /etc/cni/net.d
      - name: dockershim
        hostPath:
          path: /var/run/dockershim.sock
      - name: log-dir
        hostPath:
          path: /var/log/aws-routed-eni
          type: DirectoryOrCreate
      - name: run-dir
        hostPath:
          path: /var/run/aws-node
          type: DirectoryOrCreate
      - name: xtables-lock
        hostPath:
          path: /run/xtables.lock

Yaml creates -

ls
aws-vpc-cni-cn.yaml             calico_individual-resources.tar  cni-metrics-helper-us-gov-east-1.yaml  helm
aws-vpc-cni-us-gov-east-1.yaml  calico.yaml                      cni-metrics-helper-us-gov-west-1.yaml  individual-resources
aws-vpc-cni-us-gov-west-1.yaml  cni_individual-resources.tar     cni-metrics-helper.yaml
aws-vpc-cni.yaml                cni-metrics-helper-cn.yaml       cni_metrics_individual-resources.tar


 % [individual-resources]
 ls
aws-calico  aws-vpc-cni  cni-metrics-helper

Auto upload of yamls to GitHub -
https://github.com/jayanthvn/amazon-vpc-cni-k8s/releases/tag/v1.8

make version
v1.7.5

Automation added to e2e:

No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

Yes

/charts/README has the details to install using helm

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

bwagner5

Nice work, thanks for taking this on! 🚀 I left a few minor comments on the charts.

Makefile

bwagner5 · 2020-11-03T15:24:54Z

charts/aws-calico/README.md

+Install the Calico CRDs:
+
+```shell
+kubectl apply -k github.com/aws/eks-charts/tree/master/stable/aws-calico/crds


Can these not be installed w/ helm? If not, can these be uploaded to the release assets in the aws-calico repo? I don't think it's a good practice to download crds directly from the default branch.

sure Brandon, I have exported these from eks-charts. Will fix this as part of following PR.

bwagner5 · 2020-11-03T15:32:30Z

charts/aws-calico/templates/daemon-set.yaml

+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:


does calico support ARM and windows? If not, you may want to set an OS and arch node selector.

Yes, it does support ARM. I am working on this - #1218. Once I validate will update the image.

charts/aws-calico/templates/deployment.yaml

charts/aws-calico/values.yaml

charts/aws-vpc-cni/templates/daemonset.yaml

scripts/generate-cni-yaml.sh

jayanthvn · 2020-11-03T17:02:25Z

Nice work, thanks for taking this on! 🚀 I left a few minor comments on the charts.

Thanks for the comments Brandon. Appreciate your help on this :)

bwagner5 · 2020-11-12T14:21:24Z

charts/aws-calico/README.md

+To install the chart with the release name `aws-calico` and default configuration:
+
+```shell
+$ helm install --name aws-calico --namespace kube-system eks/aws-calico


This is the helm2 install command. We updated this not too long ago in the aws-node-termination-handler readme. It's probably worth it to change to the helm3 version. Keeping the helm2 instructions in addition to helm3 is up to you.

#Helm2 $ helm install --name aws-calico --namespace kube-system eks/aws-calico #Helm3 $ helm install aws-calico --namespace kube-system eks/aws-calico

Sure will update it in the following PR. There are 2 AIs for the following PR - calico multi arch and helm3 in README.

bwagner5 · 2020-11-12T14:45:17Z

charts/aws-vpc-cni/README.md

+To install the chart with the release name `aws-vpc-cni` and default configuration:
+
+```shell
+$ helm install --name aws-vpc-cni --namespace kube-system eks/aws-vpc-cni


same here for helm2 and helm3 installation instructions

bwagner5

left a minor comment on each of the chart readmes, but otherwise, looks good! 👍

diego-leapyear · 2020-11-25T23:12:38Z

Any chance merging this anytime soon?
Thank you guys!

jayanthvn · 2020-11-27T15:44:01Z

Any chance merging this anytime soon?
Thank you guys!

Hi @diego-leapyear, I will try to merge this next week.

diego-leapyear · 2020-11-27T15:46:44Z

Any chance merging this anytime soon?
Thank you guys!

Hi @diego-leapyear, I will try to merge this next week.

Thank you @jayanthvn !

jaypipes

\o/

Yay :)

Great work on this @jayanthvn

jaypipes · 2020-12-01T19:26:53Z

charts/regions.json

+        "ecrRegion": "cn-northwest-1",
+        "ecrAccount": "961992271922",
+        "ecrDomain": "amazonaws.com.cn"
+    }


It will be so nice to get rid of these and use ECR Public repo (soon) :)

nithu0115

Yay!! ship it 🚀

jayanthvn marked this pull request as draft October 27, 2020 21:05

jayanthvn force-pushed the helm-auto-install branch 3 times, most recently from 4c221b6 to e22efe3 Compare October 31, 2020 02:55

jayanthvn changed the title ~~Initial commit for auto gen of deployment thru helm~~ Initial commit for auto gen of deployment through helm Oct 31, 2020

jayanthvn force-pushed the helm-auto-install branch 2 times, most recently from 4a0e0d0 to 6599eca Compare November 3, 2020 01:31

jayanthvn requested a review from jaypipes November 3, 2020 01:32

jayanthvn marked this pull request as ready for review November 3, 2020 01:32

jayanthvn changed the title ~~Initial commit for auto gen of deployment through helm~~ Auto gen of AWS CNI, metrics helper and calico artifacts through helm Nov 3, 2020

bwagner5 suggested changes Nov 3, 2020

View reviewed changes

Initial commit for auto gen of deployment thru helm

f665046

jayanthvn force-pushed the helm-auto-install branch from 6599eca to f665046 Compare November 3, 2020 18:53

jayanthvn requested a review from kishorj November 4, 2020 23:01

bwagner5 reviewed Nov 12, 2020

View reviewed changes

bwagner5 approved these changes Nov 12, 2020

View reviewed changes

jayanthvn requested a review from nithu0115 November 30, 2020 14:44

jaypipes approved these changes Dec 1, 2020

View reviewed changes

nithu0115 approved these changes Dec 1, 2020

View reviewed changes

jayanthvn merged commit 95ac430 into aws:master Dec 1, 2020

jayanthvn deleted the helm-auto-install branch December 1, 2020 20:06

jayanthvn added this to the v1.8.0 milestone Dec 1, 2020

This was referenced Dec 9, 2020

Refactor configuration file handling #758

Closed

Please remove reference to helm chart from README.md #847

Closed

Update the v1.7 calico manifest to Calico v3.17.1 #1328

Closed

This was referenced Mar 3, 2021

Document project versioning policy and upgrade guides where applicable #1395

Closed

Document release policies and practices #685

Closed

jayanthvn restored the helm-auto-install branch June 7, 2021 23:51

eks-bot mentioned this pull request Jun 11, 2021

🥳 aws-vpc-cni v1.8.0 Automated Release! 🥑 aws/eks-charts#538

Merged

jayanthvn mentioned this pull request Aug 2, 2021

ServiceAccount should precede DaemonSet in yaml #1491

Closed

TBBle mentioned this pull request Aug 19, 2021

Added config map for env vars #1088

Closed

rittneje mentioned this pull request Sep 22, 2021

ServiceAccount should precede DaemonSet in yaml #1632

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto gen of AWS CNI, metrics helper and calico artifacts through helm #1271

Auto gen of AWS CNI, metrics helper and calico artifacts through helm #1271

jayanthvn commented Oct 27, 2020 •

edited

bwagner5 left a comment

bwagner5 Nov 3, 2020

jayanthvn Nov 3, 2020

bwagner5 Nov 3, 2020

jayanthvn Nov 3, 2020

jayanthvn commented Nov 3, 2020

bwagner5 Nov 12, 2020 •

edited

jayanthvn Nov 12, 2020

bwagner5 Nov 12, 2020

bwagner5 left a comment

diego-leapyear commented Nov 25, 2020

jayanthvn commented Nov 27, 2020

diego-leapyear commented Nov 27, 2020

jaypipes left a comment

jaypipes Dec 1, 2020

nithu0115 left a comment

Auto gen of AWS CNI, metrics helper and calico artifacts through helm #1271

Auto gen of AWS CNI, metrics helper and calico artifacts through helm #1271

Conversation

jayanthvn commented Oct 27, 2020 • edited

bwagner5 left a comment

Choose a reason for hiding this comment

bwagner5 Nov 3, 2020

Choose a reason for hiding this comment

jayanthvn Nov 3, 2020

Choose a reason for hiding this comment

bwagner5 Nov 3, 2020

Choose a reason for hiding this comment

jayanthvn Nov 3, 2020

Choose a reason for hiding this comment

jayanthvn commented Nov 3, 2020

bwagner5 Nov 12, 2020 • edited

Choose a reason for hiding this comment

jayanthvn Nov 12, 2020

Choose a reason for hiding this comment

bwagner5 Nov 12, 2020

Choose a reason for hiding this comment

bwagner5 left a comment

Choose a reason for hiding this comment

diego-leapyear commented Nov 25, 2020

jayanthvn commented Nov 27, 2020

diego-leapyear commented Nov 27, 2020

jaypipes left a comment

Choose a reason for hiding this comment

jaypipes Dec 1, 2020

Choose a reason for hiding this comment

nithu0115 left a comment

Choose a reason for hiding this comment

jayanthvn commented Oct 27, 2020 •

edited

bwagner5 Nov 12, 2020 •

edited