Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update helm chart for cilium-operator to implement per-provider opera… #11837

Closed
wants to merge 3 commits into from
Closed

Update helm chart for cilium-operator to implement per-provider opera… #11837

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d6060e6
install: Fix up version/pullPolicy for multiple values files
joestringer Jun 2, 2020
ab55205
Implement per-provider operator deployments
seanmwinn Jun 2, 2020
656a952
fix rendering for image names with a / to not append suffixes
seanmwinn Jun 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/helm-check.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
- name: Build docker images
run: |
make docker-image-no-clean
make docker-operator-image
make docker-operator-generic-image

- name: Prevent K8s from pulling images
run: |
Expand All @@ -58,7 +58,7 @@ jobs:
- name: Load local images into kind cluster
run: |
kind load docker-image --name chart-testing cilium/cilium:latest
kind load docker-image --name chart-testing cilium/operator:latest
kind load docker-image --name chart-testing cilium/operator-generic:latest

# This test is just to make sure the helm chart can be installed. It will not replace existing integration tests
- name: Run chart-testing (install)
Expand All @@ -77,4 +77,4 @@ jobs:
run: |
cd test
export PATH=$PATH:$HOME/go/bin
ginkgo -v --focus="K8sConformance Portmap Chaining Check one node.*" -- -cilium.provision=false -test.v --cilium.kubeconfig=${HOME}/.kube/config -cilium.image=cilium/cilium:latest -cilium.operator-image=cilium/operator:latest
ginkgo -v --focus="K8sConformance Portmap Chaining Check one node.*" -- -cilium.provision=false -test.v --cilium.kubeconfig=${HOME}/.kube/config -cilium.image=cilium/cilium:latest -cilium.operator-image=cilium/operator-generic:latest
6 changes: 3 additions & 3 deletions Documentation/contributing/testing/e2e.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ An example invocation is

::

CNI_INTEGRATION=eks K8S_VERSION=1.13 ginkgo --focus="K8s*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator:latest" -cilium.passCLIEnvironment=true
CNI_INTEGRATION=eks K8S_VERSION=1.13 ginkgo --focus="K8s*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator-generic:latest" -cilium.passCLIEnvironment=true

GKE (experimental)
^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -495,7 +495,7 @@ cluster.

::

CNI_INTEGRATION=gke K8S_VERSION=1.13 ginkgo -v --focus="K8sDemo*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator:latest" -cilium.passCLIEnvironment=true
CNI_INTEGRATION=gke K8S_VERSION=1.13 ginkgo -v --focus="K8sDemo*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator-generic:latest" -cilium.passCLIEnvironment=true

.. note:: The kubernetes version defaults to 1.13 but can be configured with
versions between 1.13 and 1.15. Check with ``kubectl version``
Expand All @@ -513,7 +513,7 @@ cluster.

::

CNI_INTEGRATION=eks K8S_VERSION=1.14 ginkgo -v --focus="K8sDemo*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator:latest" -cilium.passCLIEnvironment=true
CNI_INTEGRATION=eks K8S_VERSION=1.14 ginkgo -v --focus="K8sDemo*" -noColor -- -cilium.provision=false -cilium.kubeconfig=`echo ~/.kube/config` -cilium.image="docker.io/cilium/cilium:latest" -cilium.operator-image="docker.io/cilium/operator-generic:latest" -cilium.passCLIEnvironment=true
seanmwinn marked this conversation as resolved.
Show resolved Hide resolved

Be sure to pass ``--cilium.passCLIEnvironment=true`` to allow kubectl to invoke ``aws-iam-authenticator``

Expand Down
25 changes: 13 additions & 12 deletions install/kubernetes/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ MANAGED_ETCD_VERSION := "v2.0.7"
QUICK_INSTALL := "$(ROOT_DIR)/$(RELATIVE_DIR)/quick-install.yaml"
EXPERIMENTAL_INSTALL := "$(ROOT_DIR)/$(RELATIVE_DIR)/experimental-install.yaml"
MANAGED_ETCD_PATH := "$(ROOT_DIR)/$(RELATIVE_DIR)/cilium/charts/managed-etcd/values.yaml"
CILIUM_CHARTS := "$(ROOT_DIR)/$(RELATIVE_DIR)/cilium/"
CILIUM_VALUES := "$(CILIUM_CHARTS)/values.yaml"
CILIUM_CHARTS := "$(ROOT_DIR)/$(RELATIVE_DIR)/cilium"
CILIUM_VALUES := "$(CILIUM_CHARTS)/values.yaml" "$(CILIUM_CHARTS)/charts/operator/values.yaml"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When rebasing, this change will still be necessary.


VERSION_REGEX := '[0-9]\+\.[0-9]\+\.[0-9]\+.*'
LATEST_VERSION_REGEX := '[0-9]\+\.[0-9]\+\.90'
DEV_VERSION_REGEX := '[0-9]\+\.[0-9]\+\.[0-9]\+-dev'
CILIUM_CHART_REGEX := '\([vV]ersion:\) '$(VERSION_REGEX)
CILIUM_TAG_REGEX := '\(tag:\) \(v'$(VERSION_REGEX)'\|latest\)'
CILIUM_PULLPOLICY_REGEX := '\(pullPolicy:\) .*'
CILIUM_PULLPOLICY_REGEX := '\([pP]ullPolicy:\) .*'
EXPERIMENTAL_OPTIONS := \
--set global.hubble.enabled=true \
--set global.hubble.listenAddress=":4244" \
Expand All @@ -35,20 +35,21 @@ $(EXPERIMENTAL_INSTALL): $(shell find cilium/ -type f)
update-versions:
$(ECHO_GEN) " -> Updating version to $(VERSION)"
@# Update chart versions to point to the current version.
$(QUIET)grep -lRZ -e "version:" -e "appVersion:" $(CILIUM_CHARTS) | \
$(QUIET)grep -lRZ -e "version:" -e "appVersion:" $(CILIUM_CHARTS)/ | \
xargs -0 -l sed -i -e 's/'$(CILIUM_CHART_REGEX)'/\1 $(VERSION)/g'
@# Fix up the cilium tag
$(QUIET)if echo $(VERSION) | grep -q $(LATEST_VERSION_REGEX); then \
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 latest/' $(CILIUM_VALUES); \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 Always/' $(CILIUM_VALUES); \
$(QUIET)for chart in $(CILIUM_VALUES); do \
if echo $(VERSION) | grep -q $(LATEST_VERSION_REGEX); then \
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 latest/' $$chart; \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 Always/' $$chart; \
elif echo $(VERSION) | grep -q $(DEV_VERSION_REGEX); then \
DEV_BRANCH=$$(echo $(VERSION) | sed 's/-dev//') \
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 $(DEV_BRANCH)/' $(CILIUM_VALUES); \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 Always/' $(CILIUM_VALUES); \
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 $(DEV_BRANCH)/' $$chart; \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 Always/' $$chart; \
else \
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 v$(VERSION)/' $(CILIUM_VALUES); \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 IfNotPresent/' $(CILIUM_VALUES); \
fi
sed -i 's/'$(CILIUM_TAG_REGEX)'/\1 v$(VERSION)/' $$chart; \
sed -i 's/'$(CILIUM_PULLPOLICY_REGEX)'/\1 IfNotPresent/' $$chart; \
fi; done
@# Fix up the managed etcd version, as that has its own scheme
$(QUIET)sed -i 's/'$(VERSION)'/'$(MANAGED_ETCD_VERSION)'/' $(MANAGED_ETCD_PATH)

Expand Down
32 changes: 28 additions & 4 deletions install/kubernetes/cilium/charts/operator/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,13 @@ spec:
- --config-dir=/tmp/cilium/config-map
- --debug=$(CILIUM_DEBUG)
command:
- cilium-operator
{{- if .Values.global.eni }}
seanmwinn marked this conversation as resolved.
Show resolved Hide resolved
- cilium-operator-aws
{{- else if .Values.global.azure.enabled }}
- cilium-operator-azure
{{- else }}
- cilium-operator-generic
{{- end }}
env:
- name: K8S_NODE_NAME
valueFrom:
Expand Down Expand Up @@ -101,12 +107,30 @@ spec:
- name: AZURE_CLIENT_SECRET
value: {{ .Values.global.azure.clientSecret }}
{{- end }}
{{- if contains "/" .Values.image }}
image: "{{ .Values.image }}"
{{- if .Values.global.eni }}
{{- if contains "/" .Values.image.repository }}
image: "{{ .Values.image.repository }}"
{{- else }}
image: "{{ .Values.global.registry }}/{{ .Values.image.repository }}-aws:{{ .Values.image.tag }}"
{{- end }}
{{- else if .Values.global.azure.enabled }}
{{- if contains "/" .Values.image.repository }}
image: "{{ .Values.image.repository }}"
{{- else }}
image: "{{ .Values.global.registry }}/{{ .Values.image.repository }}-azure:{{ .Values.image.tag }}"
{{- end }}
{{- else }}
{{- if contains "/" .Values.image.repository }}
image: "{{ .Values.image.repository }}"
{{- else }}
image: "{{ .Values.global.registry }}/{{ .Values.image }}:{{ .Values.global.tag }}"
image: "{{ .Values.global.registry }}/{{ .Values.image.repository }}-generic:{{ .Values.image.tag }}"
{{- end }}
{{- end }}
{{- if .Values.global.pullPolicy }}
imagePullPolicy: {{ .Values.global.pullPolicy }}
{{- else }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
joestringer marked this conversation as resolved.
Show resolved Hide resolved
{{- end }}
name: cilium-operator
{{- if .Values.global.prometheus.enabled }}
ports:
Expand Down
6 changes: 5 additions & 1 deletion install/kubernetes/cilium/charts/operator/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
image: operator
image:
repository: operator
tag: latest
joestringer marked this conversation as resolved.
Show resolved Hide resolved
pullPolicy: Always

Comment on lines +1 to +5
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of the other changes were already applied in other PRs, except for this change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Well, and obviously when this changes the actual daemonset needs to take this into account)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to look at this closely from an upgrade instructions angle because changing the "image" value from a string to a map may cause helm to reject the upgrade.

# Deprecated: please use synchronizeK8sNodes in install/kubernetes/cilium/values.yaml
synchronizeK8sNodes: true

Expand Down
4 changes: 2 additions & 2 deletions install/kubernetes/experimental-install.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -841,7 +841,7 @@ spec:
- --config-dir=/tmp/cilium/config-map
- --debug=$(CILIUM_DEBUG)
command:
- cilium-operator
- cilium-operator-generic
env:
- name: K8S_NODE_NAME
valueFrom:
Expand Down Expand Up @@ -877,7 +877,7 @@ spec:
key: AWS_DEFAULT_REGION
name: cilium-aws
optional: true
image: "docker.io/cilium/operator:latest"
image: "docker.io/cilium/operator-generic:latest"
imagePullPolicy: Always
name: cilium-operator
livenessProbe:
Expand Down
4 changes: 2 additions & 2 deletions install/kubernetes/quick-install.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -620,7 +620,7 @@ spec:
- --config-dir=/tmp/cilium/config-map
- --debug=$(CILIUM_DEBUG)
command:
- cilium-operator
- cilium-operator-generic
env:
- name: K8S_NODE_NAME
valueFrom:
Expand Down Expand Up @@ -656,7 +656,7 @@ spec:
key: AWS_DEFAULT_REGION
name: cilium-aws
optional: true
image: "docker.io/cilium/operator:latest"
image: "docker.io/cilium/operator-generic:latest"
imagePullPolicy: Always
name: cilium-operator
livenessProbe:
Expand Down
2 changes: 1 addition & 1 deletion jenkinsfiles/ginkgo-eks.Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ pipeline {
}
steps {
dir("${TESTDIR}"){
sh 'CILIUM_IMAGE=$(./print-node-ip.sh)/cilium/cilium:latest CILIUM_OPERATOR_IMAGE=$(./print-node-ip.sh)/cilium/operator:latest ginkgo --focus="$(echo ${ghprbCommentBody} | sed -r "s/([^ ]* |^[^ ]*$)//" | sed "s/^$/K8s*/")" -v --failFast=${FAILFAST} -- -cilium.provision=false -cilium.timeout=${GINKGO_TIMEOUT} -cilium.kubeconfig=${TESTDIR}/eks/eks-kubeconfig -cilium.passCLIEnvironment=true -cilium.registry=$(./print-node-ip.sh)'
sh 'CILIUM_IMAGE=$(./print-node-ip.sh)/cilium/cilium:latest CILIUM_OPERATOR_IMAGE=$(./print-node-ip.sh)/cilium/operator-generic:latest ginkgo --focus="$(echo ${ghprbCommentBody} | sed -r "s/([^ ]* |^[^ ]*$)//" | sed "s/^$/K8s*/")" -v --failFast=${FAILFAST} -- -cilium.provision=false -cilium.timeout=${GINKGO_TIMEOUT} -cilium.kubeconfig=${TESTDIR}/eks/eks-kubeconfig -cilium.passCLIEnvironment=true -cilium.registry=$(./print-node-ip.sh)'
seanmwinn marked this conversation as resolved.
Show resolved Hide resolved
}
}
post {
Expand Down
4 changes: 2 additions & 2 deletions test/helpers/kubectl.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ var (
"agent.image": "cilium-dev",
"preflight.image": "cilium-dev", // Set again in init to match agent.image!
"global.tag": "latest",
"operator.image": "operator",
"operator.image.repository": "operator",
"global.debug.enabled": "true",
"global.k8s.requireIPv4PodCIDR": "true",
"global.pprof.enabled": "true",
Expand Down Expand Up @@ -239,7 +239,7 @@ func Init() {
"CILIUM_REGISTRY": "global.registry",
"CILIUM_TAG": "global.tag",
"CILIUM_IMAGE": "agent.image",
"CILIUM_OPERATOR_IMAGE": "operator.image",
"CILIUM_OPERATOR_IMAGE": "operator.image.repository",
"HUBBLE_RELAY_IMAGE": "hubble-relay.image",
} {
if v := os.Getenv(envVar); v != "" {
Expand Down
8 changes: 4 additions & 4 deletions test/k8sT/Updates.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,10 @@ func InstallAndValidateCiliumUpgrades(kubectl *helpers.Kubectl, oldHelmChartVers
By("Deploying Cilium %s", oldHelmChartVersion)

opts := map[string]string{
"global.tag": oldImageVersion,
"global.registry": "docker.io/cilium",
"agent.image": "cilium",
"operator.image": "operator",
"global.tag": oldImageVersion,
"global.registry": "docker.io/cilium",
"agent.image": "cilium",
"operator.image.repository": "operator",
}
if helpers.RunsWithoutKubeProxy() {
opts["global.nodePort.device"] = privateIface
Expand Down
2 changes: 1 addition & 1 deletion test/kubernetes-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ helm template install/kubernetes/cilium \
--set global.registry=k8s1:5000/cilium \
--set global.tag=latest \
--set agent.image=cilium-dev \
--set operator.image=operator \
--set operator.image.repository=operator \
--set global.debug.enabled=true \
--set global.k8s.requireIPv4PodCIDR=true \
--set global.pprof.enabled=true \
Expand Down