Adding multi arch support in all kyverno components (AMD6 and ARM64) #1542

imrajdas · 2021-02-04T20:13:06Z

Related issue

Issue:
#1535

Adding multi arch support with docker buildx
ARCHs- AMD64, ARM64
file changes
- github actions build
- Makefile
- Dockerfiles(kyverno, initcontainer, cli)

Docker hub screenshot

Running on ARM k8s cluster

Running on AMD k8s cluster

For demo purpose use these image

kyverno- imrajdas/kyverno:latest
kyverno-initcontainer- imrajdas/kyvernopre:latest
cli - imrajdas/kyverno-cli:latest

What type of PR is this?

Proposed changes

Checklist

I have read the contributing guidelines.
I have added tests that prove my fix is effective or that my feature
works.
I have added or changed the documentation.

Further comments

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas · 2021-02-04T20:14:01Z

cc: @JimBugwadia / @realshuting / @chipzoller

Makefile

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas · 2021-02-04T21:07:02Z

build is failed due to this

kubectl cluster-info --context kind-kind
35

36
Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
37
ERROR: image: "ghcr.io/kyverno/kyverno:v1.3.2-rc2-8-gcbe586d2" not present locally
38

This is because docker buildx builds in image builder cache and it is not able to find it in docker engine. Either we have to user docker or docker buildx?

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas · 2021-02-08T10:07:42Z

@realshuting Build and e2e has been passed :)

Recent changes

docker images will build and push for both arm64 and amd64 ---- (using docker buildx)
For e2e, we are building all docker image in amd64 arch and testing that image. --- (using docker build)

Please let me know, if you have any question for this PR. Thanks

realshuting

See comments.

realshuting · 2021-02-09T22:52:29Z

Makefile

@@ -36,14 +36,17 @@ initContainer: fmt vet
 docker-publish-initContainer: docker-build-initContainer docker-tag-repo-initContainer docker-push-initContainer


Since docker-push-initContainer does everything build tag push, can we remove docker-publish-initContainer?

realshuting · 2021-02-09T22:52:58Z

Makefile

@@ -62,15 +65,18 @@ kyverno: fmt vet
 docker-publish-kyverno: docker-build-kyverno  docker-tag-repo-kyverno  docker-push-kyverno


realshuting · 2021-02-09T22:53:32Z

Makefile

-	@docker push $(REPO)/$(KYVERNO_CLI_IMAGE):$(IMAGE_TAG)
-	@docker push $(REPO)/$(KYVERNO_CLI_IMAGE):latest
+	@docker buildx build --file $(PWD)/$(CLI_PATH)/Dockerfile --progress plane --push --platform linux/arm64,linux/amd64 --tag $(REPO)/$(KYVERNO_CLI_IMAGE):$(IMAGE_TAG) .
+	@docker buildx build --file $(PWD)/$(CLI_PATH)/Dockerfile --progress plane --push --platform linux/arm64,linux/amd64 --tag $(REPO)/$(KYVERNO_CLI_IMAGE):latest .

 ##################################
 docker-publish-all: docker-publish-initContainer docker-publish-kyverno docker-publish-cli


realshuting · 2021-02-09T22:58:43Z

.github/workflows/build.yaml

+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+        id: buildx
+        with:
+          install: true
+


We also need to add this to:
https://github.com/kyverno/kyverno/blob/main/.github/workflows/image.yaml
https://github.com/kyverno/kyverno/blob/main/.github/workflows/release.yaml

realshuting · 2021-02-09T23:01:03Z

@rajdas98 Can you please merge the main branch to check all builds? Thanks.

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas · 2021-02-13T11:10:01Z

@realshuting Made the changes

Adding docker buildx action to release.yaml and image.yaml
Remove all docker tag commands
Now, make docker-publish-initContainer will build and push. we need this command because buildx push required build command

realshuting · 2021-02-17T03:11:27Z

Makefile


 docker-build-kyverno:
 	@docker buildx build --file $(PWD)/$(KYVERNO_PATH)/Dockerfile --progress plane --platform linux/arm64,linux/amd64 --tag $(REPO)/$(KYVERNO_IMAGE):$(IMAGE_TAG) . --build-arg LD_FLAGS=$(LD_FLAGS)

 docker-build-kyverno-amd64:
 	@docker build -f $(PWD)/$(KYVERNO_PATH)/Dockerfile -t $(REPO)/$(KYVERNO_IMAGE):$(IMAGE_TAG) . --build-arg LD_FLAGS=$(LD_FLAGS) --build-arg TARGETPLATFORM="linux/amd64"

-docker-build-local-kyverno:


Can we not remove this local build? Some are using this to test locally.

realshuting · 2021-02-18T19:53:10Z

@rajdas98 Can we keep this? #1542 (comment)

imrajdas · 2021-02-18T19:54:54Z

@rajdas98 Can we keep this? #1542 (comment)

**Oh I forgot that. Let me add docker-build-local-kyverno

Signed-off-by: rajdas98 <mail.rajdas@gmail.com>

imrajdas · 2021-02-18T20:09:48Z

@realshuting I reverted the docker-build-local-kyverno step in Makefile.

…yverno#1542) * Adding multi arch support Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding multi arch support Signed-off-by: Raj Das <mail.rajdas@gmail.com> * minor refactors Signed-off-by: Raj Das <mail.rajdas@gmail.com> * adding buildx action in e2e.yaml Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding kyvernopre Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding kyvernopre Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding amd build Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding go env Signed-off-by: Raj Das <mail.rajdas@gmail.com> * minor fix Signed-off-by: Raj Das <mail.rajdas@gmail.com> * removing docker tag Signed-off-by: Raj Das <mail.rajdas@gmail.com> * Adding local dockerfile build command Signed-off-by: rajdas98 <mail.rajdas@gmail.com> Signed-off-by: vyankatesh <vyankatesh@neualto.com>

imrajdas added 3 commits February 5, 2021 01:14

Adding multi arch support

2e6037d

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

Adding multi arch support

1ac95fd

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

minor refactors

ff83de3

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas commented Feb 4, 2021

View reviewed changes

Makefile Show resolved Hide resolved

imrajdas added 3 commits February 5, 2021 01:47

adding buildx action in e2e.yaml

3a077aa

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

Adding kyvernopre

da297f6

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

Adding kyvernopre

16938f8

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

imrajdas added 3 commits February 8, 2021 14:48

Adding amd build

e2f94b8

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

Adding go env

70057e5

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

minor fix

36ca684

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

realshuting requested changes Feb 9, 2021

View reviewed changes

imrajdas added 2 commits February 13, 2021 15:47

removing docker tag

3f24721

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

resolve merge conflicts

f23602c

Signed-off-by: Raj Das <mail.rajdas@gmail.com>

realshuting reviewed Feb 17, 2021

View reviewed changes

realshuting approved these changes Feb 17, 2021

View reviewed changes

imrajdas added 2 commits February 19, 2021 01:33

Merge remote-tracking branch 'upstream/main' into multi-arch

91756d5

Adding local dockerfile build command

22043ee

Signed-off-by: rajdas98 <mail.rajdas@gmail.com>

realshuting merged commit 0b832a0 into kyverno:main Feb 19, 2021

realshuting mentioned this pull request Feb 19, 2021

Request for multi-arch images of kyverno components (ARM support) #1535

Closed

imrajdas deleted the multi-arch branch June 26, 2021 16:30

imrajdas restored the multi-arch branch June 26, 2021 16:30

imrajdas deleted the multi-arch branch August 19, 2021 14:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding multi arch support in all kyverno components (AMD6 and ARM64) #1542

Adding multi arch support in all kyverno components (AMD6 and ARM64) #1542

imrajdas commented Feb 4, 2021 •

edited

imrajdas commented Feb 4, 2021

imrajdas commented Feb 4, 2021 •

edited

imrajdas commented Feb 8, 2021 •

edited

realshuting left a comment

realshuting Feb 9, 2021

realshuting Feb 9, 2021

realshuting Feb 9, 2021

realshuting Feb 9, 2021

realshuting commented Feb 9, 2021

imrajdas commented Feb 13, 2021 •

edited

realshuting Feb 17, 2021

realshuting commented Feb 18, 2021

imrajdas commented Feb 18, 2021 •

edited

imrajdas commented Feb 18, 2021

		@@ -36,14 +36,17 @@ initContainer: fmt vet
		docker-publish-initContainer: docker-build-initContainer docker-tag-repo-initContainer docker-push-initContainer

		@@ -62,15 +65,18 @@ kyverno: fmt vet
		docker-publish-kyverno: docker-build-kyverno docker-tag-repo-kyverno docker-push-kyverno

Adding multi arch support in all kyverno components (AMD6 and ARM64) #1542

Adding multi arch support in all kyverno components (AMD6 and ARM64) #1542

Conversation

imrajdas commented Feb 4, 2021 • edited

Related issue

Proposed changes

Checklist

Further comments

imrajdas commented Feb 4, 2021

imrajdas commented Feb 4, 2021 • edited

imrajdas commented Feb 8, 2021 • edited

realshuting left a comment

Choose a reason for hiding this comment

realshuting Feb 9, 2021

Choose a reason for hiding this comment

realshuting Feb 9, 2021

Choose a reason for hiding this comment

realshuting Feb 9, 2021

Choose a reason for hiding this comment

realshuting Feb 9, 2021

Choose a reason for hiding this comment

realshuting commented Feb 9, 2021

imrajdas commented Feb 13, 2021 • edited

realshuting Feb 17, 2021

Choose a reason for hiding this comment

realshuting commented Feb 18, 2021

imrajdas commented Feb 18, 2021 • edited

imrajdas commented Feb 18, 2021

imrajdas commented Feb 4, 2021 •

edited

imrajdas commented Feb 4, 2021 •

edited

imrajdas commented Feb 8, 2021 •

edited

imrajdas commented Feb 13, 2021 •

edited

imrajdas commented Feb 18, 2021 •

edited