Adding Power support for image quay.io/brancz/kube-rbac-proxy at quay.io

[seth-priya]

Hi All,

I have a requirement for using the kube-rbac-proxy image on Power(ppc64le) architecture for knative deployment. However, the images available here - https://quay.io/repository/brancz/kube-rbac-proxy and here https://quay.io/repository/coreos/kube-rbac-proxy?tab=tags have support for "amd64" only as seen below

docker image inspect quay.io/brancz/kube-rbac-proxy:v0.4.0 | grep Arch
"Architecture": "amd64",

I was able to build this image locally on a Power machine using "make container"

docker images | grep kube-rbac-proxy
quay.io/brancz/kube-rbac-proxy v0.4.0 6e9cecf4d991 41 minutes ago 41.2MB

docker image inspect quay.io/brancz/kube-rbac-proxy:v0.4.0 | grep Arch
"Architecture": "ppc64le",

I was looking for help in making the image @ quay.io, multi-arch, thank you.

[brancz]

I would need to double check myself, does quay support multi arch images? If so I'm happy to publish those images, but would like to avoid having to maintain multiple repositories for each arch.

[seth-priya]

@brancz yes, I found one example of a multi arch images on quay here https://quay.io/repository/coreos/flannel?tab=tags
They are distinguishing based on tag names, as seen.

[brancz]

That's not what I was thinking, the docker registry spec has a way to have multiple architectures under the same image+tag.

[seth-priya]

Yes, so there are multiple ways to do this. On dockerhub they have different namespaces, so for example, for postgres,
the ppc64le image is here https://hub.docker.com/r/ppc64le/postgres
the amd64 is here https://hub.docker.com/r/amd64/postgres

and https://hub.docker.com/_/postgres is actually the multi-arch manifest that, when pulled, will pull the correct image

Steps are here https://docs.docker.com/engine/reference/commandline/manifest/

Images are built here https://doi-janky.infosiftr.net/job/multiarch/

In cases where we have single namespace either image names can differ
https://cloud.docker.com/u/ibmcom/repository/docker/ibmcom/kibana-ppc64le
https://cloud.docker.com/u/ibmcom/repository/docker/ibmcom/kibana-amd64
https://cloud.docker.com/u/ibmcom/repository/docker/ibmcom/kibana

or tags can differ like in the above example.

[brancz]

So that means we do still always have to maintain multiple repositories no matter what?

[seth-priya]

Different images, will need to be built, yes.

It would help to simplify things from a maintenance perspective if we can have all the docker builds going on on Travis - we have Power support for Travis and the travis.yml file for Intel / Power would essentially be the same, with only one additional flag enabled. example here https://github.com/fluent/fluentd-docker-image/blob/master/.travis.yml

If this sounds like a reasonable solution, I can help with the Power changes and validation.

Please suggest.

[seth-priya]

@brancz , any thoughts on this? thank you!!

[brancz]

I'm having a conversation with Kubernetes SIG auth to potentially donate this project to sig-auth, I'd prefer to hold off on this until we figure that out, as the build/publish process would likely change anyways.

[seth-priya]

@brancz - okay, thanks for letting me know.

[shift]

Any updates on this? Would love to not have to build my own amd64/arm64/arm images.

[amitsadaphule]

@brancz I have a requirement for using the kube-rbac-proxy image on Power(ppc64le) architecture for IBM Cloud Operators deployment. I tried using the images available at https://quay.io/repository/brancz/kube-rbac-proxy?tab=tags and https://console.cloud.google.com/gcr/images/kubebuilder/GLOBAL/kube-rbac-proxy?gcrImageListsize=30. But it seems that the images tagged with ppc64le actually belong to amd64 arch. I checked the same for s390x image as well and that image is amd64 arch as well.

I checked the CI logs @ https://github.com/brancz/kube-rbac-proxy/runs/3659930457 and found that the binaries are correctly built for each ARCH. But the make container command always pulls in the amd64 based gcr.io/distroless/static:nonroot base image. I could see two ways to fix this:

1. add buildx usage to build multi-arch image to the CI
2. pass ARCH as parameter to the dockerfile and use that to distinguish the base images for different ARCHs. Patch below:

diff --git a/Dockerfile b/Dockerfile
index 8ae83311..a937b785 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,5 @@
-FROM gcr.io/distroless/static:nonroot
+ARG GOARCH=amd64
+FROM gcr.io/distroless/static:nonroot-$GOARCH

ARG BINARY=kube-rbac-proxy-linux-amd64
COPY _output/$BINARY /usr/local/bin/kube-rbac-proxy
diff --git a/Makefile b/Makefile
index 50e90c23..bb6c7d90 100644
--- a/Makefile
+++ b/Makefile
@@ -45,7 +45,7 @@ $(OUT_DIR)/$(BIN)-%:
build: $(OUT_DIR)/$(BIN)

container: $(OUT_DIR)/$(BIN)-$(GOOS)-$(GOARCH) Dockerfile
-       docker build --build-arg BINARY=$(BIN)-$(GOOS)-$(GOARCH) -t $(DOCKER_REPO):$(VERSION)-$(GOARCH) .
+       docker build --build-arg BINARY=$(BIN)-$(GOOS)-$(GOARCH) --build-arg GOARCH=$(GOARCH) -t $(DOCKER_REPO):$(VERSION)-$(GOARCH) .
ifeq ($(GOARCH), amd64)
        docker tag $(DOCKER_REPO):$(VERSION)-$(GOARCH) $(DOCKER_REPO):$(VERSION)
endif

Looking at the current scenario/implementation, I prefer option 2. Please let me know your feedback, so that I can work on raising a PR.

[amitsadaphule]

@brancz @s-urbaniak can you please let me know your thoughts on my comment above?

[s-urbaniak]

@amitsadaphule happy to accept a PR if you have a fix. I don't have much knowledge for that archictecture.

[amitsadaphule]

Thanks @s-urbaniak! Will raise a PR with the above fix shortly.

[amitsadaphule]

@s-urbaniak @brancz kindly review #147.

[amitsadaphule]

@s-urbaniak can you please let me know when you're planning to push out a new tag/release? I need to use that for adding Power support to IBM Cloud Operators repo.

[amitsadaphule]

@s-urbaniak can you please let me know when you're planning to push out a new tag/release? I need to use that for adding Power support to IBM Cloud Operators repo.

@s-urbaniak @brancz any update on the above?