ARM Container Images #113

seanmalloy · 2020-11-24T06:45:56Z

I would like to run the scheduler-plugins on ARM(specifically ARM64) hardware. I'd like an official scheduler-plugins container image for this use case. This would be in addition to the already provided x86_64 container image.

seanmalloy · 2020-11-24T06:49:27Z

@Huang-Wei @denkensk @yuanchen8911 any concerns if we also provide official ARM64 container images?

I'm willing to do the work if no one has any concerns.

/kind feature

Huang-Wei · 2020-11-24T20:58:07Z

@seanmalloy that sounds good to me :)

seanmalloy · 2020-11-24T21:10:28Z

Cool. I'll assign this to myself when I'm ready to start working on it. Should be fairly soon.

AlexeyPerevalov · 2020-11-25T14:13:04Z

we also interested in arm64 image, I already added it for https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin. So I'm in line for the task too ), but only since the next week.

seanmalloy · 2020-12-03T07:08:14Z

I have this all sorted out for deschdduler. See kubernetes-sigs/descheduler#449 for reference, so now it should be fairly easy for me to get this setup here.

Hoping to have a PR submitted in the next few days.

/assign

fejta-bot · 2021-03-03T08:04:16Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

seanmalloy · 2021-03-03T17:27:02Z

/remove-lifecycle stale

seanmalloy · 2021-03-08T22:06:43Z

Well ... it almost works. :-)

Log for the automated container image build job:

Running...
$ARTIFACTS is set, sending logs to /logs/artifacts
2021/03/08 21:37:06 Build directory: .
2021/03/08 21:37:06 Config directory: /home/prow/go/src/github.com/kubernetes-sigs/scheduler-plugins
2021/03/08 21:37:06 cd-ing to build directory: .
2021/03/08 21:37:06 Creating source tarball at /tmp/146047716...
2021/03/08 21:37:06 Uploading /tmp/146047716 to gs://k8s-staging-scheduler-plugins-gcb/source/42f716dd-89b1-49df-aa2b-23a1213512a8.tgz...
Copying file:///tmp/146047716 [Content-Type=application/octet-stream]...
/ [0 files][    0.0 B/  2.1 MiB]                                                
/ [1 files][  2.1 MiB/  2.1 MiB]                                                
Operation completed over 1 objects/2.1 MiB.                                      
2021/03/08 21:37:08 Running build jobs...
2021/03/08 21:37:08 No variants.yaml, starting single build job...
Created [https://cloudbuild.googleapis.com/v1/projects/k8s-staging-scheduler-plugins/builds/99bd2d78-f27f-4b00-b5b4-146452508512].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/99bd2d78-f27f-4b00-b5b4-146452508512?project=96918712006].
----------------------------- REMOTE BUILD OUTPUT ------------------------------
starting build "99bd2d78-f27f-4b00-b5b4-146452508512"

FETCHSOURCE
Fetching storage object: gs://k8s-staging-scheduler-plugins-gcb/source/1615239428.87-112fee7d095c4d96af755e700e60eb25.tgz#1615239429025509
Copying gs://k8s-staging-scheduler-plugins-gcb/source/1615239428.87-112fee7d095c4d96af755e700e60eb25.tgz#1615239429025509...
/ [0 files][    0.0 B/  2.0 MiB]                                                
/ [1 files][  2.0 MiB/  2.0 MiB]                                                
Operation completed over 1 objects/2.0 MiB.                                      
BUILD
Pulling image: gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
v20190906-745fed4: Pulling from k8s-testimages/gcb-docker-gcloud
9d48c3bd43c5: Pulling fs layer
7f94eaf8af20: Pulling fs layer
9fe9984849c1: Pulling fs layer
cf0db633a67d: Pulling fs layer
0f7136d71739: Pulling fs layer
36b51914f992: Pulling fs layer
b5f72b4b9a52: Pulling fs layer
cf0db633a67d: Waiting
0f7136d71739: Waiting
36b51914f992: Waiting
b5f72b4b9a52: Waiting
7f94eaf8af20: Download complete
9fe9984849c1: Verifying Checksum
9fe9984849c1: Download complete
0f7136d71739: Verifying Checksum
0f7136d71739: Download complete
9d48c3bd43c5: Download complete
36b51914f992: Verifying Checksum
36b51914f992: Download complete
9d48c3bd43c5: Pull complete
7f94eaf8af20: Pull complete
9fe9984849c1: Pull complete
cf0db633a67d: Verifying Checksum
cf0db633a67d: Download complete
b5f72b4b9a52: Verifying Checksum
b5f72b4b9a52: Download complete
cf0db633a67d: Pull complete
0f7136d71739: Pull complete
36b51914f992: Pull complete
b5f72b4b9a52: Pull complete
Digest: sha256:f17f1d2a58b8273e184a7c9f89a91d0193a6f458167891115868781610467ddd
Status: Downloaded newer image for gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
rm -rf ./bin
docker build -f ./build/scheduler/Dockerfile --build-arg ARCH="amd64" --build-arg RELEASE_VERSION="v20210308-v0.18.800-118-g29da026" -t gcr.io/k8s-staging-scheduler-plugins/kube-scheduler:v20210308-v0.18.800-118-g29da026-amd64 .
Sending build context to Docker daemon  3.047MB

Step 1/11 : ARG ARCH
Step 2/11 : FROM golang:1.15.8
1.15.8: Pulling from library/golang
0ecb575e629c: Already exists
7467d1831b69: Already exists
feab2c490a3c: Already exists
f15a0f46f8c3: Already exists
2ea92ed63b96: Pulling fs layer
223c5fc7af76: Pulling fs layer
764ae8cdbfbc: Pulling fs layer
764ae8cdbfbc: Download complete
2ea92ed63b96: Verifying Checksum
2ea92ed63b96: Download complete
223c5fc7af76: Verifying Checksum
223c5fc7af76: Download complete
2ea92ed63b96: Pull complete
223c5fc7af76: Pull complete
764ae8cdbfbc: Pull complete
Digest: sha256:acbe8c73a42159154c1eecc28472926e94f5a7f4cd790655d15dc6b6fbe46c92
Status: Downloaded newer image for golang:1.15.8
 ---> 7185d074e387
Step 3/11 : WORKDIR /go/src/sigs.k8s.io/scheduler-plugins
 ---> Running in e8973e2ddaa4
Removing intermediate container e8973e2ddaa4
 ---> 55d3adb106b0
Step 4/11 : COPY . .
 ---> 6249893cbc2f
Step 5/11 : ARG ARCH
 ---> Running in a5c8b206f40f
Removing intermediate container a5c8b206f40f
 ---> 97a132381121
Step 6/11 : ARG RELEASE_VERSION
 ---> Running in 293000fe06cc
Removing intermediate container 293000fe06cc
 ---> 5e3372b2e5cd
Step 7/11 : RUN RELEASE_VERSION=${RELEASE_VERSION} make build-scheduler.$ARCH
 ---> Running in a18224485db4
�[91mmake: *** No rule to make target 'build-scheduler.amd64'.  Stop.
The command '/bin/sh -c RELEASE_VERSION=${RELEASE_VERSION} make build-scheduler.$ARCH' returned a non-zero code: 2
make: *** [Makefile:60: release-image.amd64] Error 2
ERROR: (gcloud.builds.submit) build 99bd2d78-f27f-4b00-b5b4-146452508512 completed with status "FAILURE"
ERROR
ERROR: build step 0 "gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4" failed: step exited with non-zero status: 2
�[0m
--------------------------------------------------------------------------------


2021/03/08 21:37:57 Failed to run some build jobs: [error running [gcloud builds submit --verbosity info --config /home/prow/go/src/github.com/kubernetes-sigs/scheduler-plugins/cloudbuild.yaml --substitutions _PULL_BASE_REF=master,_GIT_TAG=v20210308-v0.18.800-118-g29da026 --project k8s-staging-scheduler-plugins --gcs-log-dir gs://k8s-staging-scheduler-plugins-gcb/logs --gcs-source-staging-dir gs://k8s-staging-scheduler-plugins-gcb/source gs://k8s-staging-scheduler-plugins-gcb/source/42f716dd-89b1-49df-aa2b-23a1213512a8.tgz]: exit status 1]

/reopen

k8s-ci-robot · 2021-03-08T22:06:47Z

@seanmalloy: Reopened this issue.

In response to this:

Well ... it almost works. :-)

Log for the automated container image build job:

Running...
$ARTIFACTS is set, sending logs to /logs/artifacts
2021/03/08 21:37:06 Build directory: .
2021/03/08 21:37:06 Config directory: /home/prow/go/src/github.com/kubernetes-sigs/scheduler-plugins
2021/03/08 21:37:06 cd-ing to build directory: .
2021/03/08 21:37:06 Creating source tarball at /tmp/146047716...
2021/03/08 21:37:06 Uploading /tmp/146047716 to gs://k8s-staging-scheduler-plugins-gcb/source/42f716dd-89b1-49df-aa2b-23a1213512a8.tgz...
Copying file:///tmp/146047716 [Content-Type=application/octet-stream]...
/ [0 files][    0.0 B/  2.1 MiB]                                                
/ [1 files][  2.1 MiB/  2.1 MiB]                                                
Operation completed over 1 objects/2.1 MiB.                                      
2021/03/08 21:37:08 Running build jobs...
2021/03/08 21:37:08 No variants.yaml, starting single build job...
Created [https://cloudbuild.googleapis.com/v1/projects/k8s-staging-scheduler-plugins/builds/99bd2d78-f27f-4b00-b5b4-146452508512].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/99bd2d78-f27f-4b00-b5b4-146452508512?project=96918712006].
----------------------------- REMOTE BUILD OUTPUT ------------------------------
starting build "99bd2d78-f27f-4b00-b5b4-146452508512"

FETCHSOURCE
Fetching storage object: gs://k8s-staging-scheduler-plugins-gcb/source/1615239428.87-112fee7d095c4d96af755e700e60eb25.tgz#1615239429025509
Copying gs://k8s-staging-scheduler-plugins-gcb/source/1615239428.87-112fee7d095c4d96af755e700e60eb25.tgz#1615239429025509...
/ [0 files][    0.0 B/  2.0 MiB]                                                
/ [1 files][  2.0 MiB/  2.0 MiB]                                                
Operation completed over 1 objects/2.0 MiB.                                      
BUILD
Pulling image: gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
v20190906-745fed4: Pulling from k8s-testimages/gcb-docker-gcloud
9d48c3bd43c5: Pulling fs layer
7f94eaf8af20: Pulling fs layer
9fe9984849c1: Pulling fs layer
cf0db633a67d: Pulling fs layer
0f7136d71739: Pulling fs layer
36b51914f992: Pulling fs layer
b5f72b4b9a52: Pulling fs layer
cf0db633a67d: Waiting
0f7136d71739: Waiting
36b51914f992: Waiting
b5f72b4b9a52: Waiting
7f94eaf8af20: Download complete
9fe9984849c1: Verifying Checksum
9fe9984849c1: Download complete
0f7136d71739: Verifying Checksum
0f7136d71739: Download complete
9d48c3bd43c5: Download complete
36b51914f992: Verifying Checksum
36b51914f992: Download complete
9d48c3bd43c5: Pull complete
7f94eaf8af20: Pull complete
9fe9984849c1: Pull complete
cf0db633a67d: Verifying Checksum
cf0db633a67d: Download complete
b5f72b4b9a52: Verifying Checksum
b5f72b4b9a52: Download complete
cf0db633a67d: Pull complete
0f7136d71739: Pull complete
36b51914f992: Pull complete
b5f72b4b9a52: Pull complete
Digest: sha256:f17f1d2a58b8273e184a7c9f89a91d0193a6f458167891115868781610467ddd
Status: Downloaded newer image for gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4
rm -rf ./bin
docker build -f ./build/scheduler/Dockerfile --build-arg ARCH="amd64" --build-arg RELEASE_VERSION="v20210308-v0.18.800-118-g29da026" -t gcr.io/k8s-staging-scheduler-plugins/kube-scheduler:v20210308-v0.18.800-118-g29da026-amd64 .
Sending build context to Docker daemon  3.047MB

Step 1/11 : ARG ARCH
Step 2/11 : FROM golang:1.15.8
1.15.8: Pulling from library/golang
0ecb575e629c: Already exists
7467d1831b69: Already exists
feab2c490a3c: Already exists
f15a0f46f8c3: Already exists
2ea92ed63b96: Pulling fs layer
223c5fc7af76: Pulling fs layer
764ae8cdbfbc: Pulling fs layer
764ae8cdbfbc: Download complete
2ea92ed63b96: Verifying Checksum
2ea92ed63b96: Download complete
223c5fc7af76: Verifying Checksum
223c5fc7af76: Download complete
2ea92ed63b96: Pull complete
223c5fc7af76: Pull complete
764ae8cdbfbc: Pull complete
Digest: sha256:acbe8c73a42159154c1eecc28472926e94f5a7f4cd790655d15dc6b6fbe46c92
Status: Downloaded newer image for golang:1.15.8
---> 7185d074e387
Step 3/11 : WORKDIR /go/src/sigs.k8s.io/scheduler-plugins
---> Running in e8973e2ddaa4
Removing intermediate container e8973e2ddaa4
---> 55d3adb106b0
Step 4/11 : COPY . .
---> 6249893cbc2f
Step 5/11 : ARG ARCH
---> Running in a5c8b206f40f
Removing intermediate container a5c8b206f40f
---> 97a132381121
Step 6/11 : ARG RELEASE_VERSION
---> Running in 293000fe06cc
Removing intermediate container 293000fe06cc
---> 5e3372b2e5cd
Step 7/11 : RUN RELEASE_VERSION=${RELEASE_VERSION} make build-scheduler.$ARCH
---> Running in a18224485db4
�[91mmake: *** No rule to make target 'build-scheduler.amd64'.  Stop.
The command '/bin/sh -c RELEASE_VERSION=${RELEASE_VERSION} make build-scheduler.$ARCH' returned a non-zero code: 2
make: *** [Makefile:60: release-image.amd64] Error 2
ERROR: (gcloud.builds.submit) build 99bd2d78-f27f-4b00-b5b4-146452508512 completed with status "FAILURE"
ERROR
ERROR: build step 0 "gcr.io/k8s-testimages/gcb-docker-gcloud:v20190906-745fed4" failed: step exited with non-zero status: 2
�[0m
--------------------------------------------------------------------------------


2021/03/08 21:37:57 Failed to run some build jobs: [error running [gcloud builds submit --verbosity info --config /home/prow/go/src/github.com/kubernetes-sigs/scheduler-plugins/cloudbuild.yaml --substitutions _PULL_BASE_REF=master,_GIT_TAG=v20210308-v0.18.800-118-g29da026 --project k8s-staging-scheduler-plugins --gcs-log-dir gs://k8s-staging-scheduler-plugins-gcb/logs --gcs-source-staging-dir gs://k8s-staging-scheduler-plugins-gcb/source gs://k8s-staging-scheduler-plugins-gcb/source/42f716dd-89b1-49df-aa2b-23a1213512a8.tgz]: exit status 1]

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

seanmalloy · 2021-03-09T03:22:41Z

Confirmed working. The AMD64 and ARM64 container images are being built and successfully pushed to the staging registry.

…mage [KNI] update base image from ubi8 to ubi9

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Nov 24, 2020

k8s-ci-robot assigned seanmalloy Dec 3, 2020

seanmalloy mentioned this issue Dec 12, 2020

Add Support For Container Image Multi-Arch Builds #133

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 3, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 3, 2021

k8s-ci-robot closed this as completed in #133 Mar 8, 2021

k8s-ci-robot reopened this Mar 8, 2021

seanmalloy mentioned this issue Mar 8, 2021

Revert "Simplify Multi Arch Build Targets" #159

Merged

k8s-ci-robot closed this as completed in #159 Mar 8, 2021

swatisehgal pushed a commit to swatisehgal/scheduler-plugins that referenced this issue Jul 31, 2023

Merge pull request kubernetes-sigs#113 from swatisehgal/update-base-i…

b7eb347

…mage [KNI] update base image from ubi8 to ubi9

This was referenced May 22, 2024

Add support for ppc64le and s390x architectures #745

Open

Add support for ppc64le and s390x architectures and distroless builds #746

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARM Container Images #113

ARM Container Images #113

seanmalloy commented Nov 24, 2020

seanmalloy commented Nov 24, 2020

Huang-Wei commented Nov 24, 2020

seanmalloy commented Nov 24, 2020 •

edited

Loading

AlexeyPerevalov commented Nov 25, 2020

seanmalloy commented Dec 3, 2020

fejta-bot commented Mar 3, 2021

seanmalloy commented Mar 3, 2021

seanmalloy commented Mar 8, 2021

k8s-ci-robot commented Mar 8, 2021

seanmalloy commented Mar 9, 2021

ARM Container Images #113

ARM Container Images #113

Comments

seanmalloy commented Nov 24, 2020

seanmalloy commented Nov 24, 2020

Huang-Wei commented Nov 24, 2020

seanmalloy commented Nov 24, 2020 • edited Loading

AlexeyPerevalov commented Nov 25, 2020

seanmalloy commented Dec 3, 2020

fejta-bot commented Mar 3, 2021

seanmalloy commented Mar 3, 2021

seanmalloy commented Mar 8, 2021

k8s-ci-robot commented Mar 8, 2021

seanmalloy commented Mar 9, 2021

seanmalloy commented Nov 24, 2020 •

edited

Loading