New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm should use multi-architecture images when GCR supports Docker distribution API v2 schema 2 #51

Closed
mikedanese opened this Issue Nov 22, 2016 · 12 comments

Comments

@mikedanese
Member

mikedanese commented Nov 22, 2016

From @errordeveloper on October 3, 2016 11:19

cc kubernetes/kubernetes#33916

Copied from original issue: kubernetes/kubernetes#33917

@luxas

This comment has been minimized.

Member

luxas commented Nov 25, 2016

I've been told v2 schema 2 will be rolled out to gcr.io in mid-December, so we can probably target this v1.6

@luxas luxas self-assigned this Nov 25, 2016

@luxas luxas added this to the v1.6 milestone Nov 25, 2016

@luxas luxas removed this from the v1.6 milestone Feb 16, 2017

@sudhagarc

This comment has been minimized.

sudhagarc commented Apr 22, 2017

I'm new to k8 & stuck/blocked by this issue. I'm using kubeadm v1.6.1. Is the issue addressed post that? Alternatively, if there is a manual way to solve it, I appreciate the instructions.

@codesnk

This comment has been minimized.

codesnk commented May 3, 2017

Hi all,
Any updates on this? My cluster (Linux hosts & VMs) works fine till I try to join a Pi3 in the mix. The issue is that the Master (Ubuntu) downloads amd64 images for kube-proxy and weave net on the Pi node. I thought this shouldn't happen as the master labels the node as arm architecture.

@jamiehannaford

This comment has been minimized.

Member

jamiehannaford commented Oct 9, 2017

@luxas So this is just a case of swapping out the arch type in the URL? If so, I can have a go at this

@jamiehannaford

This comment has been minimized.

Member

jamiehannaford commented Oct 9, 2017

Oh it seems that we already do this.

@errordeveloper Can I close this?

@luxas

This comment has been minimized.

Member

luxas commented Oct 20, 2017

This is about k8s pushing manifest lists docker-library/official-images#2289 for all control plane images and kubeadm starting to consume them. We do the ARCH suffixing now, but that's bad. Currently this is stuck on GCR.io not supporting the Docker Registry API v2 schema 2, but hopefully they will in some month

@vielmetti

This comment has been minimized.

vielmetti commented Dec 13, 2017

GCR.io now supports the new schema and multiarch images and thus this work can be evaluated to proceed.

@aitorhh

This comment has been minimized.

aitorhh commented Dec 14, 2017

I have being using a multi-platform cluster since last May using the docker registry API v2 and the manifest-tool to push the images to my own private repository.

In kubeadm (v1.6.1), I only needed the following change (extracted from a patch file):

---
 cmd/kubeadm/app/images/images.go | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/cmd/kubeadm/app/images/images.go b/cmd/kubeadm/app/images/images.go
index 3f714c9..b0cbfc7 100644
--- a/cmd/kubeadm/app/images/images.go
+++ b/cmd/kubeadm/app/images/images.go
@@ -18,7 +18,6 @@ package images
 
 import (
        "fmt"
-       "runtime"
 
        kubeadmapi "k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm"
 )
@@ -39,10 +38,10 @@ func GetCoreImage(image string, cfg *kubeadmapi.MasterConfiguration, overrideIma
        }
        repoPrefix := kubeadmapi.GlobalEnvParams.RepositoryPrefix
        return map[string]string{
-               KubeEtcdImage:              fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "etcd", runtime.GOARCH, etcdVersion),
-               KubeAPIServerImage:         fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-apiserver", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeControllerManagerImage: fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-controller-manager", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeSchedulerImage:         fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-scheduler", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeProxyImage:             fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-proxy", runtime.GOARCH, cfg.KubernetesVersion),
+               KubeEtcdImage:              fmt.Sprintf("%s/%s:%s", repoPrefix, "etcd", etcdVersion),
+               KubeAPIServerImage:         fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-apiserver", cfg.KubernetesVersion),
+               KubeControllerManagerImage: fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-controller-manager", cfg.KubernetesVersion),
+               KubeSchedulerImage:         fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-scheduler", cfg.KubernetesVersion),
+               KubeProxyImage:             fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-proxy", cfg.KubernetesVersion),
        }[image]
 }
-- 

Then, it's just a matter of setting the KUBE_REPO_PREFIX

KUBE_REPO_PREFIX=<docker-registry-v2> /usr/local/bin/kubeadm

If the multi-platform images are already available, and can make a PR.

@luxas

This comment has been minimized.

Member

luxas commented Dec 14, 2017

We still need to do something like this: kubernetes/release#248

If you @aitorhh could open a similar PR that is up-to-date with that functionality and sync with @ixdy to get it merged soon-ish, I'd be more than happy!

We also need to coordinate/sync this up with kubernetes/kubernetes#54174

@timothysc

This comment has been minimized.

Member

timothysc commented Apr 7, 2018

/assign @detiber

@timothysc timothysc removed the triaged label Apr 7, 2018

@timothysc timothysc added this to the v1.11 milestone Apr 7, 2018

@luxas luxas removed this from the v1.11 milestone May 14, 2018

@neolit123

This comment has been minimized.

Member

neolit123 commented Aug 6, 2018

/assign
/kind documentation

for docs update:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue Aug 28, 2018

Merge pull request #66960 from rosti/drop-arch-suffixes
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubeadm: Drop arch suffixes

**What this PR does / why we need it**:

This change removes arch suffixes from control plane images (etcd, kube-apiserver, kube-scheduler, kube-proxy, etc.). These are not needed, as almost all control plane images have a fat manifest now. It also adds a missing unit test for `GetGenericImage`

We have arch suffixes only for kube-dns images now.

Sample output of `kubeadm config images pull` with this change:

```
$ ./_output/bin/kubeadm config images pull
[config/images] Pulled k8s.gcr.io/kube-apiserver:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-scheduler:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-proxy:v1.11.1
[config/images] Pulled k8s.gcr.io/pause:3.1
[config/images] Pulled k8s.gcr.io/etcd:3.2.18
[config/images] Pulled k8s.gcr.io/coredns:1.1.3
$ ./_output/bin/kubeadm config images pull --feature-gates=CoreDNS=false
[config/images] Pulled k8s.gcr.io/kube-apiserver:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-scheduler:v1.11.1
[config/images] Pulled k8s.gcr.io/kube-proxy:v1.11.1
[config/images] Pulled k8s.gcr.io/pause:3.1
[config/images] Pulled k8s.gcr.io/etcd:3.2.18
[config/images] Pulled k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10
[config/images] Pulled k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
[config/images] Pulled k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Refs kubernetes/kubeadm#1030 kubernetes/kubeadm#51

**Special notes for your reviewer**:
/cc @kubernetes/sig-cluster-lifecycle-pr-reviews
/area kubeadm
/kind feature
/assign @luxas
/assign @timothysc
/assign @dims 
/assign @neolit123 

**Release note**:

```release-note
kubeadm: Control plane images (etcd, kube-apiserver, kube-proxy, etc.) don't use arch suffixes. Arch suffixes are kept for kube-dns only.
```
@timothysc

This comment has been minimized.

Member

timothysc commented Oct 6, 2018

This is done!

@timothysc timothysc closed this Oct 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment