Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and release kindest/node:VERSION on kubernetes release #197

Open
chuckha opened this issue Jan 2, 2019 · 15 comments
Open

Build and release kindest/node:VERSION on kubernetes release #197

chuckha opened this issue Jan 2, 2019 · 15 comments

Comments

@chuckha
Copy link
Member

@chuckha chuckha commented Jan 2, 2019

Right now the node image that kind uses is published by hand by @BenTheElder or @munnerz. It would be good to get this into the release pipeline somehow so that when a new version of kubernetes is published we also get a new node image for kind to use. The tooling exists, it's just a matter of getting the image pushed to the correct place. See the tooling at https://github.com/kubernetes-sigs/kind/blob/master/hack/build/push-node.sh

For complete context, please see this slack thread.

There are a few approaches discussed in the thread:

  1. Shove this into anago in a similar way that the conformance image works. This is easy but the downside is that anago is getting too big and we shouldn't be adding more stuff to anago. This may become a slippery slope and we don't want anago to grow.

  2. periodically check if the release tags have changed. If they have changed, do a build, otherwise don't do anything.

  3. @BenTheElder has been looking at extending prow to trigger off GCS/GCR pubsub so we can kick off a build after a normal release is finished.

We would like to move this off dockerhub before 1.0, but there isn't really a great place to put it right now. Ideally we could have a CNCF sponsored gcr.io bucket so that google isn't responsible for the storage.

@BenTheElder BenTheElder self-assigned this Jan 3, 2019
@BenTheElder BenTheElder added this to To do in 1.0 via automation Jan 3, 2019
@BenTheElder BenTheElder added this to the 1.0 milestone Jan 3, 2019
@BenTheElder BenTheElder removed this from the 1.0 milestone Jan 3, 2019
@BenTheElder BenTheElder added this to the 2018 Goals milestone Jan 3, 2019
@BenTheElder BenTheElder removed this from the 2018 Goals milestone Jan 3, 2019
@BenTheElder BenTheElder added this to the 1.0 milestone Jan 3, 2019
@neolit123
Copy link
Member

@neolit123 neolit123 commented Jan 3, 2019

i guess, i'm only -1 only on the anago approach.

We would like to move this off dockerhub before 1.0, but there isn't really a great place to put it right now. Ideally we could have a CNCF sponsored gcr.io bucket so that google isn't responsible for the storage.

this overlaps with the work by the k8s-infra-team, so if they manage to do the above soon the kind image can be on CNCF ground, otherwise dockerhub seems like a viable option for 1.0.

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Jan 3, 2019

Ditto w/ @neolit123 on anago per our previous discussion, I don't think Kubernetes the core project should be in the business of releasing SIG subprojects just yet, we can ensure we publish images for each release by other means.

For now if anyone really wants them today, they can check out a k8s release tag and build an "unnoficial image" with the same tools we use.

I'm also ambivalent on dockerhub, there haven't been major downsides so far but I'd be happy to move it to whatever hosting k8s-infra-team settles on going forward. The current dockerhub was indeed a stopgap to have joint access with @munnerz but has worked fine.

@chuckha
Copy link
Member Author

@chuckha chuckha commented Jan 3, 2019

cross posting a few communication links

email to steering committee about this
https://groups.google.com/a/kubernetes.io/forum/#!topic/steering/ASZGGcvJQts

CNCF slack discussion about this
https://cloud-native.slack.com/archives/C08PSKWPN/p1546470742035300

k8s-infra slack message about this
https://kubernetes.slack.com/archives/CCK68P2Q2/p1546528406005400

tracking issue
kubernetes/k8s.io#158

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Jan 16, 2019

@munnerz and I have been looking into this more. We think we can set up jobs based on kubernetes/kubernetes git tags to publish new images.

@fejta-bot
Copy link

@fejta-bot fejta-bot commented Apr 28, 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Apr 30, 2019

/remove-lifecycle stale

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Jun 26, 2019

we're at least not too far behind doing it manually, the current pattern is to publish a suite of images to go with each kind release, and avoid pushing much otherwise so users stick to stable image + kind combos

obviously this is not what we wanted here, but it's doing okay as a stopgap

for 0.4.0 releasing ~now we have:

  • kindest/node:v1.15.0@sha256:b4d092fd2b507843dd096fe6c85d06a27a0cbd740a0b32a880fe61aba24bb478
  • kindest/node:v1.14.3@sha256:583166c121482848cd6509fbac525dd62d503c52a84ff45c338ee7e8b5cfe114
  • kindest/node:v1.13.7@sha256:f3f1cfc2318d1eb88d91253a9c5fa45f6e9121b6b1e65aea6c7ef59f1549aaaf
  • kindest/node:v1.12.9@sha256:bcb79eb3cd6550c1ba9584ce57c832dcd6e442913678d2785307a7ad9addc029
  • kindest/node:v1.11.10@sha256:176845d919899daef63d0dbd1cf62f79902c38b8d2a86e5fa041e491ab795d33

@aojea
Copy link
Contributor

@aojea aojea commented Aug 2, 2019

Im publishing an unofficial image built using kind master and kubernetes master nightly

The image is in dockerhub aojea/kindnode:latest https://cloud.docker.com/repository/docker/aojea/kindnode

This is the repo that kicks off the job in circle ci https://github.com/aojea/kind-images

Example of a successful job https://circleci.com/gh/aojea/kind-images/19

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Dec 10, 2019

FWIW the v1.17.0 kind node image released within ~minutes of the kubernetes release (not automated yet, just bentheelder-bot watching the release ;-))

@drewwells
Copy link

@drewwells drewwells commented Apr 24, 2020

v1.15.7 is the latest vailable, v1.15.10 is not available. I don't see a clear way to even help make this image available.

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Apr 24, 2020

https://kind.sigs.k8s.io/docs/user/quick-start/#building-images

we currently more or less only publish "official" kind images with each kind release.
this is because we still need to make breaking changes sometimes (right now related to #148).

you can build your own images, which ideally are built and consumed by the same kind version currently.

I'm going to overhaul building images in the future. #148 is the current priority.

@mflendrich
Copy link

@mflendrich mflendrich commented Jan 28, 2021

I'd like to request availability of kindest/node images for pre-release versions of Kubernetes (alpha, beta).

We intend to utilize kind for regression testing against multiple versions of Kubernetes. Being able to include the upcoming Kubernetes version (which is available as an alpha/beta build) in our tests would shorten our feedback loop by weeks/months.

cc @shaneutt

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Jan 28, 2021

Currently you'll need to build your own, we have documentation for this in user guide.
In the near future we will offer builds that don't require building Kubernetes from source but will result in slightly larger images.

mflendrich added a commit to Kong/kubernetes-ingress-controller that referenced this issue Jan 29, 2021
Makes it possible to specify the version of Kubernetes running KIC's integration test.

The choice is limited to available tags of the `docker/io/kindest/node` docker registry (which doesn't cover all releases at this moment). Asked `kind` maintainers (kubernetes-sigs/kind#197 (comment)) to increase coverage (specifically: start providing images for pre-release k8s builds as well)
@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Mar 11, 2021

I expect to propose a concrete plan for this, #381, #166, and #1895 in the next month.

@BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Mar 11, 2021

We intend to utilize kind for regression testing against multiple versions of Kubernetes. Being able to include the upcoming Kubernetes version (which is available as an alpha/beta build) in our tests would shorten our feedback loop by weeks/months.

For this one specifically there's the additional constraint that using kind with versions of kubernetes that released after kind released may not work.

While Kubernetes shies away from making breaking end-user changes without a clear migration path over multiple releases, Kubernetes / kubeadm do have [Action Required] changes in a single release sometimes (e.g. kubernetes/kubeadm#2376 landing in Kubernetes v1.21.0).

Because of this we must test Kubernetes at HEAD with kind from HEAD containing mitigations for these pre-release changes. You could use kind from HEAD yourself but currently it may also have breaking changes between releases and not yet have a detailed migration path etc.

This is a trick problem that may improve (e.g. perhaps kind will be more complete and release smaller bugfixes more often as we won't be in the middle of larger changes) but currently it somewhat limits the usefulness of us pre-building these versions. #381 will at the very least make it cheaper to try without us having published one anyhow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
1.0
  
To do
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants