Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Image Tag management for releases #1060

Closed
jlewi opened this issue Jun 21, 2018 · 5 comments
Closed

Simplify Image Tag management for releases #1060

jlewi opened this issue Jun 21, 2018 · 5 comments

Comments

@jlewi
Copy link
Contributor

jlewi commented Jun 21, 2018

Right now we tag our docker images with v${DATE}-${SHA}. There are two problems with this approach

  1. This format isn't very informative for users

  2. It's really cumbersome to update images as part of an RC

    • We need to build the images
    • Then create a PR to update the images in the ksonnet configs.

I propose we tag images with v${RELEASE} e.g. v0.2.0 and v.0.2.1 and this work as follows

  1. The tags v0.2.0 will be mutable until a release candidate is promoted to actual release

    • As we cut RC's the tag for the upcoming release will be updated to point at the most recent RC
    • Once an RC is promoted to the actual release the tag is no longer mutated
  2. When we cut the release branch; we update all the images to use the tag of the next release e.g. 0.2.1

    • This means we don't have to update the configs each time we cut a new image; we can just move the release tag (v0.2.1) to the new image
  3. To provide an audit trail we use a simple YAML that maps immutable sha based images to tags
    e.g.

images:
   - image: gcr.io/some-image/image:sha256@abcd...
        tags: 
            - gcr.io/some-image/image:v0.2.0
  1. We can use scripts to apply the tags to a given image
  2. To push a new image as part of an RC we just need to move the tags by
    i. updating the YAML file
    ii. rerunning scripts to reapply the tags

/cc @pdmack @kunmingg

@pdmack
Copy link
Member

pdmack commented Jun 21, 2018

This sounds awesome!

jlewi added a commit to jlewi/kubeflow that referenced this issue Jun 21, 2018
* See kubeflow#1060
* Use a YAML file to keep track of a map from images referenced by sha
  to tags to apply

* Add a script to apply the tags as listed in the YAML file.

* This will make it easy to promote images as part of our release process.
@jlewi
Copy link
Contributor Author

jlewi commented Jun 21, 2018

#1061 is an initial stab.

I think what we'll need some scripts to manage that YAML file for example

  • Get the shas for the most recent tensorflow notebook images
  • Apply some release tag to them
  • Remove some release tag from some set of images.

@pdmack
Copy link
Member

pdmack commented Jun 21, 2018

/cc @cheyang

k8s-ci-robot pushed a commit that referenced this issue Jun 21, 2018
…#1061)

* See #1060
* Use a YAML file to keep track of a map from images referenced by sha
  to tags to apply

* Add a script to apply the tags as listed in the YAML file.

* This will make it easy to promote images as part of our release process.
jlewi added a commit to jlewi/kubeflow that referenced this issue Jun 22, 2018
…ypes

* Provide some python scripts to match images and then apply a tag to
  them like "v0.2.0". This makes it possible to easily apply a new release
  tag to set of images like the Jupyter images.

* Creeate a shell script to use sed and other twos to update images
  in our ksonnet prototypes.

* Add instructions for doing this.

* I used the scripts to add the v0.2.0 tag to our Jupyter images.

Related to kubeflow#1060
k8s-ci-robot pushed a commit that referenced this issue Jun 22, 2018
…ypes (#1066)

* Provide some python scripts to match images and then apply a tag to
  them like "v0.2.0". This makes it possible to easily apply a new release
  tag to set of images like the Jupyter images.

* Creeate a shell script to use sed and other twos to update images
  in our ksonnet prototypes.

* Add instructions for doing this.

* I used the scripts to add the v0.2.0 tag to our Jupyter images.

Related to #1060
@jbottum
Copy link
Contributor

jbottum commented Sep 30, 2018

/area 0.4.0

@chrisheecho chrisheecho added this to To do in 0.4.0 Nov 6, 2018
@chrisheecho chrisheecho moved this from To do to Stablization in 0.4.0 Nov 6, 2018
@carmine carmine added this to the 0.4.0 milestone Nov 6, 2018
@jlewi
Copy link
Contributor Author

jlewi commented Nov 26, 2018

Closing this issue since we've committed a bunch of scripts to make this easier and there is no immediate follow on work.

@jlewi jlewi closed this as completed Nov 26, 2018
0.4.0 automation moved this from Kubeflow infra improvement - Doc/stablization to Done Nov 26, 2018
saffaalvi pushed a commit to StatCan/kubeflow that referenced this issue Feb 11, 2021
…kubeflow#1061)

* See kubeflow#1060
* Use a YAML file to keep track of a map from images referenced by sha
  to tags to apply

* Add a script to apply the tags as listed in the YAML file.

* This will make it easy to promote images as part of our release process.
saffaalvi pushed a commit to StatCan/kubeflow that referenced this issue Feb 11, 2021
…ypes (kubeflow#1066)

* Provide some python scripts to match images and then apply a tag to
  them like "v0.2.0". This makes it possible to easily apply a new release
  tag to set of images like the Jupyter images.

* Creeate a shell script to use sed and other twos to update images
  in our ksonnet prototypes.

* Add instructions for doing this.

* I used the scripts to add the v0.2.0 tag to our Jupyter images.

Related to kubeflow#1060
surajkota pushed a commit to surajkota/kubeflow that referenced this issue Jun 13, 2022
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
0.4.0
  
Done
Development

No branches or pull requests

5 participants