Docs should show how to pull a particular release #524

jlewi · 2018-03-28T20:56:09Z

Update the docs so that we explain to users how to pull a particular version of the ksonnet configs.

Related to

#506 Cut a 0.1 release
#215 Minimal release process

This change is

jlewi · 2018-03-28T21:03:01Z

/assign @willb

jlewi · 2018-03-28T21:46:20Z

/test all

jlewi · 2018-03-28T21:50:15Z

Here was the first failure

+ /usr/local/bin/checkout.sh /src
+ SRC_DIR=/src
+ mkdir -p /src/kubeflow
+ git clone https://github.com/kubeflow/kubeflow.git /src/kubeflow/kubeflow
Cloning into '/src/kubeflow/kubeflow'...
+ echo Job Name = kubeflow-presubmit
+ REPO_DIR=/src/kubeflow/kubeflow
+ cd /src/kubeflow/kubeflow
Job Name = kubeflow-presubmit
+ '[' '!' -z 524 ']'
+ git fetch origin pull/524/head:pr
From https://github.com/kubeflow/kubeflow
 * [new ref]         refs/pull/524/head -> pr
+ '[' '!' -z 7d9c2633d7ddf2a1957e685f9e7f287340be0e34 ']'
+ git checkout 7d9c2633d7ddf2a1957e685f9e7f287340be0e34
fatal: reference is not a tree: 7d9c2633d7ddf2a1957e685f9e7f287340be0e34

The ref is somehow missing the commit.

ankushagarwal · 2018-03-29T02:03:33Z

I think there is a bug in ksonnet.

VERSION=v0.1.0-rc.0
ks registry add kubeflow github.com/kubeflow/kubeflow/tree/${VERSION}/kubeflow
ks pkg install kubeflow/core

These instructions seem to install the kubeflow/core from master branch rather than branch v0.1.0-rc.0

ankushagarwal · 2018-03-29T02:16:30Z

ksonnet/ksonnet#398

jlewi · 2018-03-29T03:10:40Z

The following seems to work

ks pkg install kubeflow/tf-job@${VERSION}

at least based on app.yaml.

kkasravi · 2018-03-29T06:43:12Z

@jlewi @ankushagarwal FYI I have a bash script called kf. I can slack it to you although it's being converted to python as part of the tooling proposal which i'll be submitting this week. It creates things correctly including enabling IAP as shown in the output below. It will also set up soft links if the repo is a fork and cloned locally as described in the developer_guide.md (which isn't done below since we're just using a tagged version from the kubeflow/kubeflow repo). This script may be helpful since I've created kubeflow namespaces literally several hundred times as part of baking the script, including IAP enabled ones.
Note the hostname below when ks generates iap-ingress which you can try logging into rather than dev.kubeflow.com. You should get the IAP auth challenge and be redirected to jupyterhub w/ no challenge since it's using the IAPAuthenticator.

BTW @ankushagarwal iap.md is a great write up 👍

kf create -h

kf create [-h] [-d] [-r repo] [-b branch] [-i] namespace
 -h prints help info
 -d launches jupyterHubImage with "bash sleep inf"
 -r use a different github org/repo (default kubeflow/kubeflow)
 -b use a different github branch (default master)
 -i enable iap

 Create will create a kubernetes namespace and populate it with functionality to process ml models.

Below is the output of the command.
Note: kf -V will echo all commands and I've filtered out ks INFO statements.

$ ☞  kf -V create -b v0.1.0-rc.0 -i kam-kf-22
kubectl create ns kam-kf-22
namespace "kam-kf-22" created
kubectl config set-context gke_constant-cubist-173123_us-west1-a_dls-projects --namespace=kam-kf-22
Context "gke_constant-cubist-173123_us-west1-a_dls-projects" modified.
ks init kam-kf-22 --api-spec=version:v1.8.0
ks registry add kubeflow github.com/kubeflow/kubeflow/tree/v0.1.0-rc.0/kubeflow
ks pkg install kubeflow/core@v0.1.0-rc.0
ks pkg install kubeflow/argo@v0.1.0-rc.0
ks prototype use io.ksonnet.pkg.argo argo --namespace kam-kf-22 --name argo
ks generate core kubeflow-core --name=kubeflow-core --namespace=kam-kf-22
ks env add cloud
ks param set kubeflow-core cloud gke --env=cloud
ks apply cloud -c kubeflow-core
gcloud compute --project=constant-cubist-173123 addresses create kam-kf-22 --global
Created [https://www.googleapis.com/compute/v1/projects/constant-cubist-173123/global/addresses/kam-kf-22].
ks generate cert-manager cert-manager --namespace=kam-kf-22 --acmeEmail=kamkasravi@gmail.com
ks apply cloud -c cert-manager
ks generate iap-ingress iap-ingress --ipName=kam-kf-22 --namespace=kam-kf-22 --hostname=kamkasravi.com --clientID=XXXX --clientSecret=XXXX
ks apply cloud -c iap-ingress
ks param set kubeflow-core jupyterHubAuthenticator iap
ks apply cloud -c kubeflow-core
kubectl delete -n kam-kf-22 pods tf-hub-0
pod "tf-hub-0" deleted

Here are the resources created which show envoy, whoami and others as part of IAP

USER       NAME                              RESOURCE
kam-kf-22  envoy-config                      ConfigMap
kam-kf-22  jupyterhub-config                 ConfigMap
kam-kf-22  tf-job-operator-config            ConfigMap
kam-kf-22  ambassador                        Deployment
kam-kf-22  cert-manager                      Deployment
kam-kf-22  envoy                             Deployment
kam-kf-22  tf-job-operator                   Deployment
kam-kf-22  whoami-app                        Deployment
kam-kf-22  ambassador                        Endpoints
kam-kf-22  ambassador-admin                  Endpoints
kam-kf-22  envoy                             Endpoints
kam-kf-22  k8s-dashboard                     Endpoints
kam-kf-22  tf-hub-0                          Endpoints
kam-kf-22  tf-hub-lb                         Endpoints
kam-kf-22  whoami-app                        Endpoints
kam-kf-22  envoy-ingress                     Ingress
kam-kf-22  ambassador-5b74cd9c6b-928xr       Pod
kam-kf-22  ambassador-5b74cd9c6b-bnqcz       Pod
kam-kf-22  ambassador-5b74cd9c6b-zb4zm       Pod
kam-kf-22  cert-manager-64d8456756-n294m     Pod
kam-kf-22  envoy-869cbc7b9f-h8bbl            Pod
kam-kf-22  envoy-869cbc7b9f-tmsvk            Pod
kam-kf-22  envoy-869cbc7b9f-vwvxn            Pod
kam-kf-22  tf-hub-0                          Pod
kam-kf-22  tf-job-operator-5765ff7ccc-c4vzl  Pod
kam-kf-22  whoami-app-6d9d8dc867-gntzl       Pod
kam-kf-22  ambassador-5b74cd9c6b             ReplicaSet
kam-kf-22  cert-manager-64d8456756           ReplicaSet
kam-kf-22  envoy-869cbc7b9f                  ReplicaSet
kam-kf-22  tf-job-operator-5765ff7ccc        ReplicaSet
kam-kf-22  whoami-app-6d9d8dc867             ReplicaSet
kam-kf-22  ambassador                        ServiceAccount
kam-kf-22  cert-manager                      ServiceAccount
kam-kf-22  default                           ServiceAccount
kam-kf-22  envoy                             ServiceAccount
kam-kf-22  jupyter-hub                       ServiceAccount
kam-kf-22  tf-job-operator                   ServiceAccount
kam-kf-22  ambassador-token-6t25g            Secret
kam-kf-22  cert-manager-token-k624c          Secret
kam-kf-22  default-token-lbkm7               Secret
kam-kf-22  envoy-ingress-tls                 Secret
kam-kf-22  envoy-token-wchv5                 Secret
kam-kf-22  jupyter-hub-token-rd2kv           Secret
kam-kf-22  letsencrypt-prod-secret           Secret
kam-kf-22  tf-job-operator-token-qksjt       Secret
kam-kf-22  ambassador                        Service
kam-kf-22  ambassador-admin                  Service
kam-kf-22  envoy                             Service
kam-kf-22  k8s-dashboard                     Service
kam-kf-22  tf-hub-0                          Service
kam-kf-22  tf-hub-lb                         Service
kam-kf-22  whoami-app                        Service
kam-kf-22  tf-hub                            StatefulSet

kkasravi · 2018-03-29T09:21:44Z

Update - after rebasing from upstream/master i'm not able to go the iap domain - i noticed that centralui was added when i rebased - maybe envoy ingress is not redirecting correctly?
This is from master - not the release tag so it may not be relevant.

jlewi · 2018-03-29T12:30:29Z

@kkasravi #525 (Central Dashboard) didn't make any changes to Ambassador routing so I'm not sure why it would have affected IAP. Can you open an issue if you're having problems with IAP.

inc0 · 2018-03-29T15:42:19Z

README.md

+# Which version of Kubeflow to use
+# For a list of releases refer to:
+# https://github.com/kubeflow/kubeflow/releases
+VERSION=v0.1.0-rc.0


Do we want to set rc.0 if this is last thing to do before actual 0.1.0?:) Also we should document how to use master besides pinned version.

I wanted to put a tag that exists. We can update it in a subsequent change once we have an actual release.

willb · 2018-03-29T18:31:59Z

@jlewi these instructions work for me and install the correct images (using the @${VERSION} syntax for ks pkg install). Minor nit: why rc.0 instead of rc0? Also, I agree with @inc0 that documenting how to use master is a good idea. (Release branches should show how to use the release in question first though!)

/lgtm

jlewi · 2018-03-30T20:58:21Z

K8s does rc.0
https://github.com/kubernetes/kubernetes/releases/tag/v1.10.0-rc.1
https://github.com/kubernetes/kubernetes/releases?after=v1.10.0-beta.2

I suspect its so that its easy to parse the "tag" (rc/alpha/beta) and the number.

jlewi · 2018-03-30T23:18:44Z

I thought about adding instructions to pull from master.
I'm hesitant to include that in the docs. It might make sense in the developer_guide but its less clear to me why we would want to encourage users to pull from head rather than using one of the releases.

jlewi · 2018-03-30T23:18:49Z

/approve

jlewi · 2018-04-02T13:00:14Z

/retest

ankushagarwal · 2018-04-02T23:31:26Z

/lgtm

k8s-ci-robot · 2018-04-02T23:31:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ankushagarwal, jlewi, willb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ankushagarwal,jlewi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Update the docs to explain how to pin to a particular version. * Create a new commit to see about the presubmits. * Version needs to be included in the pkg install step.

k8s-ci-robot requested review from wbuchwalter and zjj2wry March 28, 2018 20:56

k8s-ci-robot added the size/XL label Mar 28, 2018

jlewi closed this Mar 28, 2018

jlewi force-pushed the docs branch from 97fa30a to c0f9b49 Compare March 28, 2018 20:57

k8s-ci-robot added approved size/XS and removed size/XL labels Mar 28, 2018

Update the docs to explain how to pin to a particular version.

7d9c263

jlewi reopened this Mar 28, 2018

k8s-ci-robot added size/S and removed approved size/XS labels Mar 28, 2018

k8s-ci-robot assigned willb Mar 28, 2018

jlewi mentioned this pull request Mar 28, 2018

checkout.sh doesn't work with branches kubeflow/testing#89

Closed

Create a new commit to see about the presubmits.

0af4f8c

Version needs to be included in the pkg install step.

db0c361

jlewi mentioned this pull request Mar 29, 2018

Make it easy to get started with Kubeflow #105

Closed

inc0 suggested changes Mar 29, 2018

View reviewed changes

k8s-ci-robot added the lgtm label Mar 29, 2018

k8s-ci-robot added the approved label Mar 30, 2018

Merge remote-tracking branch 'upstream/master' into docs

6c42ef6

k8s-ci-robot removed the lgtm label Apr 2, 2018

k8s-ci-robot assigned ankushagarwal Apr 2, 2018

k8s-ci-robot added the lgtm label Apr 2, 2018

k8s-ci-robot merged commit 20dec55 into kubeflow:master Apr 2, 2018

yanniszark pushed a commit to arrikto/kubeflow that referenced this pull request Feb 15, 2021

Fix ut test and enable ut-test of v1alpha2 (kubeflow#524)

3d1a8ca

elenzio9 pushed a commit to arrikto/kubeflow that referenced this pull request Oct 31, 2022

Transition of Bobgy (kubeflow#524)

3b9e20a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docs should show how to pull a particular release #524

Docs should show how to pull a particular release #524

jlewi commented Mar 28, 2018 •

edited

jlewi commented Mar 28, 2018

jlewi commented Mar 28, 2018

jlewi commented Mar 28, 2018

ankushagarwal commented Mar 29, 2018

ankushagarwal commented Mar 29, 2018

jlewi commented Mar 29, 2018 •

edited

kkasravi commented Mar 29, 2018

kkasravi commented Mar 29, 2018 •

edited

jlewi commented Mar 29, 2018

inc0 Mar 29, 2018

jlewi Mar 30, 2018

willb commented Mar 29, 2018 •

edited

jlewi commented Mar 30, 2018

jlewi commented Mar 30, 2018

jlewi commented Mar 30, 2018

jlewi commented Apr 2, 2018

ankushagarwal commented Apr 2, 2018

k8s-ci-robot commented Apr 2, 2018

Docs should show how to pull a particular release #524

Docs should show how to pull a particular release #524

Conversation

jlewi commented Mar 28, 2018 • edited

jlewi commented Mar 28, 2018

jlewi commented Mar 28, 2018

jlewi commented Mar 28, 2018

ankushagarwal commented Mar 29, 2018

ankushagarwal commented Mar 29, 2018

jlewi commented Mar 29, 2018 • edited

kkasravi commented Mar 29, 2018

kkasravi commented Mar 29, 2018 • edited

jlewi commented Mar 29, 2018

inc0 Mar 29, 2018

Choose a reason for hiding this comment

jlewi Mar 30, 2018

Choose a reason for hiding this comment

willb commented Mar 29, 2018 • edited

jlewi commented Mar 30, 2018

jlewi commented Mar 30, 2018

jlewi commented Mar 30, 2018

jlewi commented Apr 2, 2018

ankushagarwal commented Apr 2, 2018

k8s-ci-robot commented Apr 2, 2018

jlewi commented Mar 28, 2018 •

edited

jlewi commented Mar 29, 2018 •

edited

kkasravi commented Mar 29, 2018 •

edited

willb commented Mar 29, 2018 •

edited