Migrate Kubeflow from k8s-prow #177

clarketm · 2019-12-04T17:59:21Z

Copied kubeflow configuration from config and plugins and add tide and jobs.

cjwagner · 2019-12-04T22:40:34Z

Needed before this can merge:

Give @google-oss-robot admin access to the Kubeflow org.
Configure webhooks from the Kubeflow org to be sent to this Prow instance.
Copy ProwJob configs from https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubeflow (We need to do this before enabling Tide so that Tide knows all the merge requirements.), Migrate Kubeflow jobs #93
Disable Tide and plugins for the Kubeflow org on the prow.k8s.io instance to avoid conflict/duplication (Migrate Kubeflow kubernetes/test-infra#15607).

Should be done soon (ideally before this merges as well):

Copy pre-existing job results to the oss-prow GCS bucket.
Replace TestGrid dashboards (Migrate Kubeflow kubernetes/test-infra#15607).

This is all I can think of off the top of my head. Lets keep track of all the steps we take so we can leave a guide for transferring service of an org from one Prow instance to another.

jlewi · 2019-12-04T23:44:29Z

I invited google-oss-robot to the kubeflow org; once the invite is accepted the bot should have write access to Kubeflow repositories

Is admin permissions really needed? I'm giving the google-oss-robot the same permissions as k8s-ci-robot so unless something changed hopefully that's sufficient.

What's the URL to use for these webhooks?

Related to: kubernetes/test-infra#14343.

clarketm · 2019-12-06T05:52:36Z

@cjwagner

Copy ProwJob configs from https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubeflow (We need to do this before enabling Tide so that Tide knows all the merge requirements.), Migrate Kubeflow jobs #93

Paradoxically, the jobs (more specifically checkconfig) needs the trigger plugin configured for kubeflow.

https://storage.googleapis.com/oss-prow/pr-logs/pull/GoogleCloudPlatform_oss-test-infra/93/pull-prow-config-validate/1201922395468730368/build-log.txt

cjwagner · 2019-12-06T17:34:32Z

Paradoxically, the jobs (more specifically checkconfig) needs the trigger plugin configured for kubeflow.

You can add the jobs in this PR as well or wait to enable Tide until the jobs are added (trigger > jobs > tide).

fejta · 2019-12-06T22:38:22Z

/hold

jlewi · 2019-12-10T12:34:16Z

Related to:

kubeflow/testing#475 - Kubeflow issue tracking migration to GoogleCloud Prow
kubernetes/test-infra#14343 - Migrate Kubeflow to GoogleCloudProw

jlewi · 2019-12-10T12:37:44Z

google-oss-robot is now a member of the kubeflow GitHub org.

/lgm

@fejta @clarketm what is blocking this PR? Does #93 need to be merged first?

clarketm · 2019-12-11T00:45:12Z

@fejta @clarketm what is blocking this PR? Does #93 need to be merged first?

@jlewi - Apologies for the latency; I have been supporting the Istio 1.4.2 release which just went out today. I need to sync with @cjwagner to disambiguate his last comment.

You can add the jobs in this PR as well or wait to enable Tide until the jobs are added (trigger > jobs > tide).

@cjwagner - if I opt to go for the latter (i.e. wait to enable Tide) this: (trigger > jobs > tide) just means that I can omit the Tide-specific changes from this PR and reintroduce them in a PR after the jobs are merged (#93)?

Should these changes (trigger, tide, jobs) be atomic (i.e. one PR)? How can we prevent both duplicate and dropped webhooks in the transition.

clarketm · 2019-12-12T18:06:47Z

@cjwagner
I disabled the tide configuration as requested. Is this now safe to merge?

clarketm · 2019-12-13T22:38:25Z

@fejta / @cjwagner
I (should) have all the required PRs (I might additionally need something for testgrid?). Please verify and advise on the order and rollout strategy (either here or via this document). Additionally, I need to coordinate a time to do this with @jlewi next week.

fejta

/hold

jlewi · 2019-12-16T20:14:41Z

@clarketm is there anything blocking this PR? Are you just waiting on confirmation from me or the Kubeflow build copy (@richardsliu )?

My expectation is this PR won't actually cause any downtime so its safe to merge.
Downtime should also be fine just coordinate with @richardsliu

googlebot · 2020-03-05T21:04:00Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

clarketm · 2020-03-05T21:28:31Z

@clarketm: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-prow-config-validate 605582e link /test pull-prow-config-validate

This is a warning not an error? So I disabled it in checkconfig (#313).

googlebot · 2020-03-06T00:23:07Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

google-oss-robot · 2020-03-06T00:25:13Z

The following users are mentioned in OWNERS file(s) but are not members of the GoogleCloudPlatform org.

Once all users have been added as members of the org, you can trigger verification by writing /verify-owners in a comment.

abhi-g
- prow/prowjobs/kubeflow/OWNERS
kunmingg
- prow/prowjobs/kubeflow/OWNERS
lluunn
- prow/prowjobs/kubeflow/OWNERS

scottilee · 2020-03-06T07:20:58Z

@googlebot I consent

fejta · 2020-03-10T21:00:37Z

I think someone from kubeflow should do this review

scottilee · 2020-03-15T04:48:34Z

@jlewi can you take a look?

jlewi · 2020-03-15T20:19:07Z

/assign @jlewi

jlewi · 2020-03-15T20:23:26Z

/lgtm
/approve

Thanks @clarketm . My hunch is that some of our more recent changes to our prow jobs may need to be ported over. I think we can do that after this PR is merged though.

When this PR is merged; will our tests be running simultaneously on the GCP prow instance as well as the K8s instance? Can we have some transition period during which we verify everything is working correctly on the new GCP instance before turning down the K8s instance?

google-oss-robot · 2020-03-15T20:23:33Z

@jlewi: changing LGTM is restricted to collaborators

In response to this:

/lgtm
/approve

Thanks @clarketm . My hunch is that some of our more recent changes to our prow jobs may need to be ported over. I think we can do that after this PR is merged though.

When this PR is merged; will our tests be running simultaneously on the GCP prow instance as well as the K8s instance? Can we have some transition period during which we verify everything is working correctly on the new GCP instance before turning down the K8s instance?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-robot · 2020-03-15T20:23:48Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: clarketm, fejta, jlewi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [clarketm,fejta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

clarketm · 2020-03-23T09:58:57Z

Replaced by: kubernetes/test-infra#16898

google-oss-robot requested review from amwat and chases2 December 4, 2019 17:59

google-oss-robot added the approved label Dec 4, 2019

clarketm assigned fejta Dec 4, 2019

google-oss-robot added the size/L label Dec 4, 2019

clarketm assigned cjwagner Dec 4, 2019

fejta approved these changes Dec 6, 2019

View reviewed changes

google-oss-robot added the lgtm label Dec 6, 2019

google-oss-robot added the do-not-merge/hold label Dec 6, 2019

clarketm force-pushed the add-kubeflow branch from d963231 to be37da9 Compare December 12, 2019 18:04

google-oss-robot removed the lgtm label Dec 12, 2019

clarketm requested a review from fejta December 12, 2019 18:07

clarketm removed the do-not-merge/hold label Dec 12, 2019

This was referenced Dec 12, 2019

Migrate Kubeflow jobs #93

Closed

Migrate Kubeflow kubernetes/test-infra#15607

Closed

Add kubeflow enable tide #192

Closed

fejta approved these changes Dec 14, 2019

View reviewed changes

google-oss-robot added do-not-merge/hold lgtm labels Dec 14, 2019

google-oss-robot added size/XL do-not-merge/invalid-owners-file and removed size/M labels Mar 5, 2020

clarketm force-pushed the add-kubeflow branch 3 times, most recently from 581a27d to 605582e Compare March 5, 2020 21:19

google-oss-robot added size/XXL and removed size/XL labels Mar 5, 2020

clarketm force-pushed the add-kubeflow branch from 9cb5492 to 5387bee Compare March 5, 2020 21:41

clarketm mentioned this pull request Mar 5, 2020

checkconfig: exclude warning = non-decorated-jobs #313

Merged

clarketm force-pushed the add-kubeflow branch 2 times, most recently from 2f1e35f to 2901f4b Compare March 6, 2020 00:21

clarketm and others added 2 commits March 5, 2020 16:22

Migrate Kubeflow

df48b1d

sync Kubeflow jobs from k8s

a9d8be5

clarketm force-pushed the add-kubeflow branch from 2901f4b to a9d8be5 Compare March 6, 2020 00:23

google-oss-robot assigned jlewi Mar 15, 2020

scottilee mentioned this pull request Mar 21, 2020

Move kubeflow jobs to https://github.com/GoogleCloudPlatform/oss-test-infra/tree/master/prow/prowjobs kubernetes/test-infra#14343

Closed

clarketm closed this Mar 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate Kubeflow from k8s-prow #177

Migrate Kubeflow from k8s-prow #177

clarketm commented Dec 4, 2019 •

edited

Loading

cjwagner commented Dec 4, 2019 •

edited by clarketm

Loading

jlewi commented Dec 4, 2019

clarketm commented Dec 6, 2019 •

edited

Loading

cjwagner commented Dec 6, 2019

fejta commented Dec 6, 2019

jlewi commented Dec 10, 2019

jlewi commented Dec 10, 2019

clarketm commented Dec 11, 2019 •

edited

Loading

clarketm commented Dec 12, 2019

clarketm commented Dec 13, 2019 •

edited

Loading

fejta left a comment

jlewi commented Dec 16, 2019

googlebot commented Mar 5, 2020

clarketm commented Mar 5, 2020 •

edited

Loading

googlebot commented Mar 6, 2020

google-oss-robot commented Mar 6, 2020

scottilee commented Mar 6, 2020

fejta commented Mar 10, 2020

scottilee commented Mar 15, 2020

jlewi commented Mar 15, 2020

jlewi commented Mar 15, 2020

google-oss-robot commented Mar 15, 2020

google-oss-robot commented Mar 15, 2020

clarketm commented Mar 23, 2020

Migrate Kubeflow from k8s-prow #177

Migrate Kubeflow from k8s-prow #177

Conversation

clarketm commented Dec 4, 2019 • edited Loading

cjwagner commented Dec 4, 2019 • edited by clarketm Loading

jlewi commented Dec 4, 2019

clarketm commented Dec 6, 2019 • edited Loading

cjwagner commented Dec 6, 2019

fejta commented Dec 6, 2019

jlewi commented Dec 10, 2019

jlewi commented Dec 10, 2019

clarketm commented Dec 11, 2019 • edited Loading

clarketm commented Dec 12, 2019

clarketm commented Dec 13, 2019 • edited Loading

fejta left a comment

Choose a reason for hiding this comment

jlewi commented Dec 16, 2019

googlebot commented Mar 5, 2020

clarketm commented Mar 5, 2020 • edited Loading

googlebot commented Mar 6, 2020

google-oss-robot commented Mar 6, 2020

scottilee commented Mar 6, 2020

fejta commented Mar 10, 2020

scottilee commented Mar 15, 2020

jlewi commented Mar 15, 2020

jlewi commented Mar 15, 2020

google-oss-robot commented Mar 15, 2020

google-oss-robot commented Mar 15, 2020

clarketm commented Mar 23, 2020

clarketm commented Dec 4, 2019 •

edited

Loading

cjwagner commented Dec 4, 2019 •

edited by clarketm

Loading

clarketm commented Dec 6, 2019 •

edited

Loading

clarketm commented Dec 11, 2019 •

edited

Loading

clarketm commented Dec 13, 2019 •

edited

Loading

clarketm commented Mar 5, 2020 •

edited

Loading