Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial change - add pipeline entry to kubeflow registry #1973

Merged
merged 6 commits into from Nov 20, 2018

Conversation

IronPan
Copy link
Member

@IronPan IronPan commented Nov 20, 2018

Mostly copied from existing registry
https://github.com/kubeflow/pipelines/tree/master/ml-pipeline/ml-pipeline

  • Removed argo from the pipeline package. Rely on kubeflow argo instead
  • Keep Spartakus entry for tracking pipeline specific usage

I'll have a separate PR to integrate the one-click and kfctl deployment


This change is Reviewable

@IronPan
Copy link
Member Author

IronPan commented Nov 20, 2018

/assign @jlewi @kunmingg

@IronPan
Copy link
Member Author

IronPan commented Nov 20, 2018

/assign @Ark-kun

@IronPan
Copy link
Member Author

IronPan commented Nov 20, 2018

The tests failed with formatting error. Is there any tool I can use to format jsonnet code?

@kunmingg
Copy link
Contributor

local pipeline_scheduledworkflow = import "kubeflow/pipeline/pipeline-scheduledworkflow.libsonnet",
local pipeline_persistenceagent = import "kubeflow/pipeline/pipeline-persistenceagent.libsonnet",
local pipeline_ui = import "kubeflow/pipeline/pipeline-ui.libsonnet",
local spartakus = import "kubeflow/pipeline/spartakus.libsonnet",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will pipeline always be installed? If so maybe we can remove it .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to install pipelines by default.
@abhi-g is looking into how we can provide additional information (e.g. about installed components) with spartakus.

So maybe we can remove spartakus for now and look into how to add pipeline specific data to that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to install pipelines by default.
@abhi-g is looking into how we can provide additional information (e.g. about installed components) with spartakus.

So maybe we can remove spartakus for now and look into how to add pipeline specific data to that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG. Removed it.

@kunmingg
Copy link
Contributor

/lgtm

@kunmingg
Copy link
Contributor

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kunmingg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@IronPan
Copy link
Member Author

IronPan commented Nov 20, 2018

/test kubeflow-presubmit

@k8s-ci-robot k8s-ci-robot merged commit 01f4bc2 into kubeflow:master Nov 20, 2018
r2d4 pushed a commit to r2d4/kubeflow that referenced this pull request Nov 26, 2018
* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus
@r2d4 r2d4 mentioned this pull request Nov 26, 2018
k8s-ci-robot pushed a commit that referenced this pull request Nov 26, 2018
* Add IronPan as Argo owner (#1974)

* initial change - add pipeline entry to kubeflow registry (#1973)

* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus

* Add IronPan to pipeline ks package owner (#1980)

* Add IronPan to pipeline ks package owner

* Update OWNERS

* cherry-pick pipeline component onto 0.3-branch

* remove travis config (#1790)
IronPan added a commit to IronPan/kubeflow that referenced this pull request Dec 6, 2018
* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus
k8s-ci-robot pushed a commit that referenced this pull request Dec 6, 2018
* initial change - add pipeline entry to kubeflow registry (#1973)

* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus

* merge 01f4bc2

* merge b5bfe47

* grant KF user SA CMLE admin permission (#2013)

* Activate CMLE during one-click deployment (#2017)

* merge 14ff472

* add permission to get pod's log (#2037)

Pipeline component needs the permission to  get tf serving pod log
https://github.com/kubeflow/pipelines/blob/master/components/kubeflow/deployer/src/deploy.sh#L202

* Add roles/dataproc.editor for kubeflow user account (#2035)

Pipeline needs the dataproc editor permission to run the xgboost dataproc samples
For details, see
https://github.com/kubeflow/pipelines/tree/master/components/dataproc

* Update Pipeline version to v0.1.3 (#2045)

* merge 21ff6ce

* resolve conflict

* Update kf_app.yaml

* add central dashboard

* merge
k8s-ci-robot pushed a commit that referenced this pull request Dec 21, 2018
* initial change - add pipeline entry to kubeflow registry (#1973)

* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus

* merge 01f4bc2

* merge b5bfe47

* grant KF user SA CMLE admin permission (#2013)

* Activate CMLE during one-click deployment (#2017)

* merge 14ff472

* add permission to get pod's log (#2037)

Pipeline component needs the permission to  get tf serving pod log
https://github.com/kubeflow/pipelines/blob/master/components/kubeflow/deployer/src/deploy.sh#L202

* Add roles/dataproc.editor for kubeflow user account (#2035)

Pipeline needs the dataproc editor permission to run the xgboost dataproc samples
For details, see
https://github.com/kubeflow/pipelines/tree/master/components/dataproc

* Update Pipeline version to v0.1.3 (#2045)

* merge 21ff6ce

* resolve conflict

* Update kf_app.yaml

* add central dashboard

* merge

* Add job to load pipeline samples (#2071)

* Add job to load pipeline samples

Adding a job meets the requirement of 
- Load samples once when pipeline is deployed
- Not run job again during kf upgrade
- If loading samples failed, not blocking api server from start

* format

* update pipeline version (#2151)

* update pipeline version

* Update Dockerfile
saffaalvi pushed a commit to StatCan/kubeflow that referenced this pull request Feb 11, 2021
* add pipeline entry

* use camel case for params

* formatting

* format

* fix

* remove spartakus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants