Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Samples - Moved secret application to the pipeline definition #536

Conversation

Ark-kun
Copy link
Contributor

@Ark-kun Ark-kun commented Dec 13, 2018

P.S. How did samples/tfx/taxi-cab-classification-pipeline.py work if the confusion_matrix_op did not use a secret?


This change is Reviewable

@Ark-kun Ark-kun force-pushed the Samples---Moved-use_gcp_secret-application-to-the-pipeline-definition branch from 4258b02 to 83feffd Compare December 13, 2018 09:00
@IronPan
Copy link
Member

IronPan commented Dec 14, 2018

just wondering why this is better.
not strong opinion on this. LGTM

@IronPan
Copy link
Member

IronPan commented Dec 14, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot removed the lgtm label Dec 14, 2018
@Ark-kun
Copy link
Contributor Author

Ark-kun commented Dec 14, 2018

just wondering why this is better.
not strong opinion on this. LGTM

The component definitions should be shareable/portable. Adding secrets to the definitions ties them to a particular installation so at least the components are portable/shareable.

In future we can try to move this to even higher level (pipeline level or cluster level), so that the pipeline can also be portable.

@hongye-sun
Copy link
Contributor

The implementation of the component depends on a gcp secret. It's better to let component author to declare it. Ideally, user just need to setup the secrete in their pipeline service once and no need to configure any secret in their pipelines.

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Dec 14, 2018

The implementation of the component depends on a gcp secret. It's better to let component author to declare it.

That's true for some components (e.g. CMLE), but most of other components can actually work on files.

Ideally, user just need to setup the secrete in their pipeline service once and no need to configure any secret in their pipelines.

That's the goal, yes.

@qimingj
Copy link
Contributor

qimingj commented Dec 14, 2018

I think it is a good idea since the component code is supposed to be "binary" and not modifiable. Setting credential in component code causes it less portable. I though of doing this when we have a packaging story (yaml).

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Dec 14, 2018

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

@gaoning777 gaoning777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot merged commit dd24c80 into kubeflow:master Dec 14, 2018
@hongye-sun
Copy link
Contributor

One implementation option is that we let component author to put a secret hint in the component spec and it pass through as a label on pod. We can leverage podpreset to mount the GCP secret volume by matching label.

@Ark-kun Ark-kun deleted the Samples---Moved-use_gcp_secret-application-to-the-pipeline-definition branch January 21, 2019 11:19
magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this pull request Oct 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants