Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubeflow Runner/BaseComponent dont force inject GCP credentials/secret into pods #201

Closed
rummens opened this issue Jun 5, 2019 · 1 comment
Assignees

Comments

@rummens
Copy link
Contributor

rummens commented Jun 5, 2019

Description

As many people we don’t deploy Kubeflow on GCP, therefore we don’t need/have GCP cred. Unfortunately the GCP env vars and more annoyingly the secret is hardcoded into the BaseComponent. Therefore no component pod can be started without the required secret (named user-gcp-sa per default).

Possible Solution

Please make this mount optionally, controlled via an option given to the runner.
This can go in hand with a better documentation of the additional arguments of the Pipeline class.
A possible solution is to create an overall platform parameter, similar to the solution in the Pipelines example:

The relevant code is around line 124 in tfx.orchestration.kubeflow.base_component. I am happy to assist if needed.

Workaround

Deploy a fake secret:

apiVersion: v1
kind: Secret
metadata:
  name: user-gcp-sa
type: Opaque
data:
  user-gcp-sa.json: YXBpVXJsOiAiaHR0cHM6Ly9teS5hcGkuY29tL2FwaS92MSIKdXNlcm5hbWU6IHt7dXNlcm5hbWV9fQpwYXNzd29yZDoge3twYXNzd29yZH19

@rummens rummens changed the title Kubeflow Runner dont force inject GCP credentials into pods Kubeflow Runner dont force inject GCP credentials/secret into pods Jun 5, 2019
@rummens rummens changed the title Kubeflow Runner dont force inject GCP credentials/secret into pods Kubeflow Runner/BaseComponent dont force inject GCP credentials/secret into pods Jun 5, 2019
@neuromage
Copy link

We'll make this optional. I'll send out a fix shortly.

@zhitaoli zhitaoli assigned zhitaoli and neuromage and unassigned zhitaoli Jun 5, 2019
tfx-copybara pushed a commit that referenced this issue Jun 5, 2019
KubeflowConfigRunner. This enables users to perform operations suchs as
specifying custom k8s secrets or mounting volumes when running TFX pipelines
under Kubeflow.

Fixes #201 and also solves the issue in PR #202.

PiperOrigin-RevId: 251677229
tfx-copybara pushed a commit that referenced this issue Jun 6, 2019
KubeflowConfigRunner. This enables users to perform operations suchs as
specifying custom k8s secrets or mounting volumes when running TFX pipelines
under Kubeflow.

Fixes #201 and also solves the issue in PR #202.

PiperOrigin-RevId: 251677229
tfx-copybara pushed a commit that referenced this issue Jun 7, 2019
KubeflowConfigRunner. This enables users to perform operations suchs as
specifying custom k8s secrets or mounting volumes when running TFX pipelines
under Kubeflow.

Fixes #201 and also solves the issue in PR #202.

PiperOrigin-RevId: 251677229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants