Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programmatically define multiple AML workspaces. #57

Open
albscui opened this issue Sep 10, 2021 · 0 comments
Open

Programmatically define multiple AML workspaces. #57

albscui opened this issue Sep 10, 2021 · 0 comments

Comments

@albscui
Copy link
Contributor

albscui commented Sep 10, 2021

Problem: how do we enable users to have multiple Azure ML workspaces, but talk to a single Pachyderm datastore?

We want to create a new syncer for each AML workspace.

Solution:

two scenarios for existing pachyderm cluster

  1. Pachyderm cluster was created outside of the current Terraform context, and the user can provide their existing workspaces
  • add a variable to indicate existing pachyderm cluster
  • use this variable to avoid creating a new pachyderm cluster
  • user would have to provide their own kubeconfig and copy it to the new syncer
export TF_VAR_existing_resource_group_name="resources-8cfbb924"
export TF_VAR_existing_pachyderm_cluster_name="pachyderm-8cfbb924"
bash scripts/setup.sh
  1. Pachyderm cluster was created inside the current Terraform context
  • In this case, we only need to create a new syncer per workspace.
  • We should configure Terraform to be able to create N syncers given N workspaces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant