Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add env support and remove params from env #231

Merged
merged 11 commits into from
Sep 6, 2023
Merged

Add env support and remove params from env #231

merged 11 commits into from
Sep 6, 2023

Conversation

samos123
Copy link
Contributor

@samos123 samos123 commented Sep 4, 2023

Allow setting environment variables directly from the Substratus resources.

E.g.

spec:
  env:
    MY_KEY: value
    SECRETY_SECRET: ${{ secrets.ai.HUGGING_FACE_HUB_TOKEN }}

This also removes the functionality of Substratus of setting PARAM_ environment variables. So this will require updating our images and possibly examples.

This will make working with Hugging Face tokens easier. This would allow
you to create a secret with the following content:
```
HUGGING_FACE_HUB_TOKEN: my-token
```

Afterwards all Substratus resources will be able to use this
HuggingFace token. This also removes the need for custom entrypoints in
the images that strip PARAM_ from the env variable.
internal/controller/model_controller.go Outdated Show resolved Hide resolved
internal/controller/model_controller.go Outdated Show resolved Hide resolved
internal/controller/model_controller.go Outdated Show resolved Hide resolved
@samos123 samos123 changed the title secrets as env Add ability to set env and remove PARAM_ env variables Sep 5, 2023
@samos123 samos123 changed the title Add ability to set env and remove PARAM_ env variables Introduce env support and remove params from env Sep 5, 2023
@samos123 samos123 changed the title Introduce env support and remove params from env Add env support and remove params from env Sep 5, 2023
api/v1/dataset_types.go Outdated Show resolved Hide resolved
internal/controller/utils.go Outdated Show resolved Hide resolved
@@ -314,9 +314,14 @@ func nbPodName(nb *apiv1.Notebook) string {
}

// notebookPod constructs a Pod for the given Notebook.
func (r *NotebookReconciler) notebookPod(notebook *apiv1.Notebook, model *apiv1.Model, dataset *apiv1.Dataset) (*corev1.Pod, error) {
func (r *NotebookReconciler) notebookPod(ctx context.Context, notebook *apiv1.Notebook, model *apiv1.Model, dataset *apiv1.Dataset) (*corev1.Pod, error) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can remove the ctx here since it's technically not needed anymore after latest refactor. What you think? @nstogner

@samos123
Copy link
Contributor Author

samos123 commented Sep 5, 2023

PR for updating the images: substratusai/images#45 . I will also start updating the examples

@samos123
Copy link
Contributor Author

samos123 commented Sep 5, 2023

Verified on Kind with this model:

apiVersion: substratus.ai/v1
kind: Model
metadata:
  name: llama-2-7b
spec:
  image: substratusai/model-loader-huggingface:pr-45
    #build:
    #  git:
    #    url: https://github.com/substratusai/images
    #    branch: no-param-envs
    #    path: model-loader-huggingface
  env:
    HUGGING_FACE_HUB_TOKEN: ${{ secrets.ai.HUGGING_FACE_HUB_TOKEN }}
  params:
    name: meta-llama/Llama-2-7b-hf

nstogner
nstogner previously approved these changes Sep 5, 2023
Copy link
Contributor

@nstogner nstogner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Some small suggestions, feel free to merge either way

internal/controller/server_controller.go Outdated Show resolved Hide resolved

actual, err := resolveEnv(tc.input)
if err != nil {
t.Errorf("error with case %v: %v", tc.input, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer to use assert or require pkg

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for equality check below

@samos123 samos123 merged commit b8a1fc9 into main Sep 6, 2023
4 checks passed
@samos123 samos123 deleted the secrets-as-env branch September 6, 2023 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants