Add CLI interface for managing Dagster Cloud variables#28
Add CLI interface for managing Dagster Cloud variables#28stevenayers wants to merge 1 commit intodagster-io:mainfrom
Conversation
|
Amazing! May I ask if there is going to be an idempotent "create-or-update" operation? |
Hi @danielgafni! Because you can have multiple variables with the same name, and there can be many depending on the scopes, locations and deployments set on each one, this means that there is no user-friendly unique identifier for a variable. This means that creating a create-or-update operation out of the box would be tricky and potentially quite inflexible. You would need to specify the exact combination of name and deployment,scopes and locations your variable is associated with, while also specifying the new ones. That's a lot of parameters! My advice would be use the CLI wrapped in a bit of shell script conditional logic if you want to perform a create or update. These CLI commands should provide you with all the building blocks you need to do so :) ID=$(dagster-cloud variable get-id FOO_ENV_ID --full-deployment --location cookie_monster)
if [ $? -ne 0 ]; then
# if we failed to get the ID, create a new variable
dagster-cloud variable create FOO_ENV_ID --full-deployment --location cookie_monster --value BAR
else
# otherwise update the existing one
dagster-cloud variable update $ID --value "new_value"
fi
|
|
I'm mostly thinking about CI/CD pipelines. In this context the deployment name and the code location name is already known and used as arguments (or environment variables) for other It would be convenient if there was a straightforward way to just run one |
That does make sense, I'm just not sure how best to implement it considering how variables are implemented in the graphql API. Would you be willing to take a look at the code in the PR so far and share a suggestion on the logic? |
|
Was this feature ever added? I have the same use case, tried using the new dg cli as documented here, but it requires browser authentication, not suited for CI/CD environments |
|
Looks like there is a |
3 participants
Add CLI interface for managing Dagster Cloud variables
General
% dagster-cloud variable -h Usage: dagster-cloud variable [OPTIONS] COMMAND [ARGS]... Commands for managing Dagster Cloud variables. ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit. │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ create Create a new variable. │ │ delete Delete a variable. │ │ describe Get details about a specific variable. │ │ get-id Get only the ID of a specific variable, useful for update and delete operations. │ │ get-value Get only the value of a specific variable, useful for scripting. │ │ list List all variables in your deployment. │ │ update Update an existing variable. │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Usage Notes
As you can have variables with the same name but different locations, scopes and deployments:
This avoids having to use complex filtering for CRUD operations which are prone to human error.
Example: