Feature spec discussion: proper env: {}
support at multiple levels
#6206
aaronsteers
announced in
Spec Discussions
Replies: 1 comment
-
@aaronsteers is this discussion still necessary with the work @cjohnhanson and @kgpayne have done? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Creating this discussion to confirm the spec we are planning to build.
I want to confirm the below are correct in terms of desired behavior, and that having
env: {}
formally supported at each of these resolution points makes sense in terms of our desired functionality.The red box and grey boxes are not technically supported, but users may have used them with unpredictable/unexpected results, as @pnadolny13 discovered in our Meltano
squared
repo.Implementation Points
Among other things, this would change the
expand_env_vars()
function behavior inmeltano.core.utils
.Sample use case 1 - top-level env support, overridable and inheritable at lower levels
As a user, I want to set
env.SNOWFLAKE_ACCOUNT_NAME
at a single point, as a top-level constant in mymeltano.yml
file.This then gets referenced as
$SNOWFLAKE_ACCOUNT_NAME
in 5 plugins:target-snowflake
,dbt-snowflake
,tap-snowflake
,great-expectations
, andsuperset
.If I need to override the
SNOWFLAKE_ACCOUNT_NAME
for a single environment (such ascicd
), I would be able to do so also, using and environment-levelenv: {}
declaration which would override the top-level one.Sample use case 2 - env vars referencing other environment variables
As a user, I may want to have
SQLACHEMY_URL
also inherit and derive from$SNOWFLAKE_ACCOUNT_NAME
.Beta Was this translation helpful? Give feedback.
All reactions