Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add env as first-class attribute of MeltanoFile and ProjectPlugin classes (not an extra) #6207

Closed
kgpayne opened this issue Jun 15, 2022 · 3 comments · Fixed by #6267
Closed

Comments

@kgpayne
Copy link
Contributor

kgpayne commented Jun 15, 2022

Note: env is already a first-class attribute of the Environment class.
Note: Just adding these will not make inheritance work (as per #5982 and #5983).

Spec:

  • env is a first-class attribute of MeltanoFile accessible via top-level env: key in meltano.yml)
  • env is a first-class attribute of ProjectPlugin accessible via plugin-scoped env: keys in base and environment-specific config
  • env is settable via the CLI, including with the '.' syntax (e.g. meltano --no-environment config meltano set env.MY_ENV_VAR value)
  • New attributes are added to jsonschema, tested and documented

Release:

As both #5982 and #5983 depend on an env attribute, they should include these changes in their PR to master. This issue will be closed when either of those issues is merged to main.

@kgpayne kgpayne self-assigned this Jun 15, 2022
@kgpayne kgpayne added this to the June 2022 milestone Jun 15, 2022
@aaronsteers
Copy link
Contributor

aaronsteers commented Jun 15, 2022

@kgpayne - We've closed #5969 in favor of this issue. Can you include the docs as part of this?

(I'm a fan of "docs driven development", so writing docs first might not be a bad idea. 😉 )

@z3z1ma
Copy link
Contributor

z3z1ma commented Jun 24, 2022

@kgpayne Is this the idea?

just putting some lame pseudo-code yaml to illustrate the question

env:
  DATABASE: COOL_DB
  DB_PASS: ${FROM_SHELL_ENV_VAR}
  
plugin:
  transformer:
    config:
      database: ${DATABASE}
      password: ${DB_PASS}
  loader:
    config:
      target_db: ${DATABASE}
      postgres_pass: ${DB_PASS}

@tayloramurphy
Copy link
Collaborator

@z3z1ma yes and you'd be able to set env at the plugin level as well. This is basically taking https://docs.meltano.com/concepts/environments#the-env-mapping and bringing it to all levels of your meltano.yml.

kgpayne pushed a commit that referenced this issue Jul 6, 2022
…and ProjectPlugin classes (not an extra)" (#6267)

Co-authored-by: Amanda Folson <afolson@users.noreply.github.com>
BuzzCutNorman pushed a commit to BuzzCutNorman/meltano that referenced this issue Jul 6, 2022
…noFile and ProjectPlugin classes (not an extra)" (meltano#6267)

Co-authored-by: Amanda Folson <afolson@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment