bug: meltano run doesn't support --environment argument

[seajhawk]

Meltano Version

2.2.0

Python Version

3.9

Bug scope

CLI (options, error messages, logging, etc.)

Operating System

Windows 11

Description

This command fails:
meltano --environment dev run tap-csv target-postgres

Due to some other limitations in meltano, I have to create multiple environments in my meltano.yml. Therefore, I would like to be able to call meltano run without having to change the env environment variable each time, and it seems I should be able to specify the --environment argument, but it fails with this error:

> meltano --environment dev-id run tap-csv target-postgres
meltano.core.environment.EnvironmentPluginConfig() argument after ** must be a mapping, not str

Looking at meltano --help, it appears this should work:

Options:
  --log-level [debug|info|warning|error|critical]
  --log-config TEXT               Path to a python logging yaml config file.
  -v, --verbose                   Not used.
  --environment TEXT              Meltano environment name.
  --no-environment                Don't use any environment.
  --version                       Show the version and exit.
  --help                          Show this message and exit.

Code

No response

[seajhawk]

This was an issue with my meltano.yml, not the CLI.

though it would have been helpful if the error message provided more useful information.

[edgarrmondragon]

@seajhawk thanks for reporting. You're right, that error is caused by an unexpected data structure, but the message is not too helpful.

Perhaps having something like #2964 could help.

[seajhawk]

@edgarrmondragon - I appreciate you triaging my issue even though it was closed.
Yes, I agree that the validation is super important. However, that may require you to have schema validation for each of the underlying plugins... hmmm...

Here are a couple of recommendations:

1. implement the validation as a warning first (don't fail). There may be some things that your schema validation would fail on, but are working today. Erroring on new validation errors should be considered a breaking change.
2. Validate opportunistically. You can obviously dictate the overall schema of the file, but I believe it will be impossible to dictate the schema of the plugins. Therefore, the schema validation should support a plugin model where each of the taps/targets can provide a schema validation file and if present the schema validation process will use it. If such file is not present for a tap/target then the schema validation process will ignore that portion of the meltano.yml file.
3. Validate fully. The validation process should validate the entire file and then report all of the issues found rather than erroring and quitting on the first issue.
4. meltano cli override. The cli needs a flag to either skip validation or just ignore any errors. As this is an open source project, it's entirely likely a plugin contributor could make a mistake in validation schema and it should be super easy for someone running the CLI to ignore errors. Perhaps the best bet to handle Authentication for Meltano Analyze #1 and Document how to use extractors on their own #4 is a new argument --validation with possible values of "disabled" and "strict" where

• by default if the argument is not used, validation runs and issues are reported as warnings, but the process continues
• "disabled" - skips the validation entirely (doesn't run, no output)
• "strict" - issues are reported as errors and the process stops after completing the validation
  This should also be supported in the meltano.yml file as default_validation. I don't want to have to remember every time how to a particular project should be run. However, if I use the cli argument or an env variable, it will override the default behavior specified in the file - I believe this is just how default_environment works.

5. Per plugin override. There also needs to be a way to specify the "--validation" behavior on a per plugin-basis. Just because one plugin has broken validation, I don't want to disable or ignore all other validation. I believe this should be configured in each plugin's section of the meltano.yml.