Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Ignore default environment for some commands #6862

Merged
merged 6 commits into from Oct 12, 2022

Conversation

WillDaSilva
Copy link
Member

Given the urgency of #6677, and the relative ease of #6834, I decided to quickly put together an implementation.

In this implementation, commands which require an environment call activate_environment, and those which will only use an environment if it is explicitly provided (e.g. via --environment) call activate_explicitly_provided_environment. By default no environment is required nor used, which comes with the side benefit of avoiding the "default environment activated" log message for most commands.

Closes #6834

@netlify
Copy link

netlify bot commented Oct 7, 2022

Deploy Preview for meltano ready!

Name Link
🔨 Latest commit 5a669b7
🔍 Latest deploy log https://app.netlify.com/sites/meltano/deploys/63462d995906fa0008924483
😎 Deploy Preview https://deploy-preview-6862--meltano.netlify.app/getting-started/installation
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@WillDaSilva WillDaSilva marked this pull request as ready for review October 7, 2022 03:52
@codecov
Copy link

codecov bot commented Oct 7, 2022

Codecov Report

Merging #6862 (5a669b7) into main (a12adf6) will increase coverage by 0.02%.
The diff coverage is 96.15%.

@@            Coverage Diff             @@
##             main    #6862      +/-   ##
==========================================
+ Coverage   88.68%   88.71%   +0.02%     
==========================================
  Files         283      283              
  Lines       20405    20419      +14     
  Branches     2010     2011       +1     
==========================================
+ Hits        18097    18114      +17     
+ Misses       1972     1970       -2     
+ Partials      336      335       -1     
Impacted Files Coverage Δ
tests/meltano/cli/test_cli.py 100.00% <ø> (ø)
src/meltano/cli/cli.py 85.91% <83.33%> (-1.19%) ⬇️
src/meltano/cli/__init__.py 78.72% <100.00%> (-1.28%) ⬇️
src/meltano/cli/config.py 70.64% <100.00%> (-0.27%) ⬇️
src/meltano/cli/elt.py 94.97% <100.00%> (+0.02%) ⬆️
src/meltano/cli/invoke.py 90.09% <100.00%> (+0.09%) ⬆️
src/meltano/cli/job.py 70.73% <100.00%> (+0.23%) ⬆️
src/meltano/cli/run.py 86.25% <100.00%> (+1.25%) ⬆️
src/meltano/cli/schedule.py 71.79% <100.00%> (+0.18%) ⬆️
src/meltano/cli/select.py 51.56% <100.00%> (+0.76%) ⬆️
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tayloramurphy
Copy link
Collaborator

@WillDaSilva love this PR coming through. We'll need to update the docs in the CLI reference section to make it explicit about which ones require an environment or not.

Maybe outside the scope of this PR, but I'm wondering if some short of small table at the start would be useful where we list attributes of the command such as "Uses default_environment", "Edits meltano.yml", etc.

At a minimum though we should update the text of every command to have a one-liner on whether or not default_environment is used.

@aaronsteers what else would you like to see here?

@aaronsteers
Copy link
Contributor

aaronsteers commented Oct 11, 2022

@WillDaSilva - This looks really great and I like the design choices you made here with the two functions handling the two use cases.

@tayloramurphy - I just spent some time in the docs, and writing suitable copy doesn't seem like a trivial task. In many commands, an immediate mentioning of environment-based usage does not look like it would work with the exisitng context.

I did find an example under invoke which could be adapted:

image

But that phrasing is not ideal.

What about these proposals, coming after each command's "How to Use" subsection:

Case 1 (Environment always ignored):

Using add with Environments

The add command does not run relative to a Meltano Environment. The --environment flag and default_environment setting in your meltano.yml file will be ignored if set.

Case 2 (Get only from --environment):

Using config with Environments

The config command can accept the --environment flag to target a specific Meltano Environment. However, the default_environment setting in your meltano.yml file will be ignored.

Case 3 (Get from --environment or default_environment):

Using invoke with Environments

The invoke command can accept the --environment flag to target a specific Meltano Environment. The default_environment setting in your meltano.yml file will be applied if --environment is not provided explicitly.

Case 4: (Environment always required)

Using run with Environments

The run command always requires a Meltano Environment to be set. The environment name can be provided using the --environment flag or with the default_environment setting in your meltano.yml file.

@tayloramurphy - Feel free to make any changes you'd like to see. But to minimize the number of iterations here in the code itself, would be good to have copy that @WillDaSilva can insert directly.

@WillDaSilva - I don't know if run requires an enviornment as of now, but if not, can we implement that in this PR or should it be saved for a later iteration? Also, feel free to tweak my wording above or propose alternatives.

@WillDaSilva
Copy link
Member Author

I don't know if run requires an enviornment as of now, but if not, can we implement that in this PR or should it be saved for a later iteration?

@aaronsteers Implemented in 7baebb8 - meltano run will now error if there is no environment.

@tayloramurphy
Copy link
Collaborator

@WillDaSilva @aaronsteers I've updated AJ's comment with my preferred text. Thanks for well thought out prompts @aaronsteers !

Copy link
Contributor

@aaronsteers aaronsteers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noyce. 👍

@WillDaSilva WillDaSilva merged commit d1decf6 into main Oct 12, 2022
@WillDaSilva WillDaSilva deleted the ignore-default-environment branch October 12, 2022 15:33
WillDaSilva added a commit that referenced this pull request Dec 6, 2022
In #6862 we defered the activate of environments. The `Tracker` was implicitly relying on the environment being activated before it was instantiated, or no environment being activated at all.

Unfortunately fixing that implicit dependency would be more effort than I'm willing to commit to this at this time. Instead, I have made it so that activating an environment will update `environment_name_hash` in the `ProjectContext` in the `Tracker`.

New `ProjectContext` objects created past that point will, as before, get the active environment information from the `Project` object.

I have verified that this works using Snowplow Micro locally. Before this change `environment_name_hash` was always `null`. After this change, `environment_name_hash` is set to the expected hashed string when an environment is active, and `null` when no environment is active. I would encourage at least one reviewer to double check this.

Closes meltano/internal-data#51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ignore default Meltano Environment on non-execution CLI commands
3 participants