feat: Feature flag to disable calls to Hub and discovery.yml outside of add and discover

[edgarrmondragon]

Closes #7095

• Tests
• Docs

[netlify]

✅ Deploy Preview for meltano canceled.

Name	Link
🔨 Latest commit	47455c8
🔍 Latest deploy log	https://app.netlify.com/sites/meltano/deploys/6437f77539ce520008de8602

[aaronsteers]

@edgarrmondragon - I think we have to hold this until 3.0, if it would break existing users (and there are a lot) who have not yet locked their plugin files. Can you confirm?

[edgarrmondragon]

@edgarrmondragon - I think we have to hold this until 3.0, if it would break existing users (and there are a lot) who have not yet locked their plugin files. Can you confirm?

@aaronsteers yes, invoking a plugin that isn't locked would result in a failure with this approach

[edgarrmondragon]

@aaronsteers Perhaps this can be introduced earlier behind a feature flag?

[aaronsteers]

@aaronsteers Perhaps this can be introduced earlier with a feature flag?

Great idea!

Maybe something like require_plugin_lock_files or plugin_locks_required?

I also considered a generic for:

• feature: Add setting to block all external network communication #6754

As someone noted in Slack, a possible mechanism for this could be forcing hub_api_root and discovery_url both to be false.

• https://docs.meltano.com/reference/settings#hub_api_root
• https://docs.meltano.com/reference/settings#discovery_url

That generic solve to restrict network connectivity would break new plugin additions though, so I ruled it out as a good solve here. The ideal state in 3.0, I think, is that you wouldn't be able to run plugins unless they had a local definition file. Making an opt-in flag now for this requirement is nice, because we can start recommending this flag as a best practice, and when we get close to 3.0 we can recommend it as a pre-upgrade step.

[WillDaSilva]

Perhaps this can be introduced earlier behind a feature flag?

Sounds good to me.

because we can start recommending this flag as a best practice, and when we get close to 3.0 we can recommend it as a pre-upgrade step.

@edgarrmondragon @aaronsteers We could also make not having it enabled a lint issue:

• feat: CLI command meltano lint #7285

[codecov]

Codecov Report

Merging #7097 (76b163f) into main (b2a13c3) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 76b163f differs from pull request most recent head 47455c8. Consider uploading reports for the commit 47455c8 to get more accurate results

@@            Coverage Diff             @@
##             main    #7097      +/-   ##
==========================================
+ Coverage   89.38%   89.40%   +0.02%     
==========================================
  Files         294      294              
  Lines       21658    21684      +26     
  Branches     2422     2426       +4     
==========================================
+ Hits        19358    19387      +29     
+ Misses       1939     1938       -1     
+ Partials      361      359       -2     

Impacted Files	Coverage Δ
src/meltano/cli/lock.py	91.52% <100.00%> (+0.14%)	⬆️
src/meltano/core/project_add_service.py	95.12% <100.00%> (ø)
src/meltano/core/project_plugins_service.py	92.51% <100.00%> (+1.50%)	⬆️
src/meltano/core/settings_service.py	97.87% <100.00%> (+0.43%)	⬆️
src/meltano/core/utils/__init__.py	92.36% <100.00%> (-0.03%)	⬇️
tests/meltano/cli/test_add.py	95.76% <100.00%> (ø)
tests/meltano/core/test_project_add_service.py	95.94% <100.00%> (ø)
tests/meltano/core/test_project_plugins_service.py	100.00% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[aaronsteers]

Not a high priority per se, but I've updated the PR description to reflect the feature flag-based scope discussed above, which enables this to ship before 3.0 without breaking existing users.

[tayloramurphy]

Minor docs change, otherwise approving.

[WillDaSilva]

This is great! I have one request, but aside from that this looks good to go