feat: Just-in-time plugin installation

[ReubenFrankel]

Implements just-in-time install for the following commands:

• invoke
• run
• el/elt
• test
• config <plugin> test
• select --list

[netlify]

✅ Deploy Preview for meltano canceled.

Name	Link
🔨 Latest commit	ba9a469
🔍 Latest deploy log	https://app.netlify.com/sites/meltano/deploys/66733bd00eedf80008ae3a41

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.06%. Comparing base (a396e3b) to head (ba9a469).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8482      +/-   ##
==========================================
+ Coverage   92.01%   92.06%   +0.05%     
==========================================
  Files         246      246              
  Lines       19380    19441      +61     
  Branches     3419     3436      +17     
==========================================
+ Hits        17833    17899      +66     
+ Misses       1271     1267       -4     
+ Partials      276      275       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ReubenFrankel]

We probably want to suppress install progress to stdout from meltano.cli.utils.install_plugins here, or emit as logs to stderr... For invoke, this currently breaks the common tap/target debugging technique of

meltano invoke <tap> > tap.out
cat tap.out | meltano invoke <target>

[edgarrmondragon]

We probably want to suppress install progress to stdout from meltano.cli.utils.install_plugins here, or emit as logs to stderr... For invoke, this currently breaks the common tap/target debugging technique of

meltano invoke <tap> > tap.out
cat tap.out | meltano invoke <target>

Yeah, I'd be more than OK with starting to unify our output away from click.echo and into the logging framework.

[ReubenFrankel]

Think I have covered all relevant commands here. Wondering if we want a new --no-install option for these (same convention as for meltano add) to opt-out of auto-install behaviour (e.g. for CI), or stick to the original proposal in #2089:

Interactive (TTY) experience

If a tty is available, we can simply ask the user if they'd like to auto-install the missing plugins.

Updated error message with hints

If a TTY is not available and --install is not specified, then the error message would be changed to something like:

ERROR: Plugin not installed. Please install with meltano install ... or rerun with meltano invoke --install ....

Why require confirmation / opt-in

Install operations are not always safe to run, and they can hide other issues in the invocation or installation, such as using the wrong plugin name or the wrong base image in a scheduled job. They could also trigger unnecessarily for a container-backed spec that doesn't require installation.

I haven't yet addressed install progress/status as logs, but the output is suppressed for meltano invoke for now.

[ReubenFrankel]

Should this be added for select too?

[edgarrmondragon]

Should this be added for select too?

If it's opt-in, probably yes 👍