connectors-ci: wrap airbyte-ci in dagger run

[alafanechere]

What

Closes #28690
The Dagger CLI offers a nice Terminal UI for Dagger pipeline execution.
To use this Terminal UI user should wrap pipeline execution with dagger run airbyte-ci.
We don't want our user to care about the Dagger CLI installation and to remind wrapping the CLI with dagger run.

How

• Create a airbyte-ci-bare command: this is our original, unwrapped airbyte-ci command
• Declare airbyte-ci command as the call of the main function of the dagger_run.py module. It
  • Installs the correct Dagger CLI version according to the SDK version
  • Run dagger run airbyte-ci-bare + arguments in a subprocess
• Make the CI use airbyte-ci-bare to not alter the CI behavior we've had so far.

🚨 User Impact 🚨

Users should benefit from this by updating their airbyte-ci installation with : pipx install airbyte-ci/connectors/pipelines

REVIEWERS PLEASE TRY THIS AT HOME

[cpdeethree]

Nice

[bnchrch]

One question!

[bnchrch]

Love that were brining in dagger run.

One question though why use airbyte-ci-bare and no airbyte-ci --disable-dagger-run ...?

Seems like that is possible and better ergonomics for users.

[alafanechere]

But we need an executable that dagger run calls right?

[alafanechere]

rewording: We can indeed have airbyte-ci --disable-dagger-run easily. But I'm not sure we can easily get rid of a second executable. The airbyte-ci-bare use won't be documented...

[alafanechere]

@bnchrch I'll try some things out to check how we could not rely on a second executable.

[bnchrch]

It wasn't too strong of a suggestion.

Also I think I could've worded it better as discouraging the direct use of airbyte-ci-bare by

1. updating our CI tools to use airbyte-ci --disable-dagger-run as the prefered pattern
2. updating the airbyte-ci code to check for the --disable-dagger-run flag

For example

import argparse
import sys

def run_command(command_list: List[str]) -> None:
    try:
        subprocess.run(command_list + sys.argv[1:], check=True)
    except subprocess.CalledProcessError as e:
        sys.exit(e.returncode)

parser = argparse.ArgumentParser()
parser.add_argument('--disable-dagger-run', type=bool, default=False)
def main():
    args = parser.parse_args()
    command_to_run = ["airbyte-ci-internal"]
    if not args.disable_dagger_run:
        command_to_run = ["dagger", "run"] + command_to_run
        check_dagger_cli_install()

    run_command(command_to_run)
    ```

[alafanechere]

I renamed airbyte-ci-bare to airbyte-ci-internal.
You can use airbyte-ci --no-tui to disable the terminal UI (and directly use airbyte-ci-internal).

[alafanechere]

@bnchrch I'll keep running airbyte-ci-internal in the GHA action to prevent any regression