Regression tests: run in GitHub Actions

[clnoll]

What

Adds the ability to run regression tests manually in GitHub Actions.

How

1. Adds a new workflow for regression tests that takes the connector and connection ID as input.
2. Updates the regression tests to remove auto_select_connection as an input option. Instead, we auto-select if the connection ID is "auto". This removes one degree of freedom which was complicating the connection retrieval logic.

Note -
Using the connection-retriever required some setup that isn't entirely obvious in this PR. This includes

• Accessing the airbyte-platform-internal repo during the build time of the regression test container. Locally we can use ssh to give authorized users the ability to download it, but in CI we need to use https and therefore had to modify live tests' pyproject.toml and configure the pyproject.toml with the appropriate credentials at build time.
• Authenticating with gcloud involved creating a new service account and key, GCP_INTEGRATION_TESTER_CREDENTIALS, stored in github secrets.
• This service account had to be given several permissions (in prod-ab-cloud-proj and ab-analytics).
• To connect to postgres, we needed to authenticate by running Google's Cloud SQL Auth Proxy. This allows us to authenticate without manually managing IP whitelists.

Depends on https://github.com/airbytehq/airbyte-platform-internal/pull/12286.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Apr 30, 2024 4:33am

[alafanechere]

I don't have anything blocking to raise.
I'd be curious to have more details about the need of a Google Cloud SQL proxy.
I'd also like to understand why you removed the --auto-select-connection flag. (does it require a README.md update?)

And as usual: please bump packages version and update changelogs :D

[alafanechere]

You might get a very cheap "run on multiple connections" feature if you'd use a matrix strategy here:

• Make the connection_id inputs a comma separated list
• Parse this list into an array
• Run this job for each connection id

[clnoll]

Nice, I'm going to add this as a TODO in a comment since it requires a little bit of extra testing.

[alafanechere]

Did you make sure this refacto works locally too? 🙏

[clnoll]

Yep I did, and am going to do a bit more manual testing before merge. Might be time to start thinking about adding more unit tests 😄 .

[alafanechere]

Why are you getting rid of this explicit flag and decided the go implicit with connection_id == 'auto'?

[clnoll]

Since we should only have one of auto-select or connection ID, we don't really need both flags and it creates more surface area for bugs or unexpected behavior. For example, when I passed in a connection ID via gha I couldn't find a way to turn off auto-select since pytest doesn't parse falsey values. I could have changed auto-select to default to false, but then we'd still end up in a situation where users could provide inputs that don't make sense when combined. So I decided to remove the degree of freedom.

[alafanechere]

Running it with Dagger would prevent you from managing the service lifecycle, it would shutdown if it's not used

[alafanechere]

I find it the perfect use case for dagger services :D google distributes a docker image for it:
https://cloud.google.com/sql/docs/postgres/sql-proxy#cloud-sql-auth-proxy-docker-image

[clnoll]

Nice, I've added that link. Since the existing code is working I'm going to leave it as-is for now but will circle back to try this out if time permits.