OAuth implementation testing spec

[sherifnada]

Tell us about the problem you're trying to solve

We need to have a way for verifying that oauth implementations with our Airbyte apps are working correctly. We'll need integration tests which, given Airbyte's app ID /client secret for each connector, perform an auth flow and verify it succeeds.

Describe the solution you’d like

An integration test for each connector which verifies that given valid oauth seed credentials, an auth flow happens successfully.

This would probably need to be an E2E UI test?

The initial output of this ticket should be a short RFC describing how we should setup tests which guarantee that our oauth implementations are working.

[sherifnada]

@jamakase does this make sense to have as a UI test?

[Phlair]

After spending some time on this, I'm fairly confident that automated CI testing OAuth flows will be impossible (or if not impossible, then a constant battle of cat and mouse).

Google (and I'm sure other platforms) recognise bot usage (either pure programmatic requests or Selenium) and request a captcha to sign in.

Even if we can trick the provider into thinking our ci bot is human, it would only be a matter of time before that exploit is patched and tests begin failing (hence cat and mouse). I'm not even confident we could trick it with our tests running from EC2 instances, giving services an easy indication that this could be a bot based on their IP.

Unless anyone sees something I'm missing, I'm ruling out bot-driven E2E for this and re-thinking with different approaches.

[Phlair]

Had an idea about using saved serialised cookies so we're already logged in and therefore a bot doesn't have to go through that flow. We don't care about testing the provider's login flow so skipping this seems fine. Going to high-level investigate feasibility of this, if it looks good then probably will form basis of approach in RFC.

Expected possible issue: regen of these cookies will (probably) need to be manual. If time to expiration is short, this could mean manual regen every time we run integration tests for the connector.

[sherifnada]

It might help to break down our nebulous goal of "make sure oauth works" to components. Holding fixed the oauth-backed integration we're testing our goals could be:

1. We redirect the user to the right consent URL (i.e: we use the "correct" client ID)
2. We can successfully use the auth code returned from clicking "allow" in the URL from 1 to generate a refresh token using the same client ID in step 1
3. We use the same client ID/secret in step 1 and 2 and refresh token from step 2 to generate an access token

I think the very hard one to do is 2 so we may just want to skip it. We might be able to get "good enough" results with testing 1 and 3. The trick will then be: how can we set this up such that a developer who is adding or modifying the oauth-based connector can get their changes all the way into Cloud prod with confidence? We might want to add per-connector acceptance tests that are run when releasing a connector into cloud which verify the above assertions

It would probably really help to have a real secret store backing oauth in the server as well, because then we can read the client ID/secret actually in production. But it's not a requirement