UX Issue: Challenging to authenticate against OAuth sources on PyAirbyte

[aaronsteers]

Background

There are some sources which only support OAuth, for instance the Amazon Seller Partner source connector.

I'm logging this issue for tracking and discussion around options to improve the user experience for PyAirbyte users who want to leverage these sources.

Exploring Options

Option A: Use alternative auth methods (workaround)

For some sources, it could make sense to use another auth method, when available.

However, some sources, such as the "Amazon Seller Partner" source noted above, don't have alternatives.

Option B: Obtain OAuth refresh token and client secret manually or via another means (workaround)

There are some alternative methods to collect refresh tokens and/or auth tokens needed for the OAuth flow. These are often cumbersome and challenging to implement.

Option C: PyAirbyte could add an "OAuth Flow" feature

While this sounds like an obvious addition, it has some challenges, such as needing to host a webserver for the callback function, and the challenge of doing this in a way that is both secure and OS-agnostic. Firewalls and NAT issues could prevent this from being viable in many cases - and many runtimes could be more restrictive and not permitting of the flow. (Others may be able to speak to limitations, and paths through those, better than I can.)

Option D: Leverage a 3rd-party OAuth service, such as Airbyte Cloud API

There may be other options to facilitate this process, which I'm not aware of at the time of this writing. In theory, the Airbyte API has an endpoint to initialize OAuth for a source. My understanding is that this API only stores the creds in Cloud, and requires an Airbyte Cloud account or Airbyte OSS instance running.

Update:

I ran this through ChatGPT for due diligence.