Logging to other Cloud Storages.

[davinchia]

What

#4200

How

Describe the solution

Recommended reading order

1. x.java
2. y.python

Pre-merge Checklist

Expand the checklist which is relevant for this PR.

Connector checklist

• Issue acceptance criteria met
• PR name follows PR naming conventions
• Secrets are annotated with airbyte_secret in the connector's spec
• Credentials added to Github CI if needed and not already present. instructions for injecting secrets into CI.
• Unit & integration tests added as appropriate (and are passing)
  • Community members: please provide proof of this succeeding locally e.g: screenshot or copy-paste acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• /test connector=connectors/<name> command as documented here is passing.
  • Community members can skip this, Airbyters will run this for you.
• Code reviews completed
• Documentation updated
  • README.md
  • docs/SUMMARY.md if it's a new connector
  • Created or updated reference docs in docs/integrations/<source or destination>/<name>.
  • Changelog in the appropriate page in docs/integrations/.... See changelog example
  • docs/integrations/README.md contains a reference to the new connector
  • Build status added to build page
• Build is successful
• Connector version bumped like described here
• New Connector version released on Dockerhub by running the /publish command described here
• No major blockers
• PR merged into master branch
• Follow up tickets have been created
• Associated tickets have been closed & stakeholders notified

Connector Generator checklist

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed.

[cgardens]

Nice. Skimmed this. I like the direction.

[davinchia]

@jrhizor besides the test, this is almost there. want to give it a look over?

[michel-tricot]

This is not a great interface to pass Configs. You're passing too much data and with a lot that has nothing to do with the logging feature

[jrhizor]

A class that groups the s3 fields or gcp fields into individual objects would help achieve this.

[davinchia]

Good point. I did think about this but didn't think it was too confusing at the time.

I created a logging configuration interface that has a subset of the logging related config methods. The implementation to the logging configuration interface delgates to the configs interface so force a dependency to avoid multiple ways of configuration the same thing.

[jrhizor]

A class that groups the s3 fields or gcp fields into individual objects would help achieve this.