Destination redshift: Use new interfaces

[edgao]

I intend to ship this during assemble (in spite of the large diff, it should actually be a relatively low-risk change; I'll run some tests in the perf test workspace just for extra confidence)

general PR structure:

• delete RedshiftSqlOperations / RedshiftS3StagingSqlOperations
• new class: RedshiftStorageOperation. Mostly copied out of RedshiftSqlOperations / RedshiftS3StagingSqlOperations / JdbcSqlOperations
  • plausibly this should be in the cdk?
    • All the final table stuff is completely destination-agnostic (it's identical to snowflake's version, and probably postgres will also be identical)
    • but all the raw table stuff is bespoke per destination
    • IMO we should keep these separate for now - see how postgres goes
  • replaced the jooq sql gen with native kotlin string templating, see e.g. old createTableQueryV2 vs the new RedshiftStorageOperation.Companion.createRawTableQuery
• new class: RedshiftDV2Migration. Very thin wrapper around the old V1V2Migrator to bring it into the Migrations interface.
• RedshiftDestination
  • no longer extends AbstractJdbcDestination. We were already overriding all of the superclass methods anyway, and weren't even invoking super.anything()
  • check now runs a mini-sync (similar to what destiantion-snowflake is doing in Destination Snowflake: Adapting to new connector interfaces #38658). Again, we maybe should extract this to CDK.
  • getMigrations now includes RedshiftDV2Migration, so we don't need to handle it separately in getSerializedConsumer
  • further deletions of encrypted staging code
• Delete the DAT classes, since they rely on a bunch of JDBC base classes. They were already disabled anyway, and we know we want to rewrite these tests to not rely on the container 🤷

closes https://github.com/airbytehq/airbyte-internal-issues/issues/8351

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 27, 2024 0:02am

[edgao]

• Destination redshift: Use new interfaces #39008 : 2 dependent PRs (#39334 , #40201 ) 👈
• Destination redshift: Remove encrypted staging option #39141
• Destination redshift: convert test classes to kotlin #38893
• Destination redshift: remove standard inserts #38886
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @edgao and the rest of your teammates on Graphite