Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destination CDK: Simplify AsyncStreamConsumer constructors #37106

Merged
merged 1 commit into from
Apr 12, 2024

Conversation

gisripa
Copy link
Contributor

@gisripa gisripa commented Apr 12, 2024

TL;DR

This pull request removes complex constructors and unifies into single constructor in AsyncStreamConsumer and related classes.

What changed?

  • Renamed DeserializationUtil to AirbyteMessageDeserializer
  • Simplified Injection of Transformer & Deserializer
  • Removed combinatorics constructors in AsyncStreamConsumer
  • Updated references in StagingConsumerFactory & JdbcBufferedConsumerFactory

Copy link

vercel bot commented Apr 12, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 12, 2024 8:20pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Apr 12, 2024
Copy link
Contributor Author

gisripa commented Apr 12, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @gisripa and the rest of your teammates on Graphite Graphite

@gisripa gisripa changed the title destinations-cdk-simplify-ctors Destination CDK: Simplify AsyncStreamConsumer constructors Apr 12, 2024
@gisripa gisripa marked this pull request as ready for review April 12, 2024 19:05
@gisripa gisripa requested review from a team as code owners April 12, 2024 19:05
* @return PartialAirbyteMessage if the message is valid, empty otherwise
*/
fun deserializeAirbyteMessage(
messageString: String?,
dataTransformer: StreamAwareDataTransformer,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 love to see this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love how the doc string didn't even match the parameters...

@@ -144,7 +146,7 @@ private constructor(
writeConfigs,
typerDeduper
), // todo (cgardens) - wrapping the old close function to avoid more code churn.
OnCloseFunction { _, streamSyncSummaries ->
{ _, streamSyncSummaries ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that we're in kotlin (no checked exceptions)... can we kill try-catch here? (... or just use the function returned from GeneralStagingFunctions.onCloseFunction directly)

feels weird to build a lambda that effectively just wraps another lambda

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah let me see if i can unify. i have no clue what the java equivalent code was 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified it and yet another cgardens TODO gone 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should track the number of times the string cgardens shows up in our code as a vanity metric :P

@gisripa gisripa force-pushed the gireesh/04-12-destinations-cdk-simplify-ctors branch from 358ab5b to bf9a6e4 Compare April 12, 2024 19:53
@gisripa gisripa force-pushed the gireesh/04-12-destinations-cdk-simplify-ctors branch from bf9a6e4 to 9d92625 Compare April 12, 2024 20:20
@gisripa
Copy link
Contributor Author

gisripa commented Apr 12, 2024

/publish-java-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/8667560231
✅ Successfully published Java CDK version=0.30.3!

@gisripa gisripa merged commit 6d5ecca into master Apr 12, 2024
26 checks passed
@gisripa gisripa deleted the gireesh/04-12-destinations-cdk-simplify-ctors branch April 12, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants