-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Implement KafkaSchemaTransformReadConfiguration #22403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement KafkaSchemaTransformReadConfiguration #22403
Conversation
|
Assigning reviewers. If you would like to opt out of this review, comment R: @apilloud for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reminder, please take a look at this pr: @apilloud @chamikaramj |
|
Looks like you requested @pabloem to review this, but he is out for another week. This really should get reviewed by someone working on IOs. cc: @johnjcasey |
|
Right now it looks like this is failing to compile, and spotless is also failing. Can you address these? |
Hello @johnjcasey Thank you for reviewing. Are there other gradle commands you might recommend I run to detect where compile and spotless is failing? The following gradle tasks did not detect errors using the wrong |
|
I can review this this week. Thanks y'all |
.../kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java
Outdated
Show resolved
Hide resolved
.../kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java
Outdated
Show resolved
Hide resolved
…KafkaSchemaTransformReadConfiguration.java
…KafkaSchemaTransformReadConfiguration.java
|
LGTM! |
|
Run Java PreCommit |
1 similar comment
|
Run Java PreCommit |
|
thanks @damondouglas ! |
* Implement KafkaSchemaTransformReadConfiguration * Replace with java.annotation.Nullable * Update sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java * Update sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java Co-authored-by: Pablo <pabloem@users.noreply.github.com>
This PR address #21414 with a KafkaSchemaTransformReadConfiguration implementation. It's design goals are to work with a KafkaSchemaTransformReadProvider that extends a TypedSchemaTransformProvider. Subsequent to this PR's approval/merge, the plan is to implement said corresponding KafkaSchemaTransformReadProvider.
Questions remain, however, how to configure the following in the setting of a TypedSchemaTransformProvider:
Due to historically failing beam_PreCommit_Java tests, to validate this PR, I ran the following prior to submission:
I would like to request the following to review this PR:
R: @pabloem
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username).addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.UpdateCHANGES.mdwith noteworthy changes.If this contribution is large, please file an Apache Individual Contributor License Agreement.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.