Skip to content

Conversation

@damondouglas
Copy link
Contributor

This PR address #21414 with a KafkaSchemaTransformReadConfiguration implementation. It's design goals are to work with a KafkaSchemaTransformReadProvider that extends a TypedSchemaTransformProvider. Subsequent to this PR's approval/merge, the plan is to implement said corresponding KafkaSchemaTransformReadProvider.

Questions remain, however, how to configure the following in the setting of a TypedSchemaTransformProvider:

Due to historically failing beam_PreCommit_Java tests, to validate this PR, I ran the following prior to submission:

./gradlew rat
./gradlew spotlessCheck
./gradlew sdks:java:io:kafka:check
./gradlew sdks:java:io:kafka:checkStyleMain

I would like to request the following to review this PR:
R: @pabloem


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@github-actions
Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @apilloud for label java.
R: @chamikaramj for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@github-actions
Copy link
Contributor

Reminder, please take a look at this pr: @apilloud @chamikaramj

@apilloud apilloud requested a review from pabloem July 29, 2022 16:20
@apilloud
Copy link
Member

Looks like you requested @pabloem to review this, but he is out for another week. This really should get reviewed by someone working on IOs. cc: @johnjcasey

@johnjcasey
Copy link
Contributor

Right now it looks like this is failing to compile, and spotless is also failing.

Can you address these?

@damondouglas
Copy link
Contributor Author

damondouglas commented Aug 1, 2022

Right now it looks like this is failing to compile, and spotless is also failing.

Can you address these?
Right now it looks like this is failing to compile, and spotless is also failing.

Can you address these?

Hello @johnjcasey Thank you for reviewing.

Are there other gradle commands you might recommend I run to detect where compile and spotless is failing?

The following gradle tasks did not detect errors using the wrong @Nulllable class.

./gradlew rat
./gradlew spotlessCheck
./gradlew sdks:java:io:kafka:check
./gradlew sdks:java:io:kafka:checkStyleMain

@pabloem
Copy link
Member

pabloem commented Aug 8, 2022

I can review this this week. Thanks y'all

@pabloem
Copy link
Member

pabloem commented Aug 12, 2022

LGTM!

@pabloem
Copy link
Member

pabloem commented Aug 15, 2022

Run Java PreCommit

1 similar comment
@pabloem
Copy link
Member

pabloem commented Aug 15, 2022

Run Java PreCommit

@pabloem pabloem merged commit 67cb87e into apache:master Aug 15, 2022
@pabloem
Copy link
Member

pabloem commented Aug 15, 2022

thanks @damondouglas !

@damondouglas damondouglas deleted the 21414-KafkaSchemaTransformReadConfiguration branch August 17, 2022 02:27
MarcoRob pushed a commit to MarcoRob/beam that referenced this pull request Sep 5, 2022
* Implement KafkaSchemaTransformReadConfiguration

* Replace with java.annotation.Nullable

* Update sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java

* Update sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadConfiguration.java

Co-authored-by: Pablo <pabloem@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants