[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in camel-case to be used with ProtoMessageSchema #15306

chrishinds · 2021-08-10T18:13:20Z

ProtoByteBuddyUtils.protoGetterName() previously depended on field names being snake-case. Now it attempts method identification first with snake-case, and then with camel-case.

R: @reuvenlax

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

`ValidatesRunner` compliance status (on master branch)

Lang	ULR	Twister2
Go	---	---
Java
Python	---	---
XLang		---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

…rsion 2.5.0 & grpcio to 1.34.0

[BEAM-12411] Cherry-pick apache#14888: Update Tensorflow to version 2.5.0 & grpcio to 1.34.0

…uery services

…and shutdown BigQuery services

…GBK safety checks by default

… content blockers

[BEAM-12507][BEAM-12508][release-2.31.0] Fix release content blockers

camel-case to be used with ProtoMessageSchema

TheNeuralBit · 2021-09-01T20:19:27Z

retest this please

TheNeuralBit · 2021-09-01T20:20:09Z

runners/google-cloud-dataflow-java/build.gradle

@@ -45,8 +45,8 @@ processResources {
  filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
    'dataflow.legacy_environment_major_version' : '8',
    'dataflow.fnapi_environment_major_version' : '8',
-    'dataflow.legacy_container_version' : 'beam-master-20210525',
-    'dataflow.fnapi_container_version' : 'beam-master-20210524',


It looks like maybe you picked up this change from the relase branch?

TheNeuralBit · 2021-09-01T20:20:39Z

.../io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java

+      }
+      return jobService;
+    }
+


I made this change against the v2.31.0 tag, then had to merge-in HEAD for Github to allow PR creation. Only once I'd pushed the button did I realise I'd captured irrelevant commits.
Really, really, sorry. What do you recommend?

Files reverted.

TheNeuralBit

Looks good overall, I have a few suggestions though

TheNeuralBit · 2021-09-14T23:39:04Z

...ions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoByteBuddyUtils.java

    final String camel = CaseFormat.LOWER_UNDERSCORE.to(CaseFormat.UPPER_CAMEL, name);
    return DEFAULT_PROTO_GETTER_PREFIX
        + camel
        + PROTO_GETTER_SUFFIX.getOrDefault(fieldType.getTypeName(), "");
  }

-  static String protoSetterName(String name, FieldType fieldType) {
+  static String protoSetterNameFromSnake(String name, FieldType fieldType) {


What do you think about adding a CaseFormat parameter to protoSetterName instead of having a function for each format?

Suggested change

static String protoSetterNameFromSnake(String name, FieldType fieldType) {

static String protoSetterName(String name, FieldType fieldType, CaseFormat caseFormat) {

TheNeuralBit · 2021-09-14T23:47:40Z

sdks/java/extensions/protobuf/src/test/proto/proto3_schema_messages.proto

+    bool primitiveBoolCamel = 13;
+    string primitiveStringCamel = 14;
+    bytes primitiveBytesCamel = 15;
+}


Could you add another test case to make sure we exercise the oneof logic for camel case?

aaltay · 2021-09-23T19:00:16Z

@chrishinds - Could you look at the open comments please?

aaltay · 2021-10-07T20:05:07Z

Any update on this? Should we close it?

TheNeuralBit · 2021-10-12T19:43:51Z

It would be a shame to close, it's very close. I think the only blocker is the missing test case for oneof/camel-case

aaltay · 2021-10-12T20:54:52Z

@TheNeuralBit - I am not sure the author is still active. If you have time, feel free to edit the PR. Or merge, and file a follow up jira.

github-actions · 2022-02-05T12:40:14Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

github-actions · 2022-02-12T12:44:11Z

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

apilloud and others added 13 commits June 7, 2021 13:06

Set Dataflow container to release version.

df78d07

Merge pull request apache#14888: [BEAM-12411] Update Tensorflow to ve…

a1c6f68

…rsion 2.5.0 & grpcio to 1.34.0

Merge pull request apache#14993 from aaltay/release-2.31.0

ef28cc8

[BEAM-12411] Cherry-pick apache#14888: Update Tensorflow to version 2.5.0 & grpcio to 1.34.0

Merge pull request apache#14949: [BEAM-12356] Cache and shutdown BigQ…

878cf7b

…uery services

Merge pull request apache#15009: [BEAM-12356] [release-2.31.0] Cache …

809e10b

…and shutdown BigQuery services

Merge pull request apache#15003 from zhoufek/aut

c318421

Merge pull request apache#15027: [BEAM-9487][release-2.31.0] Disable …

9161c94

…GBK safety checks by default

Merge pull request apache#15031: [BEAM-12507][BEAM-12508] Fix release…

586fb96

… content blockers

Merge pull request apache#15035 from apilloud/blockers

3839b83

[BEAM-12507][BEAM-12508][release-2.31.0] Fix release content blockers

Set version for 2.31.0 RC1

90f98a6

[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in

0f53243

camel-case to be used with ProtoMessageSchema

Merge branch 'master' into ProtoSchema-camel-field-naming

58c7ef4

Merge branch 'apache:master' into ProtoSchema-camel-field-naming

11168e3

pabloem requested a review from reuvenlax August 19, 2021 17:44

TheNeuralBit self-requested a review September 1, 2021 20:19

TheNeuralBit reviewed Sep 1, 2021

View reviewed changes

revert files mistakenly captured by [BEAM-12736] PR

9fe2652

TheNeuralBit reviewed Sep 14, 2021

View reviewed changes

github-actions bot added the stale label Feb 5, 2022

github-actions bot closed this Feb 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in camel-case to be used with ProtoMessageSchema #15306

[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in camel-case to be used with ProtoMessageSchema #15306

chrishinds commented Aug 10, 2021

TheNeuralBit commented Sep 1, 2021

TheNeuralBit Sep 1, 2021

TheNeuralBit Sep 1, 2021

chrishinds Sep 3, 2021

chrishinds Sep 3, 2021

TheNeuralBit left a comment

TheNeuralBit Sep 14, 2021

TheNeuralBit Sep 14, 2021

aaltay commented Sep 23, 2021

aaltay commented Oct 7, 2021

TheNeuralBit commented Oct 12, 2021

aaltay commented Oct 12, 2021

github-actions bot commented Feb 5, 2022

github-actions bot commented Feb 12, 2022

	static String protoSetterNameFromSnake(String name, FieldType fieldType) {
	static String protoSetterName(String name, FieldType fieldType, CaseFormat caseFormat) {

[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in camel-case to be used with ProtoMessageSchema #15306

[BEAM-12736] ProtoByteBuddyUtils fix enabling field names defined in camel-case to be used with ProtoMessageSchema #15306

Conversation

chrishinds commented Aug 10, 2021

ValidatesRunner compliance status (on master branch)

Examples testing status on various runners

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

TheNeuralBit commented Sep 1, 2021

TheNeuralBit Sep 1, 2021

Choose a reason for hiding this comment

TheNeuralBit Sep 1, 2021

Choose a reason for hiding this comment

chrishinds Sep 3, 2021

Choose a reason for hiding this comment

chrishinds Sep 3, 2021

Choose a reason for hiding this comment

TheNeuralBit left a comment

Choose a reason for hiding this comment

TheNeuralBit Sep 14, 2021

Choose a reason for hiding this comment

TheNeuralBit Sep 14, 2021

Choose a reason for hiding this comment

aaltay commented Sep 23, 2021

aaltay commented Oct 7, 2021

TheNeuralBit commented Oct 12, 2021

aaltay commented Oct 12, 2021

github-actions bot commented Feb 5, 2022

github-actions bot commented Feb 12, 2022

`ValidatesRunner` compliance status (on master branch)