Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the type of default values to match what Schema.parse() does. #3611

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

srramach
Copy link
Contributor

When calling field.defaultVal() (or equivalent compat helper methods)
under Avro 1.9.2+, we get back a type that's not consistent with what
Schema.parse() creates internally. This causes two Schemas constructed
in these two different ways (Schema.parse vs Schema.createRecord) to
be considered unequal, even though their toString() representations
are identical.

Fix this situation by calling parseJsonToObject(), which results in
the default value being interpreted similar to Schema.parse().

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

Description

  • Here are some details about my PR, including screenshots (if applicable):

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

When calling field.defaultVal() (or equivalent compat helper methods)
under Avro 1.9.2+, we get back a type that's not consistent with what
Schema.parse() creates internally. This causes two Schemas constructed
in these two different ways (Schema.parse vs Schema.createRecord) to
be considered unequal, even though their toString() representations
are identical.

Fix this situation by calling parseJsonToObject(), which results in
the default value being interpreted similar to Schema.parse().
@codecov-commenter
Copy link

codecov-commenter commented Nov 29, 2022

Codecov Report

Merging #3611 (4240794) into master (c6d6c1b) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #3611   +/-   ##
=========================================
  Coverage     46.87%   46.88%           
+ Complexity    10687    10684    -3     
=========================================
  Files          2125     2125           
  Lines         83157    83157           
  Branches       9266     9266           
=========================================
+ Hits          38983    38987    +4     
+ Misses        40598    40590    -8     
- Partials       3576     3580    +4     
Impacted Files Coverage Δ
...c/main/java/org/apache/gobblin/util/AvroUtils.java 56.50% <100.00%> (ø)
...e/gobblin/service/monitoring/GitConfigMonitor.java 95.23% <0.00%> (-4.77%) ⬇️
...a/org/apache/gobblin/cluster/GobblinHelixTask.java 62.36% <0.00%> (-2.16%) ⬇️
...a/management/copy/publisher/CopyDataPublisher.java 74.17% <0.00%> (-1.33%) ⬇️
...anagement/copy/replication/ConfigBasedDataset.java 68.87% <0.00%> (ø)
...main/java/org/apache/gobblin/yarn/YarnService.java 15.68% <0.00%> (+0.65%) ⬆️
.../gobblin/util/limiter/BatchedPermitsRequester.java 75.47% <0.00%> (+0.75%) ⬆️
.../gobblin/cluster/GobblinHelixTaskStateTracker.java 68.75% <0.00%> (+6.25%) ⬆️
...he/gobblin/source/PartitionAwareFileRetriever.java 55.55% <0.00%> (+7.40%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants