Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MINOR] remove Hive dependency from delta streamer #1493

Closed

Conversation

garyli1019
Copy link
Member

What is the purpose of the pull request

This change will remove Hive dependency from the delta streamer if the user don't trigger the hive sync.

Brief change log

  • Change the configuration type in DeltaStreamer and DeltaSync.

Verify this pull request

This pull request is already covered by existing tests, such as testBulkInsertsAndUpsertsWithSQLBasedTransformerFor2StepPipeline in
TestHoodieDeltaStreamer

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@garyli1019 garyli1019 force-pushed the separate-hive-from-deltastreamer branch from 7f57ca6 to 8d5582e Compare April 7, 2020 05:20
@codecov-io
Copy link

codecov-io commented Apr 7, 2020

Codecov Report

Merging #1493 into master will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1493      +/-   ##
============================================
+ Coverage     71.54%   71.58%   +0.03%     
+ Complexity      261      260       -1     
============================================
  Files           336      336              
  Lines         15744    15741       -3     
  Branches       1610     1610              
============================================
+ Hits          11264    11268       +4     
+ Misses         3759     3752       -7     
  Partials        721      721              
Impacted Files Coverage Δ Complexity Δ
...apache/hudi/utilities/deltastreamer/DeltaSync.java 72.00% <100.00%> (ø) 38.00 <0.00> (ø)
...i/utilities/deltastreamer/HoodieDeltaStreamer.java 78.28% <100.00%> (-0.33%) 7.00 <1.00> (-1.00)
...n/java/org/apache/hudi/common/model/HoodieKey.java 88.88% <0.00%> (ø) 0.00% <0.00%> (ø%)
...src/main/java/org/apache/hudi/metrics/Metrics.java 72.22% <0.00%> (+13.88%) 0.00% <0.00%> (ø%)
...g/apache/hudi/metrics/InMemoryMetricsReporter.java 80.00% <0.00%> (+40.00%) 0.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eaf6cc2...8d5582e. Read the comment docs.

@garyli1019 garyli1019 force-pushed the separate-hive-from-deltastreamer branch from 8d5582e to 5ce8e7a Compare April 8, 2020 01:05
@garyli1019 garyli1019 force-pushed the separate-hive-from-deltastreamer branch from 5ce8e7a to d3a2a20 Compare April 8, 2020 01:10
@garyli1019
Copy link
Member Author

combine with #1486

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants