Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move hadoop and spark ingestion libs from plugins directory to external-plugins to avoid the dependency conflict #8048

Merged

Conversation

xiangfu0
Copy link
Contributor

@xiangfu0 xiangfu0 commented Jan 20, 2022

Description

Move hadoop and spark ingestion libs from plugins directory to plugins-external to avoid the dependency conflict

The scala version different between spark(2.11) and Kafka(2.13) breaks realtime quickstart.

Upgrade Notes

Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

  • Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

  • Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

  • New configuration options
  • Deprecation of configurations
  • Signature changes to public methods/interfaces
  • New plugins added or old plugins removed
  • Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes

Documentation

@codecov-commenter
Copy link

codecov-commenter commented Jan 20, 2022

Codecov Report

Merging #8048 (6e2a89b) into master (d317c59) will decrease coverage by 3.22%.
The diff coverage is n/a.

❗ Current head 6e2a89b differs from pull request most recent head 8e26810. Consider uploading reports for the commit 8e26810 to get more accurate results
Impacted file tree graph

@@             Coverage Diff              @@
##             master    #8048      +/-   ##
============================================
- Coverage     71.35%   68.13%   -3.23%     
+ Complexity     4264     4180      -84     
============================================
  Files          1599     1203     -396     
  Lines         83075    60242   -22833     
  Branches      12402     9294    -3108     
============================================
- Hits          59282    41046   -18236     
+ Misses        19788    16298    -3490     
+ Partials       4005     2898    -1107     
Flag Coverage Δ
integration1 ?
integration2 ?
unittests1 68.13% <ø> (-0.07%) ⬇️
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...a/org/apache/pinot/common/metrics/MinionMeter.java 0.00% <0.00%> (-100.00%) ⬇️
...g/apache/pinot/common/metrics/ControllerMeter.java 0.00% <0.00%> (-100.00%) ⬇️
.../apache/pinot/common/metrics/BrokerQueryPhase.java 0.00% <0.00%> (-100.00%) ⬇️
.../apache/pinot/common/metrics/MinionQueryPhase.java 0.00% <0.00%> (-100.00%) ⬇️
...he/pinot/common/messages/SegmentReloadMessage.java 0.00% <0.00%> (-100.00%) ⬇️
...pinot/core/data/manager/realtime/TimerService.java 0.00% <0.00%> (-100.00%) ⬇️
...not/common/exception/HttpErrorStatusException.java 0.00% <0.00%> (-100.00%) ⬇️
...t/core/plan/StreamingInstanceResponsePlanNode.java 0.00% <0.00%> (-100.00%) ⬇️
...t/core/startree/plan/StarTreeDocIdSetPlanNode.java 0.00% <0.00%> (-100.00%) ⬇️
...ot/common/restlet/resources/TableMetadataInfo.java 0.00% <0.00%> (-100.00%) ⬇️
... and 615 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d317c59...8e26810. Read the comment docs.

@xiangfu0 xiangfu0 merged commit 16a0881 into apache:master Jan 20, 2022
@xiangfu0 xiangfu0 deleted the exclude_spark_plugins_from_quickstart branch January 20, 2022 23:29
KKcorps pushed a commit to KKcorps/incubator-pinot that referenced this pull request Jan 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants