Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark Connector GRPC reader fix for reading realtime tables #8824

Merged
merged 2 commits into from
Jun 4, 2022

Conversation

cbalci
Copy link
Contributor

@cbalci cbalci commented Jun 3, 2022

Change
Previously the GRPC reader in Spark Connector plugin was only able to send OFFLINE table queries. This PR fixes that and enables the connector to issue both OFFLINE and REALTIME queries based on the table type.

Testing

  • Integration tests in ExampleSparkPinotConnectorTest are updated accordingly.

bugfix

@codecov-commenter
Copy link

codecov-commenter commented Jun 3, 2022

Codecov Report

Merging #8824 (0a1e982) into master (8b2b8f5) will decrease coverage by 5.33%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #8824      +/-   ##
============================================
- Coverage     68.15%   62.82%   -5.34%     
- Complexity     4626     4641      +15     
============================================
  Files          1735     1695      -40     
  Lines         91298    89486    -1812     
  Branches      13636    13453     -183     
============================================
- Hits          62226    56218    -6008     
- Misses        24733    29223    +4490     
+ Partials       4339     4045     -294     
Flag Coverage Δ
integration2 ?
unittests1 66.19% <ø> (-0.04%) ⬇️
unittests2 14.10% <ø> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...va/org/apache/pinot/core/routing/RoutingTable.java 0.00% <0.00%> (-100.00%) ⬇️
...va/org/apache/pinot/common/config/NettyConfig.java 0.00% <0.00%> (-100.00%) ⬇️
...a/org/apache/pinot/common/metrics/MinionMeter.java 0.00% <0.00%> (-100.00%) ⬇️
...g/apache/pinot/common/metrics/ControllerMeter.java 0.00% <0.00%> (-100.00%) ⬇️
.../apache/pinot/common/metrics/BrokerQueryPhase.java 0.00% <0.00%> (-100.00%) ⬇️
.../apache/pinot/common/metrics/MinionQueryPhase.java 0.00% <0.00%> (-100.00%) ⬇️
...apache/pinot/common/helix/ExtraInstanceConfig.java 0.00% <0.00%> (-100.00%) ⬇️
...ache/pinot/server/access/AccessControlFactory.java 0.00% <0.00%> (-100.00%) ⬇️
...he/pinot/common/messages/SegmentReloadMessage.java 0.00% <0.00%> (-100.00%) ⬇️
...he/pinot/common/messages/TableDeletionMessage.java 0.00% <0.00%> (-100.00%) ⬇️
... and 371 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b2b8f5...0a1e982. Read the comment docs.

@KKcorps KKcorps added the bugfix label Jun 3, 2022
@yupeng9 yupeng9 merged commit e55c127 into apache:master Jun 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants