New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[KSHC] Support Parquet/Orc provider is splitable #5017
Conversation
...ark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/read/HiveScan.scala
Outdated
Show resolved
Hide resolved
Another question not related to this PR, what's the output filename pattern written by this connector? does it have |
Like Spark Hive v1, it has no suffix |
It's inconsistent in built-in Hive, parquet has no suffix but ORC does. |
Codecov Report
@@ Coverage Diff @@
## master #5017 +/- ##
======================================
Coverage 0.00% 0.00%
======================================
Files 563 563
Lines 31167 31167
Branches 4070 4070
======================================
Misses 31167 31167 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
...ark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/read/HiveScan.scala
Show resolved
Hide resolved
...ark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/read/HiveScan.scala
Outdated
Show resolved
Hide resolved
Thanks all, merged to master |
Why are the changes needed?
This PR amins to support Parquet/Orc provider is splitable.
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before make a pull request