Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KYLIN-1111 Ignore unsupported hive column types when sync hive table #372

Closed
wants to merge 1 commit into from

Conversation

whuwb
Copy link

@whuwb whuwb commented Dec 5, 2018

Issue : https://issues.apache.org/jira/browse/KYLIN-1111
Solution : Filter in HiveMetadataExplorer#extractColumnFromMeta
How to verify:

  1. add new UT
  2. Verified with IT:
    [INFO] Apache Kylin - Integration Test .................... SUCCESS [ 02:30 h]
    [INFO] Apache Kylin - Tomcat Extension .................... SUCCESS [ 2.303 s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 02:47 h
    [INFO] Finished at: 2018-12-05T18:29:37Z
    [INFO] ------------------------------------------------------------------------
  3. Manual add a Hive table with map<string, int>, and try to load into Kylin

@asfgit
Copy link

asfgit commented Dec 5, 2018

Can one of the admins verify this patch?

@codecov-io
Copy link

Codecov Report

Merging #372 into master will decrease coverage by <.01%.
The diff coverage is 26.66%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #372      +/-   ##
============================================
- Coverage     23.64%   23.63%   -0.01%     
- Complexity     4838     4840       +2     
============================================
  Files          1131     1131              
  Lines         68193    68199       +6     
  Branches       9684     9686       +2     
============================================
- Hits          16124    16120       -4     
- Misses        50503    50510       +7     
- Partials       1566     1569       +3
Impacted Files Coverage Δ Complexity Δ
...apache/kylin/source/hive/HiveMetadataExplorer.java 0% <0%> (ø) 0 <0> (ø) ⬇️
...a/org/apache/kylin/metadata/datatype/DataType.java 70.28% <80%> (+0.28%) 44 <3> (+3) ⬆️
...he/kylin/dict/lookup/cache/RocksDBLookupTable.java 72.97% <0%> (-5.41%) 6% <0%> (-1%)
.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java 63.84% <0%> (-2.31%) 0% <0%> (ø)
...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java 70.21% <0%> (-0.92%) 7% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f672b0...5971b6f. Read the comment docs.

@shaofengshi
Copy link
Contributor

Thanks Bin! Merged into the master branch!

@shaofengshi shaofengshi closed this Dec 6, 2018
@whuwb whuwb deleted the issue_1111_wubin branch December 6, 2018 09:33
@coveralls
Copy link

Pull Request Test Coverage Report for Build 3912

  • 5 of 15 (33.33%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.005%) to 25.943%

Changes Missing Coverage Covered Lines Changed/Added Lines %
source-hive/src/main/java/org/apache/kylin/source/hive/HiveMetadataExplorer.java 0 10 0.0%
Totals Coverage Status
Change from base Build 3913: 0.005%
Covered Lines: 17693
Relevant Lines: 68199

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants