KYLIN-4035 Calculate column cardinality by using spark engine

[majic31]

No description provided.

[asfgit]

Can one of the admins verify this patch?

[coveralls]

Pull Request Test Coverage Report for Build 4589

• 0 of 100 (0.0%) changed or added relevant lines in 4 files are covered.
• 5 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.03%) to 28.248%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java	0	1	0.0%
engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java	0	14	0.0%
server-base/src/main/java/org/apache/kylin/rest/service/TableService.java	0	17	0.0%
engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkColumnCardinality.java	0	68	0.0%

Files with Coverage Reduction	New Missed Lines	%
core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java	5	77.81%

Totals
Change from base Build 4585:	-0.03%
Covered Lines:	23243
Relevant Lines:	82282

---

💛 - Coveralls

[codecov-io]

Codecov Report

Merging #678 into master will decrease coverage by 0.03%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             master     #678      +/-   ##
============================================
- Coverage     25.79%   25.75%   -0.04%     
  Complexity     6010     6010              
============================================
  Files          1382     1383       +1     
  Lines         82195    82282      +87     
  Branches      11516    11527      +11     
============================================
- Hits          21201    21195       -6     
- Misses        58950    59042      +92     
- Partials       2044     2045       +1

Impacted Files	Coverage Δ	Complexity Δ
...org/apache/kylin/engine/spark/SparkExecutable.java	0% <0%> (ø)	0 <0> (ø)	⬇️
.../java/org/apache/kylin/common/KylinConfigBase.java	13.02% <0%> (-0.03%)	42 <0> (ø)
...va/org/apache/kylin/rest/service/TableService.java	13.84% <0%> (-0.62%)	9 <0> (ø)
...che/kylin/engine/spark/SparkColumnCardinality.java	0% <0%> (ø)	0 <0> (?)
...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java	69.3% <0%> (-1.83%)	7% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ec378c...624c78a. Read the comment docs.

[hit-lacus]

Cloud you please merge all these commit into one commit and force push? Thank you.

[majic31]

Ok, I will try it( I am sorry because I am not familiar with git operation ~ )

[majic31]

Cloud you please merge all these commit into one commit and force push? Thank you.

Excuse me, How can I merge all commits into one commit?~
May I delete fork, then re-fork, add my code and pr again? ~~

[hit-lacus]

@majic31 I think you should do the following:

1. reopen this PR
2. use git log and git reset --soft to discard all commit from you, and use git commit to commit only one commit in local branch
3. use git push -f origin ${YOUR_BRANCH_NAME} to force push to remote branch

If you want to discard all commit before commit xxx, please use git reset --soft xxx.
The following example discard three commit before b089352.

xiaoxiang.yu:SourceKylin/ (3997) $ git log -5 | cat                                                                                        [9:45:29]
commit 1f827d80de9685f067b54b6fb4ca129f94175f8d
Author: Liu Shaohui <liushaohui@xiaomi.com>
Date:   Sun Jun 9 16:47:58 2019 +0800

    KYLIN-3997: Add a health check job of Kylin

commit 670c2a98a82a699481eceddb685168104d5b185c
Author: chao long <wayne.l@qq.com>
Date:   Sun May 19 18:45:37 2019 +0800

    KYLIN-2620 TopN can't match when multi sort columns

commit 475df34fc27969414936e59a5ee2efd23cf2bcbe
Author: Na Zhai <38899902+GinaZhai@users.noreply.github.com>
Date:   Thu May 30 17:33:53 2019 +0800

    KYLIN-3893 Add validation for the encoding length and the type of encoding (#622)
    
    * KYLIN-3893 Add validation for the encoding length and the type of encoding
    * KYLIN-3893 Code review

commit b08935205a7aa1a00b6c09ef4625f45eacf9b013
Author: liapan <liapan@ebay.com>
Date:   Sun May 5 11:22:46 2019 +0800

    KYLIN-3980 Cube planner cuboid id error when dimensions too many

commit 47431ba0da7c3cce533f4d9dcd600654f33e0c90
Author: hit-lacus <hit_lacus@126.com>
Date:   Thu May 23 11:31:00 2019 +0800

    KYLIN-3958 Add quote for backtick
xiaoxiang.yu:SourceKylin/ (3997) $                                                                                                         [9:45:34]
xiaoxiang.yu:SourceKylin/ (3997) $ git reset --soft b08935205a7aa1a00b6c09ef4625f45eacf9b013                                               [9:45:36]

xiaoxiang.yu:SourceKylin/ (3997*) $ git log -5 | cat                                                                                       [9:46:02]
commit b08935205a7aa1a00b6c09ef4625f45eacf9b013
Author: liapan <liapan@ebay.com>
Date:   Sun May 5 11:22:46 2019 +0800

    KYLIN-3980 Cube planner cuboid id error when dimensions too many

commit 47431ba0da7c3cce533f4d9dcd600654f33e0c90
Author: hit-lacus <hit_lacus@126.com>
Date:   Thu May 23 11:31:00 2019 +0800

    KYLIN-3958 Add quote for backtick

commit f15b1f8a7ce05416dc1df2ee8a2ab629d1c00873
Author: javalife0312 <javalife0312@126.com>
Date:   Sun May 19 18:22:17 2019 +0800

    KYLIN-3958 MrHive-Dict support build by livy

commit 4e7b242b3009f1f194d2e66dae9b368330b89a7b
Author: shaofengshi <shaofengshi@apache.org>
Date:   Wed May 22 21:57:30 2019 +0800

    minor, switch to openjdk8 for travis

commit b48de90d046bfbef30917149e9c3426c6a7365cc
Author: Michael Hausegger <hausegger.michael@googlemail.com>
Date:   Tue May 21 04:18:57 2019 +0200

    Add unit tests (#645)
    
    * Add unit tests

If you have more question, please send me email to (xiaoxiang.yu@kyligence.io).