KYLIN-4114 Provided a self-contained docker image for Kylin

[weibin0516]

No description provided.

[codecov-io]

Codecov Report

Merging #797 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #797      +/-   ##
============================================
- Coverage     25.61%   25.61%   -0.01%     
  Complexity     6046     6046              
============================================
  Files          1398     1398              
  Lines         83449    83449              
  Branches      11685    11685              
============================================
- Hits          21378    21376       -2     
- Misses        60017    60019       +2     
  Partials       2054     2054

Impacted Files	Coverage Δ	Complexity Δ
...che/kylin/stream/core/storage/CheckPointStore.java	63.63% <0%> (-3.04%)	21% <0%> (-1%)
...he/kylin/job/impl/threadpool/DefaultScheduler.java	74.41% <0%> (-2.33%)	12% <0%> (ø)
...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java	69.6% <0%> (+0.3%)	7% <0%> (ø)	⬇️
...ream/core/storage/columnar/ColumnarStoreCache.java	51.16% <0%> (+2.32%)	8% <0%> (+1%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 73c1a85...6833856. Read the comment docs.

[coveralls]

Pull Request Test Coverage Report for Build 4933

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 7 unchanged lines in 2 files lost coverage.
• Overall coverage remained the same at 28.077%

---

Files with Coverage Reduction	New Missed Lines	%
core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java	2	68.46%
core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java	5	77.81%

Totals
Change from base Build 4935:	0.0%
Covered Lines:	23434
Relevant Lines:	83464

---

💛 - Coveralls

[hit-lacus]

Thanks @weibin0516 , this looks cool, here a list for what I want to check.

For PoC

• MapReduce Cubing
• Spark Cubing
• Segment Merge
• NRT Streaming
• Realtime OLAP

For Developer

• Package from source
• Integration test

Other Comments

• I guess we may check if we could upload image to docker hub, thus user may simplely run it in one command. (For PoC purpose.)
• Maybe we could build image in hierarchy way, thus we can switch between different hadoop distribution easily. And for PoC, we could provided a minimized image; for developer, we could add maven/nodejs.
• Make sure all basic function works well.

[hit-lacus]

Forget to use $HBASE_VERSION here?

[weibin0516]

fixed

[hit-lacus]

If Integration test can not pass currently, please remove this, thank you.

[weibin0516]

fixed

[weibin0516]

@hit-lacus , thanks for review, all functions except Integration test work fine work fine. In my opinion, Integration test is not necessarily needed, may be i will solve it in next pr.

[hit-lacus]

Hi, @weibin0516

Did you update your PR? I cannot find kylin.sh start & kylin.sh streaming start anywhere.

[hit-lacus]

Test Env

• MacOS 10.14.4
• Docker Desktop for Mac (Docker Engine: 18.09.2)

Start standalone docker container

docker run -d --rm \
-p 7070:7070 \
-p 8088:8088 \
-p 50070:50070 \
-p 8032:8032 \
-p 8042:8042 \
-p 60010:60010 \
codingforfun/apache-kylin-standalone

Attach to container

docker exec -it ${CONTAINER_ID} /bin/bash

Building Cube (Build/Refresh/Merge)

Near Realtime Streaming

Realtime OLAP

Known issue

1. Nodemanager may be unhealthy because of lack of disk space.
   This may be fixed by docker image prune -h to reduce needless disk space.

2. Container cannot restart successfully.

3. Realtime OLAP building job cubing job throw NPE for some reasons(to be checked).

[shaofengshi]

+1 Awesome! Looks good to me! Thank you weibin!

[weibin0516]

@shaofengshi , thanks very much.

[hit-lacus]

Could you please update this?