Skip to content

Add trouble shooting part for realtime olap tutorial #604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaofengshi merged 1 commit into from
Apr 14, 2019
Merged

Add trouble shooting part for realtime olap tutorial #604

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions website/_docs30/tutorial/real_time_olap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@ permalink: /docs30/tutorial/realtime_olap.html

Kylin v3.0.0 will release the real-time OLAP function, by the power of new added streaming reciever cluster, Kylin can query streaming data with sub-second latency. This doc is a step by step tutorial, illustrating how to create and build a sample streaming cube.

In this tutorial, we will use Hortonworks HDP 2.2.4 Sandbox VM + Kafka v1.0.2(Scala 2.11) as the environment.
In this tutorial, we will use Hortonworks HDP-2.4.0.0.169 Sandbox VM + Kafka v1.0.2(Scala 2.11) as the environment.

1. Basic concept
2. Prepare environment
3. Create cube
4. Start consumption
5. Monitor receiver

The configuration can be found at [Real-time OLAP configuration](http://kylin.apache.org/docs30/install/configuration.html#realtime-olap).

----

## Basic Concept
Expand Down Expand Up @@ -107,15 +109,15 @@ python user_action.py --max-uid 2000 --max-vid 2000 --msg-sec 100 --enable-hour-
This tool will send 100 records to Kafka every second. Please keep it running during this tutorial. You can check the sample message with kafka-console-consumer.sh now.

### Start Kylin Process
The kylin process will work as coordinator of the receiver cluster.
The kylin process will work as coordinator of the receiver cluster. 7070 is the default port for coordinator.
{% highlight Groff markup %}
cd /usr/local/apache-kylin-3.0.0-SNAPSHOT-bin-master
export KYLIN_HOME=`pwd`
sh bin/kylin.sh start
{% endhighlight %}

### Start Receiver Process
The receiver process will work as worker of the receiver cluster.
The receiver process will work as worker of the receiver cluster. 9090 is the default port for receiver.
{% highlight Groff markup %}
cd ../apache-kylin-3.0.0-SNAPSHOT-bin-receiver/
export KYLIN_HOME=`pwd`
Expand Down Expand Up @@ -230,4 +232,11 @@ When the mouse pointer moves over the segment icon, the segment level statistics
![image](/images/RealtimeOlap/monitor_streaming_2.png)

When the mouse pointer moves over the segment icon, the partition level statistics will be displayed.
![image](/images/RealtimeOlap/monitor_streaming_3.png)
![image](/images/RealtimeOlap/monitor_streaming_3.png)

## Trouble shooting

- Please make sure that the port 7070 and 9090 is not occupied. If you have to change port, please do this set `kylin.stream.node` in `kylin.properties` for receiver or coordinator separately.
- If you find you have messed up and want to clean up, please remove streaming metadata in Zookeeper.
This can be done by executing `rmr PATH_TO_DELETE` in `zookeeper-client` shell. By default, the root dir of streaming metadata is under `kylin.env.zookeeper-base-path` + `kylin.metadata.url` + `/stream`.
For example, if you set `kylin.env.zookeeper-base-path` to `/kylin`， set `kylin.metadata.url` to `kylin_metadata@hbase`, you should delete path `/kylin/kylin_metadata/stream`.