Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
94 lines (67 sloc) 1.99 KB

Install Kafka

install depends

apt-get install -y default-jre

install Kafka

mkdir /root/source
cd /root/source
wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz
 
mkdir -p /work/kafka
cd /work/kafka
tar -zxf /root/source/kafka_2.11-0.11.0.2.tgz --strip 1

edit conf

vim /work/kafka/config/server.properties

The broker's id is 1.

For tester: if you have only one kafka broker, set offsets.topic.replication.factor=1.

If the amount of brokers less than offsets.topic.replication.factor, high-level kafka consumer may not work properly, which means that sharelogger cannot consume any share from kafka.

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=1

# The replication factor for the offsets topic.
offsets.topic.replication.factor=3 # IMPORTANT: set to 1 if only one kafka broker

# increate message size limit
message.max.bytes=60000000
replica.fetch.max.bytes=80000000

log.dirs=/work/kafka-logs
listeners=PLAINTEXT://10.0.0.4:9092

zookeeper.connect=10.0.0.1:2181,10.0.0.2:2181,10.0.0.3:2181

start server

cd /work/kafka
# you could put this line into /etc/rc.local
nohup /work/kafka/bin/kafka-server-start.sh /work/kafka/config/server.properties > /dev/null 2>&1 &

# stop server
# ./bin/kafka-server-stop.sh /work/kafka/config/server.properties

use supervisor

Recommend to use supervisor to manage Kafka service.

apt-get install supervisor

edit conf file vim /etc/supervisor/conf.d/kafka.conf:

[program:kafka]
environment=KAFKA_HEAP_OPTS="-Xmx2G -Xms1G"
directory=/work/kafka
command=/work/kafka/bin/kafka-server-start.sh /work/kafka/config/server.properties
autostart=true
autorestart=true
startsecs=6
startretries=20
$ supervisorctl
> reread
> update
> status

or 
> start/stop kafka
You can’t perform that action at this time.