systemd services for Confluent Platform

Available for:

• Confluent Kafka
• Confluent Schema Registry
• Confluent Kafka Connect
• Confluent ZooKeeper
• Confluent Control Center

Prerequisites

Users, groups and directories

• Create the users and group:
  • kafka
  • zookeeper
  • confluent-cc
• Create directories:
  • /var/log/kafka
  • /var/log/zookeeper
  • /var/log/confluent-control-center
• Set appropriate permissions

Kafka Connect Log4j settings

Edit /etc/kafka/connect-log4j.properties, add/change the following lines

log4j.rootLogger=INFO, kafkaConnectAppender

log4j.appender.kafkaConnectAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.kafkaConnectAppender.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.kafkaConnectAppender.File=${kafka.logs.dir}/connect.log
log4j.appender.kafkaConnectAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.kafkaConnectAppender.layout.ConversionPattern=[%d] %p %m (%c)%n

Confluent Control Center

For logging into files, rename log4j-rolling.properties to log4j.properties.
By default Control Center will log to /tmp. To change, set the paths in log4j.properties after renaming.

Installation

Put the unit file into the location: /etc/systemd/system/
Reload systemd:

systemctl daemon-reload

For auto restart of these services use:

systemctl enable servicename.service

Additional settings

Some Apache Kafka settings are set via JVM parameters and environment variables. For example security settings or log paths.

To add environment parameters for systemd services "Environment" entries can be added to the "Service" section of a service. As an example the following line can be added to specify the path to the jaas configuration file needed to enable Kerberos for authentication.

Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/kafka-jaas.conf"

More information about Apache Kafka settings and environment variables can be found in the official documentation.

Summary

JMX is enabled by default. To disable JXM remove the "Environment=" line.

Confluent Kafka

Kafka properties: /etc/kafka/server.properties
Logs: /var/log/kafka
JMX Port: 10030

Confluent Schema Registry

Schema Registry properties: /etc/schema-registry/schema-registry.properties
JMX Port: 10050

Confluent Kafka Connect

Kafka Connect properties: /etc/kafka/connect-distributed.properties
Log: /var/log/kafka/connect.log
JMX Port: 10040

Confluent ZooKeeper

ZooKeeper properties: /etc/kafka/zookeeper.properties
Logs: /var/log/zookeeper
JMX Port: 10020