Skip to content
master
Go to file
Code

Files

Permalink
Failed to load latest commit information.

README.md

docker-kafka-mirror-maker

An HDP 2.5.3 based Docker image for Kafka Mirror Maker. This allows configuration and deployment of Kafka Mirror-Maker in Dockerized environments like Kubernetes/Openshift or AWS ECS.

Build

This image is available from Docker hub however, if you would like to build it yourself here are the steps:

git clone https://github.com/srotya/docker-kafka-mirror-maker.git
cd docker-kafka-mirror-maker
docker build -t mirror-maker:latest .

Note: Docker is expected to be installed where you run the build

Environment Variables

Variable Name Description Defaults
DESTINATION bootstrap.servers for the Destination Kafka Cluster localhost:6667
SOURCE bootstrap.servers for the Source Kafka Cluster localhost:6667
WHITELIST Topics to mirror *
SECURITY If kerberos is enabled, valid options: PLAINTEXT or SASL_PLAINTEXT PLAINTEXT
GROUPID Consumer group id for Kafka consumer _mirror_maker
PRINCIPAL Kerberos security principal name kafka/localhost@EXAMPLE.COM
KEYTAB_FILENAME File name for Kerberos Keytab mirror.keytab

Volumes

Image requires that the keytab file and krb5.conf be mounted if Kerberos is used when deploying this container.

Keytab files are expected in /etc/security/keytabs location

Krb5.conf file is expected in /etc/krb5.conf location

Non-Kerberized Usage

docker run -it -e DESTINATION=xxx.xxx.com:9092 -e SOURCE=xxx.xxx.com:9092 -e WHITELIST=<TOPIC NAME> mirror-maker:latest

Kerberized Usage

docker run -it -e DESTINATION=xxx.xxx.com:6667 -e SECURITY=SASL_PLAINTEXT -e SOURCE=xxx.xxx.com:6667 -e WHITELIST=<TOPIC NAME> -e KEYTAB_FILENAME=kafka.service.keytab -e PRINCIPAL=kafka/xxx.xxx.com@EXAMPLE.COM -v /etc/security/keytabs/:/etc/security/keytabs/ -v /etc/krb5.conf:/etc/krb5.conf mirror-maker:latest

Note: the image automatically applies Kerberos runtime configuration for Mirror Maker if SASL_PLAINTEXT security is configured

Kubernetes Usage

To deploy this image in Kubernetes here's the template.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mirror-maker
spec:
  replicas: 3
  template:
    metadata:
      labels:
        name: mirror-maker
    spec:
      containers:
      - name: mirror-maker
        image: ambuds/mirror-maker
        imagePullPolicy: Always
        env:
        - name: "WHITELIST"
          value: "*"
        - name: "DESTINATION"
          value: "localhost:9092"
        - name: "SOURCE"
          value: "localhost:9092"

License

Apache 2.0

Releases

No releases published

Packages

No packages published

Languages