Skip to content

streamthoughts/jikkou

master
Switch branches/tags
Code

Jikkou : Automate the management of your Apache Kafka using GitOps approach!

Introduction

Jikkou (jikkō / 実行) is an open-source tool to help you automate the management of the configurations that live on your Apache Kafka clusters. It was developed by Kafka ❤️ to make daily operations on an Apache Kafka cluster simpler for both developers and administrators.

It can efficiently manage configuration changes for Topics, ACLs, Quotas and more with the use of resource specification files. It is also applicable to quickly replicate the configuration of a production cluster to another with a few command lines or to initialize a new cluster for testing purpose.

Note: Jikkou is the project formerly known as Kafka Specs. Old versions of this project can still be found under that name.

The main usage scenarios

  • Create new resource entities on an Apache Kafka cluster (i.e., Topics, ACLs, and Quotas).

  • Update the configurations of existing resource entities.

  • Delete resource entities which are not anymore managed.

  • Describe all the configuration defined for Brokers.

  • Describe all the configuration defined for Topics, ACLs, and Quotas.

Core features that make it awesome

  • Simple command line interface (CLI) for end user.

  • Simple Java API on top of the Kafka’s Java AdminClient.

  • Completely stateless and thus does not store any state (Basically: Your kafka cluster is the state of Jikkou).

  • Pluggable validation rules to ensure that resources meet your requirement before being created or updated ona target cluster.

  • Pluggable resource manager to extend Jikkou with cloud managed services for Apache Kafka which are supported out-of-the-box.

  • Simple templating mechanism using Jinja notation.

Quick Installation

Jikkou is available:

wget https://github.com/streamthoughts/jikkou/releases/download/v0.12.2/jikkou.deb
sudo dpkg -i jikkou.deb
Note
Jikkou will install itself in the directory : /opt/jikkou

It is recommended to install the bash/zsh completion script jikkou_completion:

wget https://raw.githubusercontent.com/streamthoughts/jikkou/master/jikkou_completion . jikkou_completion

or alternatively, run the following command for generation the completion script.

source <(jikkou generate-completion)

Overview

It allows you to define the desired current state of your Topics, ACLs, or Quotas, using one or more YAML resource specification files :

kafka-topics.yml
apiVersion: 1
spec:
  topics:
  - name: 'my-first-topic-with-jikkou'
    partitions: 12
    replication_factor: 3
    configs:
      min.insync.replicas: 2

It will then take care of computing and applying the necessary changes directly to your cluster.

$ jikkou --bootstrap-servers localhost:9092 topics apply \
    --file kafka-topics.yml \
    --yes
TASK [CREATE] Create a new topic my-first-topic-with-jikkou (partitions=12, replicas=3) - CHANGED **********************
{
  "changed" : true,
  "end" : 1634071489773,
  "resource" : {
    "name" : "my-first-topic-with-jikkou",
    "operation" : "ADD",
    "partitions" : {
      "after" : 12,
      "operation" : "ADD"
    },
    "replication_factor" : {
      "after" : 3,
      "operation" : "ADD"
    },
    "configs" : {
      "min.insync.replicas" : {
        "after" : "2",
        "operation" : "ADD"
      }
    }
  },
  "failed" : false,
  "status" : "CHANGED"
}
EXECUTION in 2s 661ms (DRY_RUN)
ok : 0, created : 1, altered : 0, deleted : 0 failed : 0

Most of the time, you will use Jikkou part of your CI/CD pipeline allowing you to version and ship your Apache Kafka resources directly from Git (more on GitOps)

Jikkou can be used with self-hosted Kafka, managed Kafka and Confluent Cloud.

Note
you can use the environment variable JIKKOU_DEFAULT_KAFKA_BOOTSTRAP_SERVERS for connection to the Kafka cluster (instead of the CLI arg --bootstrap-servers)

Documentation

Check the official documentation for further installation and usage instructions.

🏭 Developers

You need to have Java and Docker installed.

Dependencies

Jikkou modules are built with:

Build project

This project includes Maven Wrapper.

For building distribution files.

$ ./mvnw clean package -Pdist

Build Docker Images (locally)

$ make

Formats

This project uses the Maven plugin Spotless to format all Java classes and to apply some code quality checks.

Bugs

This project uses the Maven plugin SpotBugs and FindSecBugs to run some static analysis to look for bugs in Java code.

Reported bugs can be analysed using SpotBugs GUI:

$ ./mvnw spotbugs:gui

💡 Contributions

Any feedback, bug reports and PRs are greatly appreciated!

🙏 Show your support

You think this project can help you or your team to manage your Apache Kafka Cluster ? Please this repository to support us!

FAQ

Licence

Copyright 2021 StreamThoughts.

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.See the NOTICE file distributed with this work for additional information regarding copyright ownership.The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License