Kafka Canary

This is a kafka-go implementation of the strimzi-canary, with the goal of being independant of the Strimzi platform while taking a lot of inspiration from it. The canary tool acts as an indicator of whether Apache Kafka® clusters are operating correctly. This is achieved by creating a canary topic and periodically producing and consuming events on the topic and getting metrics out of these exchanges.

Motivation

The kafka-go library provides a better interface than sarama which is used by strimzi-canary and its easier to maintain. Furthermore, strimzi-canary goals are aligned with those of the Strimzi platform. The goal of this project is to provide a general canary implementation, to be used with Strimzi, DYI Kafka, AWS MSK, etc.

State

⚠️ Beta ⚠️

• Tests
  • Client interface
  • Service behaviour
    • Expected metrics increase
    • Expected handling
      • Topics does not exist
      • Topic partitions diff
      • Topic config diff
      • Topic change
  • Integration tests
• Implement core monitors
  • Topic activities
  • Producer
  • Consumer
• Configuration
  • Environment variables
  • Configuration File
  • Arguments
• Support files
  • Kuberentes deployment example
  • Helm chart
• Usage docs

Requirements

• Go >= 1.20

Optional

• direnv
• Nix with Flakes

Thanks

• strimzi-canary - For the original idea and implementation
• topicctl - For the client implementation