Skip to content

Operator to provide Kafka Schema in declarative way using KubernetesCustomResources

License

Notifications You must be signed in to change notification settings

pannoi/kafka-schema-operator

Repository files navigation

Kafka Schema Operator

Artifact Hub

The Kafka Schema Operator delivers as easy way to deliver Kafka Schemas declaratively via Kubernetes CRD

Schema First approach should be implemented if you are using schema-registry according to best practices

Operator schema registry compatibility

Operator features

  • Create schemas declaratively via CRD
  • Automatic schema versions update
  • GitOps (automatic updates)

Installation

helm repo add kafka-schema-operator https://pannoi.github.io/kafka-schema-operator-helm/
helm repo update
helm upgrade --install kafka-schema-operator kafka-schema-operator/kafka-schema-operator --values values.yaml

You need to set SCHEMA_REGSITRY_HOST and SCHEMA_REGSITRY_PORT initial functionality

schemaRegistry:
  host:
  port:

If your schema registry needs API authentication add Key and Secret values in addition to host and port

schemaRegistry:
  apiKey:
  apiSecret:

You can refer from secretRef

For more values check default values

How to use

Deploy ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-schema
  namespace: default
data:
  schema: |
    {
      "namespace": "testing",
      "type": "record",
      "name": "testing",
      "fields": [
        {"name": "id", "type": "string"},
        {"name": "email", "type": "string"}
      ]
    }

Refer to created ConfigMap via CustomResource KafkaSchema

apiVersion: kafka-schema-operator.pannoi/v1beta1
kind: KafkaSchema
metadata:
  name: kafka-schema
  namespace: default
spec:
  name: testing
  schemaSerializer: string
  autoReconciliation: true # true = autoUpdate schema, false = for update CR should be re-created (not set => false)
  terminationProtection: true # true = don't delete resources on CR deletion, false = when CR deleted, deletes all resource: ConfigMap, Schema from registry (not set => false)
  data:
    configRef: kafka-schema # ConfigMap
    format: avro # avro/protobuf/json
    compatibility: # BACKWARD | BACKWARD_TRANSITIVE | FORWARD | FORWARD_TRANSITIVE | FULL | FULL_TRANSITIVE | NONE

Resources (KafkaSchema & ConfigMap) should be located in same namespace