Admin Configs API

Important

GitHub Wiki is just a mirror of our online documentation.

We highly recommend using our website docs due to Github Wiki limitations. Only some illustrations, links, screencasts, and code examples will work here, and the formatting may be broken.

Please use https://karafka.io/docs.

---

The Karafka Admin Configs API provides tools for managing configuration settings for Kafka brokers and topics. This API supports retrieving configuration details (describe) and incremental alterations (alter) to these configurations. The operations are designed to be flexible, supporting both individual and batch operations.

!!! Tip "Declarative Topics Feature For Topics Management"

The Admin Configs API provides low-level access for managing Kafka topics and broker configurations. While powerful, it requires detailed knowledge and careful management of individual settings. For a more streamlined and error-resistant approach, consider using the high-level [declarative topics](https://karafka.io/docs/Declarative-Topics/) feature provided by Karafka. This feature allows for easier and more declarative management of topic configurations, making it a superior choice for most use cases.

What are Kafka Configurations?

Kafka configurations are key-value pairs that define the behavior of brokers and topics. Managing these configurations properly is essential for optimizing performance, ensuring security, and maintaining reliability in your Kafka environment.

Types

The Karafka Admin Configs API involves several types that categorize the details of configuration management in Kafka. Understanding these types is crucial for effective configuration management.

Resource Types

Resource types refer to Kafka entities that can be configured, such as topics or brokers. These are fundamental to identifying which parts of your Kafka cluster you are managing.

Type	Description
:topic	Represents a single Kafka topic.
:broker	Represents a specific Kafka broker.

Operation Types

Operations types within this API are used to specify the configuration changes being applied, such as setting or deleting values.

Operation Type	Description
:set	Sets or updates a configuration value.
:delete	Removes a configuration entry.
:append	Adds a value to a list-based configuration.
:subtract	Removes a value from a list-based configuration.

Methods

Method	Description	Arguments	Returns	Notes
describe(*resources)	Retrieves configurations for specified Kafka resources.	resources - A single resource or an array of resources.	An array of Resource objects with configuration details.	Fetches configuration details in a single operation, even for multiple resources.
alter(*resources)	Applies configuration changes to specified Kafka resources using incremental updates.	resources - A single resource or an array of resources.	Applies the changes and provides a confirmation of the update.	This method is non-transactional and may succeed partially; validate configurations before applying.

These methods provide a robust interface for detailed management of configurations in a Kafka environment, offering both retrieval and update functionalities.

Usage

Here's how you might typically use the API to manage Kafka configurations:

# Describe topic configurations
resource = Karafka::Admin::Configs::Resource.new(type: :topic, name: 'example')
topics = Karafka::Admin::Configs.describe(resource)
topics.each do |topic|
  topic.configs.each do |config|
    puts "#{topic.name} #{config.name}: #{config.value}"
  end
end

# Alter topic configurations
resource = Karafka::Admin::Configs::Resource.new(type: :topic, name: 'example')
# Set retention to 2 hours
resource.set('retention.ms', '7200000')

# Apply the changes
Karafka::Admin::Configs.alter(resource)

Example Use Cases

• Configuration Audits: Retrieve configurations to review and ensure compliance with operational standards.

• Dynamic Configuration Tuning: Adjust topic or broker configurations in response to changes in usage patterns or operational requirements.

• Batch Configuration Management: Simultaneously update configurations for multiple brokers or topics, improving efficiency in large-scale environments.