This is the Kafka plugin for collectd. It will send data about Kafka to SignalFx, enabling built-in Kafka monitoring dashboards.
Broker: Focus on a single Kafka broker.
Brokers: Overview of data from all Kafka brokers.
clusterdimension which is available in the SignalFx Agent can be used to get a per cluster view of Brokers.
Producer: Focus on a single Java based Producer.
Producers: Overview of Java based Producers.
Consumer: Focus on a single Java based Consumer.
Consumers: Overview of Java based Consumers.
JVM: Focus on the java virtual machine performance on a single instance.
Note: Metrics from Java based Kafka consumers and producers are available by default only when using the SignalFx Agent.
REQUIREMENTS AND DEPENDENCIES
|collectd||4.9 or later|
If you are using the new Smart Agent, see the docs for the collectd/kafka monitor for more information. The configuration documentation below may be helpful as well, but consult the Smart Agent repo's docs for the exact schema.
RHEL/CentOS and Amazon Linux users: Install the Java plugin for collectd if it is not already installed.
Download SignalFx's example Kafka configuration file to
Note: If you're using a version of Kafka earlier than v0.8.2, download this sample Kafka configuration file instead_: 20-kafka.conf
Modify your Kafka configuration file to provide values that make sense for your environment, as described in Configuration, below.
Using the example configuration file 20-kafka.conf as a guide, provide values for the configuration options listed below that make sense for your environment and allow you to connect to the Kafka instance to be monitored.
|ServiceURL||URL of your JMX application.|
|Host||The name of your host. Appears as dimension
Sample of built-in dashboard in SignalFx:
For a comprehensive list of metrics, other the ones available by default, see here.
Note that the metrics prefixed by
kafka.producer are available only via the
kafka_producer monitors of SignalFx Agent. Also, if using the SignalFx Agent, metrics from Broker will be added with
a user provided
This integration is released under the Apache 2.0 license. See LICENSE for more details.