Expected metrics from Kafka is not Shown

[ericashi]

Component(s)

receiver/kafkametrics

What happened?

Description

I am trying to get metrics from a Kafka Instance. When I looked at the metrics collected, it only shows the properties of the Topics and not the metrics of the Topics.

Steps to Reproduce

Configure the .yaml configuration file, then run the .yaml configuration file using the otelcol-contrib binary.
Please refer to the "OpenTelemetry Collector configuration" section for the configurations.

Expected Result

A set of metrics from a specific topics.
Sample metrics expected:

nfs_read_data
nfs_read_latency
nfs_write_data
nfs_write_latency

Actual Result

I got a set of metrics which is considered as the properties of the topics.
Sample metrics gotten:

kafka.consumer_group.lag
kafka.consumer_group.offset
kafka.partition.current_offset

Collector version

v0.77.0

Environment information

Environment

OS: x86-64_linux_4.12_ImageSLES12SP5

OpenTelemetry Collector configuration

receivers:
  kafkametrics:
    brokers: <BROKER_TARGET>
    protocol_version: 2.0.0
    scrapers:
      - brokers
      - topics
      - consumers

processors:
  batch:

exporters:
  logging:
    verbosity: detailed

service:
  telemetry:
    metrics:
      level: detailed
      address: 0.0.0.0:8886

  pipelines:
    metrics:
      receivers: [kafkametrics]
      processors: []
      exporters: [logging]

Log output

No response

Additional context

May I know if I have configured correctly, please?
How can I configure to get the metrics that I am expecting, please?

Thank you.

[github-actions]

Pinging code owners:

• receiver/kafkametrics: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafkametrics: @dmitryax
• needs: Github issue template generation code needs this to generate the corresponding labels.

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Hello @ericashi, my understanding is that the metrics you're receiving from the kafmametrics receiver are what's expected, as defined here.

You mentioned your expected metrics, can you share why those metrics would have been expected?

[ericashi]

Hi @crobert-1 ,

I am trying to collect the metrics data from a specific topic. For instance, in a topic there are "field A", "field B", and "field C", I would like to get the field and value from the topic.

Thank you.

Best Regards,
Erica

[crobert-1]

Hello @ericashi, I was able to take another look at this, the kafkametrics receiver is using the sarama package to get metrics from kafka, so it's limited by the metrics exposed by this client.

There's a list of metrics exposed here, with some topic metrics for producers and consumers. Do any of these fit your use case?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/kafkametrics: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.