Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka exporter does not support zstd compression #26365

Closed
ikstewa opened this issue Aug 31, 2023 · 2 comments
Closed

Kafka exporter does not support zstd compression #26365

ikstewa opened this issue Aug 31, 2023 · 2 comments
Labels
bug Something isn't working exporter/kafka needs triage New item requiring triage

Comments

@ikstewa
Copy link

ikstewa commented Aug 31, 2023

Component(s)

exporter/kafka

What happened?

Description

Documentation stats zstd is supported however an older version of sarama is being used which does not support zstd.

Steps to Reproduce

Set zstd compression on a kafka exporter:

  kafka:
    brokers:
      - kafka:9092
    topic: "apo_metrics_otlp_json"
    encoding: otlp_json
    producer:
      compression: zstd

Expected Result

zstd compression is supported

Actual Result

Collector fails to start up with the following error:

2023-08-31 16:10:18 Error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)
2023-08-31 16:10:18 2023/08/31 23:10:18 collector server run finished with error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)

Collector version

otel/opentelemetry-collector-contrib:0.83.0

Environment information

Environment

docker

OpenTelemetry Collector configuration

No response

Log output

2023-08-31 16:10:18 Error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)
2023-08-31 16:10:18 2023/08/31 23:10:18 collector server run finished with error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)

Additional context

No response

@ikstewa ikstewa added bug Something isn't working needs triage New item requiring triage labels Aug 31, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@ikstewa
Copy link
Author

ikstewa commented Aug 31, 2023

Confirmed the issue was due to missing protocol_version field. Setting to a value > 2.1.0 fixed the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/kafka needs triage New item requiring triage
Projects
None yet
Development

No branches or pull requests

1 participant