Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(*) add kafka protocol suport #1121

Merged
merged 6 commits into from
Nov 3, 2020
Merged

feat(*) add kafka protocol suport #1121

merged 6 commits into from
Nov 3, 2020

Conversation

nickolaev
Copy link
Contributor

@nickolaev nickolaev commented Oct 30, 2020

Summary

add support for Kafka protocol statistics as per https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/kafka_broker_filter#statistics

Prometheus statistics as follows:

# TYPE envoy_kafka__request counter
envoy_kafka__request{kafka_name="localhost_9092",kafka_type="add_offsets_to_txn_request"} 0
envoy_kafka__request{kafka_name="localhost_9092",kafka_type="add_partitions_to_txn_request"} 0
envoy_kafka__request{kafka_name="localhost_9092",kafka_type="alter_configs_request"} 0
envoy_kafka__request{kafka_name="localhost_9092",kafka_type="alter_partition_reassignments_request"} 0
envoy_kafka__request{kafka_name="localhost_9092",kafka_type="alter_replica_log_dirs_request"} 0
...

# TYPE envoy_kafka__response counter
envoy_kafka__response{kafka_name="localhost_9092",kafka_type="add_offsets_to_txn_response"} 0
envoy_kafka__response{kafka_name="localhost_9092",kafka_type="add_partitions_to_txn_response"} 0
envoy_kafka__response{kafka_name="localhost_9092",kafka_type="alter_configs_response"} 0
envoy_kafka__response{kafka_name="localhost_9092",kafka_type="alter_partition_reassignments_response"} 0
envoy_kafka__response{kafka_name="localhost_9092",kafka_type="alter_replica_log_dirs_response"} 0
...

Documentation

Signed-off-by: Nikolay Nikolaev <nikolay.nikolaev@konghq.com>
@nickolaev nickolaev requested a review from a team as a code owner October 30, 2020 17:53
func (c *KafkaConfigurer) Configure(filterChain *envoy_listener.FilterChain) error {
pbst, err := proto.MarshalAnyDeterministic(
&envoy_kafka.KafkaBroker{
StatPrefix: kafkaStatPrefix,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be listener name, look at TCP Proxy stat prefix and pass the same value

kafkaStatPrefix = "kstat"
)

func Kafka() FilterChainBuilderOpt {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to convert kafka stats to have proper prometheus metrics with two labels

  • TYPE
  • name (from stats prefix)

Nikolay Nikolaev added 2 commits November 3, 2020 14:43
Signed-off-by: Nikolay Nikolaev <nikolay.nikolaev@konghq.com>
Nikolay Nikolaev added 2 commits November 3, 2020 17:18
Signed-off-by: Nikolay Nikolaev <nikolay.nikolaev@konghq.com>
@nickolaev nickolaev merged commit 415be1d into master Nov 3, 2020
@nickolaev nickolaev deleted the feat/kafka_support branch November 3, 2020 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants