Add prometheus metrics #65
Conversation
|
Please choose a different port than 51678. |
ipamd/datastore/data_store.go
Outdated
| assignedIPs = prometheus.NewGauge( | ||
| prometheus.GaugeOpts{ | ||
| Name: "assigned_ip_addresses", | ||
| Help: "The number of ip addresses assigned", |
There was a problem hiding this comment.
please capitalize initialisms IP and ENI in Help text. also eni should be plural ENIs
pkg/awsutils/awsutils.go
Outdated
| awsAPILatency = prometheus.NewSummaryVec( | ||
| prometheus.SummaryOpts{ | ||
| Name: "aws_api_lantency_ms", | ||
| Help: "aws API call latency in ms", |
pkg/awsutils/awsutils.go
Outdated
| awsAPIErr = prometheus.NewCounterVec( | ||
| prometheus.CounterOpts{ | ||
| Name: "aws_api_error_count", | ||
| Help: "the number of times aws API returns an err", |
| if err != nil { | ||
| awsAPIErrInc("TagResources", err) | ||
| log.Warnf("Fail to tag the newly created eni %s %v", eniID, err) |
There was a problem hiding this comment.
If we're counting this as an Error, why do we log as a Warning? Might be a good idea to make a function logAndTallyErrorf since we appear to always want them both together.
ipamd/datastore/data_store.go
Outdated
| Help: "The number of ip addresses assigned", | ||
| }, | ||
| ) | ||
| ) |
There was a problem hiding this comment.
It appears we're tracking and emiting metrics on eni count, total address, and assigned address. One can derived Available address as Total - Assigned, but it might be nice to emit that directly. Also would be nice to emit Max address (ENIs * IPsPerENI). Addresses go through a cool down period, might be nice to offer visibility into that too.
liwenwu-amazon
force-pushed
the
metrics
branch
2 times, most recently
from
155dddd
to
8eb0758
Compare
Descriptions:
Add following prometheus metrics which provide metrics at localhost:61678/metrics
Also add a tool cni-metrics-helper which can collect CNI prometheus metrics from all nodes in the cluster by
Tests Performed
Deployed 4000 busybox pods over 80 nodes. Verify the prometheus metrics from all 80 nodes, that aggregated eni_allocated is 320, aggregated total_ip_addresses is 4.48k, assigned_ip_addresses is 4k
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.