Skip to content
This repository has been archived by the owner on Nov 7, 2022. It is now read-only.

exporter/prometheusexporter: migrate code in here #479

Closed
wants to merge 4 commits into from
Closed

exporter/prometheusexporter: migrate code in here #479

wants to merge 4 commits into from

Conversation

odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 7, 2019

Hand over prometheus exporter code that I had parked
in https://github.com/orijtech/prometheus-go-metrics-exporter

That code was written for OpenCensus Authors, and was just
waiting for a final destination. This change finishes that.

Fixes #478

@codecov
Copy link

codecov bot commented Mar 7, 2019

Codecov Report

Merging #479 into master will increase coverage by 1.4%.
The diff coverage is 88.65%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #479     +/-   ##
=========================================
+ Coverage   57.02%   58.43%   +1.4%     
=========================================
  Files          69       71      +2     
  Lines        4247     4439    +192     
=========================================
+ Hits         2422     2594    +172     
- Misses       1666     1682     +16     
- Partials      159      163      +4
Impacted Files Coverage Δ
exporter/prometheusexporter/prometheus.go 74.28% <100%> (ø) ⬆️
exporter/prometheusexporter/sanitize.go 100% <100%> (ø)
exporter/prometheusexporter/exporter.go 87.57% <87.57%> (ø)
...al/collector/processor/nodebatcher/node_batcher.go 79.09% <0%> (+0.9%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51944b6...99046b1. Read the comment docs.

@flands flands added this to the 0.1.4 milestone Mar 7, 2019
Copy link

@pjanotti pjanotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @odeke-em, nice to see you getting this to the repo. I'm not done with the review but since I already had some Qs and comments, let's get the ball rolling.

exporter/prometheusexporter/sanitize.go Show resolved Hide resolved
exporter/prometheusexporter/sanitize.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Outdated Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Outdated Show resolved Hide resolved
exporter/prometheusexporter/exporter.go Show resolved Hide resolved
Hand over prometheus exporter code that I had parked
in https://github.com/orijtech/prometheus-go-metrics-exporter

That code was written for OpenCensus Authors, and was just
waiting for a final destination. This change finishes that.

Fixes #478
Bring in sanitize_test.go from OpenCensus-Go.
@odeke-em
Copy link
Member Author

Thank you for the review @pjanotti! I've addressed your feedback, PTAL.

buckets := make([]float64, 0, len(dValue.Buckets))
for index, bucket := range dBuckets {
if _, added := indicesMap[bucket]; !added {
indicesMap[bucket] = index
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @pjanotti, I don't think the check and sort is necessary here. In Proto we already specified bucket bounds are in strictly increasing order.


points := make(map[float64]uint64, len(buckets))
for _, bucket := range buckets {
index := indicesMap[bucket]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, I don't think the reverse lookup is necessary.

exporter/prometheusexporter/exporter.go Show resolved Hide resolved
exporter/prometheusexporter/sanitize.go Show resolved Hide resolved
buckets := make([]float64, 0, len(dValue.Buckets))
for index, bucket := range dBuckets {
if _, added := indicesMap[bucket]; !added {
indicesMap[bucket] = index

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No question that for the Prometheus API used here the histograms need to be cumulative. This is not about making it cumulative for Prometheus but how far we go to support data that is malformed. Giving that code in the service should be in charge of converting to proto before this point it seems that the code gives support to bad receivers.

Count: 1,
Sum: 11.9,
SumOfSquaredDeviation: 0,
Buckets: []*metricspb.DistributionValue_Bucket{

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per proto spec this is not valid proto: you should have one more entry in the Buckets field then what you have on the Bounds


func (c *collector) Describe(ch chan<- *prometheus.Desc) {
c.registeredMetricsMu.Lock()
registered := make(map[string]*prometheus.Desc)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: map capacity hint, yes, it is just a hint but one expects the standard library to take advantage of that (otherwise why implement it :)

allocate the map outside the lock

}
}

var _ http.Handler = (*Exporter)(nil)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: move close to Exporter type.

return desc, signature, ok
}

func (c *collector) registerMetrics(metrics ...*metricspb.Metric) error {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the only usage of this function passes a single *metricspb.Metric so there is no need for the variadic argument.

nit: on the usage site of registerMetrics is always rebuilds the signature, consider returning it.

protoLabelKeysToLabels(metric.GetMetricDescriptor().GetLabelKeys()),
c.opts.ConstLabels,
)
c.registeredMetricsMu.Lock()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the time we the lookup above should succeed, putting that together with the copy that needs to happen from time to time, it seems that a reader-writer mutex will be more appropriate.

return nil, err
}
derivedPrometheusValueType := prometheusValueType(metric)
desc, _, _ := c.lookupPrometheusDesc(c.opts.Namespace, metric)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it guaranteed that the metric signature was already added?

func (c *collector) ensureRegisteredOnce() error {
var finalErr error
c.registerOnce.Do(func() {
if err := c.registry.Register(c); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q.: are the costs of simply registering at construction too high? what are the downsides if registered and no metric ever registered?

@flands flands modified the milestones: 0.1.5, 0.1.6 Mar 28, 2019
@flands flands removed this from the 0.1.6 milestone Apr 24, 2019
Copy link
Contributor

@songy23 songy23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@odeke-em
Copy link
Member Author

odeke-em commented Jun 4, 2019

Sorry for the late reply @songy23! I have been swamped with work. Sure we can definitely move it over there but I'll get free cycles perhaps in the next 36 hours to move that over to the new repo. I shall close this PR and add an action item for myself.

@songy23
Copy link
Contributor

songy23 commented Jun 4, 2019

No worries, I filed census-ecosystem/opencensus-go-exporter-prometheus#11 to keep track of that.

@flands flands added this to the 0.1.8 milestone Jun 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exporter/prometheus: bring in source code from Orijtech repo
5 participants