Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add histogram support in metricstest #4177

Merged
merged 10 commits into from
Apr 26, 2023
Merged

Add histogram support in metricstest #4177

merged 10 commits into from
Apr 26, 2023

Conversation

k24dizzle
Copy link
Contributor

When you use the metricstest handler to record a histogram, you can now compare the resulting histogram bucket values.

Unlike counter or gauge samples which return a float64, for histogram samples, I introduced a new struct that contains a list of histogramBucket(s), that contain a value and upperBound.

@k24dizzle k24dizzle requested a review from a team as a code owner April 17, 2023 22:25
@@ -55,29 +57,59 @@ type (
sampleValue float64
}

histogramBucket struct {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open question: Should this be uppercased? So other packages can use this struct when running histogram tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b/c lint complained, answer is yes

common/metrics/metricstest/metricstest.go Show resolved Hide resolved
common/metrics/metricstest/metricstest.go Outdated Show resolved Hide resolved
common/metrics/metricstest/metricstest.go Outdated Show resolved Hide resolved
common/metrics/metricstest/metricstest.go Outdated Show resolved Hide resolved
common/metrics/metricstest/metricstest.go Show resolved Hide resolved
common/metrics/metricstest/metricstest.go Outdated Show resolved Hide resolved
common/metrics/metricstest/metricstest_test.go Outdated Show resolved Hide resolved
common/metrics/metricstest/metricstest_test.go Outdated Show resolved Hide resolved
Comment on lines 257 to 259
if sample.metricType != dto.MetricType_HISTOGRAM {
return nil, fmt.Errorf("metric %s not a %s type", name, dto.MetricType_HISTOGRAM.String())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get rid of this because this is the same as the condition we check before adding the sample

Copy link
Contributor Author

@k24dizzle k24dizzle Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 ...
	gauge := handler.WithTags(gaugeTags...).Gauge(gaugeName)
	gauge.Record(-2)

	s2, err := handler.Snapshot()
	require.NoError(t, err)

	gaugeVal, err = s2.Counter(gaugeName+"_total", expectedGaugeTags...)
        require.NoError(t, err)

I think these errors are mainly here to protect the test writer from making incorrect assumptions. For example for the test above, the test writer incorrectly thinks that a counter metric named: gaugeName+"_total" exists when its really a gauge.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I see. Makes sense to keep then

Comment on lines 260 to 261
if !maps.Equal(sample.labelValues, labelValues) {
return nil, fmt.Errorf("metric %s label mismatch, has %v, asked for %v", name, sample.labelValues, labelValues)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should actually just ignore this sample instead of returning an error because that's how PromQL actually works. We should be operating on the samples whose labels match

Comment on lines 254 to 256
if !ok {
return nil, fmt.Errorf("metric %s not found", name)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this should be an error. I think we should return an empty list--that's how PromQL would behave

Copy link
Contributor Author

@k24dizzle k24dizzle Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These error checks are here to protect us/quickly signal any changes the underlying handler makes like adding more default labels or changing metric names.

https://github.com/temporalio/temporal/pull/3850/files#r1088378482

Here's one that this test case would've caught ^

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Ok, yeah that makes sense

Copy link
Contributor

@MichaelSnowden MichaelSnowden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the 3 errors we return in Snapshot.Histogram, I think we should actually remove all of them. Other than that, things LGTM and thanks for making the changes

@MichaelSnowden
Copy link
Contributor

For the error-wrapping, we shouldn't be creating custom error classes, we should just do this:

var (
  ErrMetricNotFound = errors.New("metric not found")
  Err...
)

func (s Snapshot) Histogram(name string, tags ...metrics.Tag) ([]HistogramBucket, error) {
  ...
  if !ok {
    return nil, fmt.Errorf("%w: %q", ErrMetricNotFound, name)
  }
  ...
}

@k24dizzle k24dizzle enabled auto-merge (squash) April 26, 2023 22:54
@k24dizzle k24dizzle merged commit ccbfa7d into master Apr 26, 2023
@k24dizzle k24dizzle deleted the kevin/histogramtest branch April 26, 2023 23:36
samanbarghi pushed a commit to samanbarghi/temporal that referenced this pull request May 2, 2023
* Add histogram support in metricstest

* Address lint

* Address comments

* Address comments pt. 2

* Reorder fields

* Wrap errors correctly

* Use Err... naming convention for err vars
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants