Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/exp/metrics: identity types #31017

Merged
merged 10 commits into from
Feb 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ internal/collectd/ @open-telemetry/collect
internal/coreinternal/ @open-telemetry/collector-contrib-approvers @open-telemetry/collector-approvers
internal/datadog/ @open-telemetry/collector-contrib-approvers @mx-psi @dineshg13
internal/docker/ @open-telemetry/collector-contrib-approvers @rmfitzpatrick @jamesmoessis
internal/exp/metrics/ @open-telemetry/collector-contrib-approvers @sh0rez @RichieSams
internal/filter/ @open-telemetry/collector-contrib-approvers @open-telemetry/collector-approvers
internal/k8sconfig/ @open-telemetry/collector-contrib-approvers @dmitryax
internal/k8stest/ @open-telemetry/collector-contrib-approvers @crobert-1
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ body:
- internal/core
- internal/datadog
- internal/docker
- internal/exp/metrics
- internal/filter
- internal/k8sconfig
- internal/k8stest
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ body:
- internal/core
- internal/datadog
- internal/docker
- internal/exp/metrics
- internal/filter
- internal/k8sconfig
- internal/k8stest
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ body:
- internal/core
- internal/datadog
- internal/docker
- internal/exp/metrics
- internal/filter
- internal/k8sconfig
- internal/k8stest
Expand Down
1 change: 1 addition & 0 deletions internal/exp/metrics/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../../Makefile.Common
28 changes: 28 additions & 0 deletions internal/exp/metrics/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics

replace github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil => ../../../pkg/pdatautil

require (
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.0.0-00010101000000-000000000000
go.opentelemetry.io/collector/pdata v1.1.0
)

require (
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/net v0.18.0 // indirect
golang.org/x/sys v0.14.0 // indirect
golang.org/x/text v0.14.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231106174013-bbf56f31fb17 // indirect
google.golang.org/grpc v1.61.0 // indirect
google.golang.org/protobuf v1.32.0 // indirect
)

go 1.21

toolchain go1.21.1
78 changes: 78 additions & 0 deletions internal/exp/metrics/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions internal/exp/metrics/identity/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

// identity types for metrics and sample streams.
//
// Use the `Of*(T) -> I` functions to obtain a unique, comparable (==) and
// hashable (map key) identity value I for T.
package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"
71 changes: 71 additions & 0 deletions internal/exp/metrics/identity/metric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"

import (
"hash"

"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/pmetric"
)

type metric = Metric

type Metric struct {
scope

name string
unit string
ty pmetric.MetricType

monotonic bool
temporality pmetric.AggregationTemporality
}

func (i Metric) Hash() hash.Hash64 {
sum := i.scope.Hash()
sum.Write([]byte(i.name))
sum.Write([]byte(i.unit))

var mono byte
if i.monotonic {
mono = 1
}
sum.Write([]byte{byte(i.ty), mono, byte(i.temporality)})
return sum
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add functions for each identity struct to return the "inner" ID? For example:

func (i Metric) Scope() Scope {
    return i.scope
}

So Stream has a Metric(), Metric has a Scope(), and Scope has a Resource().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IE: d9b3edd

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes absolutely, already got to this conclusion during local testing

func (i Metric) Scope() Scope {
return i.scope
}

func OfMetric(scope Scope, m pmetric.Metric) Metric {
id := Metric{
scope: scope,
name: m.Name(),
unit: m.Unit(),
ty: m.Type(),
}

switch m.Type() {
case pmetric.MetricTypeSum:
sum := m.Sum()
id.monotonic = sum.IsMonotonic()
id.temporality = sum.AggregationTemporality()
case pmetric.MetricTypeExponentialHistogram:
exp := m.ExponentialHistogram()
id.monotonic = true
id.temporality = exp.AggregationTemporality()
case pmetric.MetricTypeHistogram:
hist := m.Histogram()
id.monotonic = true
id.temporality = hist.AggregationTemporality()
}

return id
}

func OfResourceMetric(res pcommon.Resource, scope pcommon.InstrumentationScope, metric pmetric.Metric) Metric {
return OfMetric(OfScope(OfResource(res), scope), metric)
}
31 changes: 31 additions & 0 deletions internal/exp/metrics/identity/resource.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"

import (
"hash"
"hash/fnv"

"go.opentelemetry.io/collector/pdata/pcommon"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil"
)

type resource = Resource

type Resource struct {
attrs [16]byte
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include the resource SchemaURL as part of the identifier?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no idea, can someone clarify? @jpkrohling @djaglowski

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This came up in the maintainers meeting yesterday and consensus seemed to be that most language libraries are not setting this field. Technically it is optional so I would be ok with leaving it out for now and adding later if necessary.

}

func (r Resource) Hash() hash.Hash64 {
sum := fnv.New64a()
sum.Write(r.attrs[:])
return sum
}

func OfResource(r pcommon.Resource) Resource {
return Resource{
attrs: pdatautil.MapHash(r.Attributes()),
}
}
43 changes: 43 additions & 0 deletions internal/exp/metrics/identity/scope.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"

import (
"hash"

"go.opentelemetry.io/collector/pdata/pcommon"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil"
)

type scope = Scope

type Scope struct {
resource resource

name string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include the scope schemaURL as part of the identifier?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave it out IMO

version string
attrs [16]byte
}

func (s Scope) Hash() hash.Hash64 {
sum := s.resource.Hash()
sum.Write([]byte(s.name))
sum.Write([]byte(s.version))
sum.Write(s.attrs[:])
return sum
}

func (s Scope) Resource() Resource {
return s.resource
}

func OfScope(res Resource, scope pcommon.InstrumentationScope) Scope {
return Scope{
resource: res,
name: scope.Name(),
version: scope.Version(),
attrs: pdatautil.MapHash(scope.Attributes()),
}
}
37 changes: 37 additions & 0 deletions internal/exp/metrics/identity/stream.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"

import (
"hash"

"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/pmetric"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil"
)

type Stream struct {
metric
attrs [16]byte
}

func (i Stream) Hash() hash.Hash64 {
sum := i.metric.Hash()
sum.Write(i.attrs[:])
return sum
}

func (i Stream) Metric() Metric {
return i.metric
}

func OfStream[DataPoint attrPoint](m Metric, dp DataPoint) Stream {
return Stream{metric: m, attrs: pdatautil.MapHash(dp.Attributes())}
}

type attrPoint interface {
pmetric.NumberDataPoint | pmetric.HistogramDataPoint | pmetric.ExponentialHistogramDataPoint | pmetric.SummaryDataPoint
Attributes() pcommon.Map
}
24 changes: 24 additions & 0 deletions internal/exp/metrics/identity/strings.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package identity // import "github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"

import (
"fmt"
)

func (r Resource) String() string {
return fmt.Sprintf("resource/%x", r.Hash().Sum64())
}

func (s Scope) String() string {
return fmt.Sprintf("scope/%x", s.Hash().Sum64())
}

func (m Metric) String() string {
return fmt.Sprintf("metric/%x", m.Hash().Sum64())
}

func (s Stream) String() string {
return fmt.Sprintf("stream/%x", s.Hash().Sum64())
}
3 changes: 3 additions & 0 deletions internal/exp/metrics/metadata.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
status:
codeowners:
active: [sh0rez, RichieSams]
1 change: 1 addition & 0 deletions versions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ module-sets:
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/common
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/docker
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/k8sconfig
- github.com/open-telemetry/opentelemetry-collector-contrib/internal/k8stest
Expand Down
Loading