Skip to content

[enrichments] Add support for inferred spans #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 29, 2024
Merged

[enrichments] Add support for inferred spans #116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea8d383
[enrichments] Add support for inferred spans
lahsivjar Oct 28, 2024
45c8777
Fix specs: remove span link if attributes match
lahsivjar Oct 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions enrichments/trace/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ type ElasticTransactionConfig struct {
Type AttributeConfig `mapstructure:"type"`
Result AttributeConfig `mapstructure:"result"`
EventOutcome AttributeConfig `mapstructure:"event_outcome"`
InferredSpans AttributeConfig `mapstructure:"inferred_spans"`
}

// ElasticSpanConfig configures the enrichment attributes for the spans
Expand All @@ -73,6 +74,7 @@ type ElasticSpanConfig struct {
EventOutcome AttributeConfig `mapstructure:"event_outcome"`
ServiceTarget AttributeConfig `mapstructure:"service_target"`
DestinationService AttributeConfig `mapstructure:"destination_service"`
InferredSpans AttributeConfig `mapstructure:"inferred_spans"`
}

// SpanEventConfig configures enrichment attributes for the span events.
Expand Down Expand Up @@ -121,6 +123,7 @@ func Enabled() Config {
Result: AttributeConfig{Enabled: true},
EventOutcome: AttributeConfig{Enabled: true},
RepresentativeCount: AttributeConfig{Enabled: true},
InferredSpans: AttributeConfig{Enabled: true},
},
Span: ElasticSpanConfig{
TimestampUs: AttributeConfig{Enabled: true},
Expand All @@ -132,6 +135,7 @@ func Enabled() Config {
ServiceTarget: AttributeConfig{Enabled: true},
DestinationService: AttributeConfig{Enabled: true},
RepresentativeCount: AttributeConfig{Enabled: true},
InferredSpans: AttributeConfig{Enabled: true},
},
SpanEvent: SpanEventConfig{
TimestampUs: AttributeConfig{Enabled: true},
Expand Down
1 change: 1 addition & 0 deletions enrichments/trace/internal/elastic/attributes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ const (
AttributeSpanDestinationServiceResource = "span.destination.service.resource"
AttributeSpanDurationUs = "span.duration.us"
AttributeSpanRepresentativeCount = "span.representative_count"
AttributeChildIDs = "child.id"

// span event attributes
AttributeParentID = "parent.id"
Expand Down
30 changes: 30 additions & 0 deletions enrichments/trace/internal/elastic/span.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,9 @@ func (s *spanEnrichmentContext) enrichTransaction(
if cfg.EventOutcome.Enabled {
s.setEventOutcome(span)
}
if cfg.InferredSpans.Enabled {
s.setInferredSpans(span)
}
}

func (s *spanEnrichmentContext) enrichSpan(
Expand Down Expand Up @@ -266,6 +269,9 @@ func (s *spanEnrichmentContext) enrichSpan(
if cfg.DestinationService.Enabled {
s.setDestinationService(span)
}
if cfg.InferredSpans.Enabled {
s.setInferredSpans(span)
}
}

// normalizeAttributes sets any dependent attributes that
Expand Down Expand Up @@ -456,6 +462,30 @@ func (s *spanEnrichmentContext) setDestinationService(span ptrace.Span) {
}
}

func (s *spanEnrichmentContext) setInferredSpans(span ptrace.Span) {
spanLinks := span.Links()
childIDs := pcommon.NewSlice()
spanLinks.RemoveIf(func(spanLink ptrace.SpanLink) (remove bool) {
spanID := spanLink.SpanID()
spanLink.Attributes().Range(func(k string, v pcommon.Value) bool {
switch k {
case "is_child", "elastic.is_child":
if v.Bool() && !spanID.IsEmpty() {
Copy link
Contributor Author

@lahsivjar lahsivjar Oct 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[For reviewers] In APM data we add an empty string in the slice if span ID is empty (ref), however, I didn't think it was important so didn't handle it here. Let me know if it needs to be added.

Copy link

@JonasKunz JonasKunz Oct 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that was not intentionally done in apm-data for that code path. I don't think span links are allowed to have an empty span ID in OTLP?

Copy link
Contributor Author

@lahsivjar lahsivjar Oct 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think span links are allowed to have an empty span ID in OTLP?

I am not sure about this, I kept the span ID check for now, let me know if you think we should remove this.

Copy link

@JonasKunz JonasKunz Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In APM data we add an empty string in the slice if span ID is empty (ref), however, I didn't think it was important so didn't handle it here

Just wanted to agree with you here. I'd keep the span ID check as is just as a safety measure.

remove = true // remove the span link if it has the child attrs
childIDs.AppendEmpty().SetStr(hex.EncodeToString(spanID[:]))
}
return false // stop the loop
}
return true
})
return remove
})

if childIDs.Len() > 0 {
childIDs.MoveAndAppendTo(span.Attributes().PutEmptySlice(AttributeChildIDs))
}
}

type spanEventEnrichmentContext struct {
exceptionType string
exceptionMessage string
Expand Down
133 changes: 113 additions & 20 deletions enrichments/trace/internal/elastic/span_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (

"github.com/elastic/opentelemetry-lib/enrichments/trace/config"
"github.com/google/go-cmp/cmp"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest/ptracetest"
"github.com/stretchr/testify/assert"
"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/ptrace"
Expand All @@ -48,10 +49,11 @@ func TestElasticTransactionEnrich(t *testing.T) {
return span
}
for _, tc := range []struct {
name string
input ptrace.Span
config config.ElasticTransactionConfig
enrichedAttrs map[string]any
name string
input ptrace.Span
config config.ElasticTransactionConfig
enrichedAttrs map[string]any
expectedSpanLinks *ptrace.SpanLinkSlice
}{
{
// test case gives a summary of what is emitted by default
Expand Down Expand Up @@ -319,20 +321,67 @@ func TestElasticTransactionEnrich(t *testing.T) {
AttributeTransactionType: "messaging",
},
},
{
name: "inferred_spans",
input: func() ptrace.Span {
span := getElasticTxn()
span.SetName("testtxn")
span.SetSpanID([8]byte{1})
normalLink := span.Links().AppendEmpty()
normalLink.SetSpanID([8]byte{2})

childLink := span.Links().AppendEmpty()
childLink.SetSpanID([8]byte{3})
childLink.Attributes().PutBool("is_child", true)

childLink2 := span.Links().AppendEmpty()
childLink2.SetSpanID([8]byte{4})
childLink2.Attributes().PutBool("elastic.is_child", true)
return span
}(),
config: config.Enabled().Transaction,
enrichedAttrs: map[string]any{
AttributeTimestampUs: startTs.AsTime().UnixMicro(),
AttributeTransactionSampled: true,
AttributeTransactionRoot: true,
AttributeTransactionID: "0100000000000000",
AttributeTransactionName: "testtxn",
AttributeProcessorEvent: "transaction",
AttributeTransactionRepresentativeCount: float64(1),
AttributeTransactionDurationUs: expectedDuration.Microseconds(),
AttributeEventOutcome: "success",
AttributeSuccessCount: int64(1),
AttributeTransactionResult: "Success",
AttributeTransactionType: "unknown",
AttributeChildIDs: []any{"0300000000000000", "0400000000000000"},
},
expectedSpanLinks: func() *ptrace.SpanLinkSlice {
spanLinks := ptrace.NewSpanLinkSlice()
// Only the span link without `is_child` or `elastic.is_child` is expected
spanLinks.AppendEmpty().SetSpanID([8]byte{2})
return &spanLinks
}(),
},
} {
t.Run(tc.name, func(t *testing.T) {
// Merge existing input attrs with the attrs added
// by enrichment to get the expected attributes.
expectedAttrs := tc.input.Attributes().AsRaw()
expectedSpan := ptrace.NewSpan()
tc.input.CopyTo(expectedSpan)

// Merge with the expected attributes and override the span links.
for k, v := range tc.enrichedAttrs {
expectedAttrs[k] = v
expectedSpan.Attributes().PutEmpty(k).FromRaw(v)
}
// Override span links
if tc.expectedSpanLinks != nil {
tc.expectedSpanLinks.CopyTo(expectedSpan.Links())
} else {
expectedSpan.Links().RemoveIf(func(_ ptrace.SpanLink) bool { return true })
Comment on lines +375 to +378
Copy link
Contributor Author

@lahsivjar lahsivjar Oct 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[For reviewers] This piece is a bit weird because CopyTo doesn't handle non initialized empty span links.

}

EnrichSpan(tc.input, config.Config{
Transaction: tc.config,
})

assert.Empty(t, cmp.Diff(expectedAttrs, tc.input.Attributes().AsRaw()))
assert.NoError(t, ptracetest.CompareSpan(expectedSpan, tc.input))
})
}
}
Expand All @@ -351,10 +400,11 @@ func TestElasticSpanEnrich(t *testing.T) {
return span
}
for _, tc := range []struct {
name string
input ptrace.Span
config config.ElasticSpanConfig
enrichedAttrs map[string]any
name string
input ptrace.Span
config config.ElasticSpanConfig
enrichedAttrs map[string]any
expectedSpanLinks *ptrace.SpanLinkSlice
}{
{
// test case gives a summary of what is emitted by default
Expand Down Expand Up @@ -827,20 +877,63 @@ func TestElasticSpanEnrich(t *testing.T) {
AttributeSpanDestinationServiceResource: "testsvc",
},
},
{
name: "inferred_spans",
input: func() ptrace.Span {
span := getElasticSpan()
span.SetName("testspan")
span.SetSpanID([8]byte{1})
normalLink := span.Links().AppendEmpty()
normalLink.SetSpanID([8]byte{2})

childLink := span.Links().AppendEmpty()
childLink.SetSpanID([8]byte{3})
childLink.Attributes().PutBool("is_child", true)

childLink2 := span.Links().AppendEmpty()
childLink2.SetSpanID([8]byte{4})
childLink2.Attributes().PutBool("elastic.is_child", true)
return span
}(),
config: config.Enabled().Span,
enrichedAttrs: map[string]any{
AttributeTimestampUs: startTs.AsTime().UnixMicro(),
AttributeSpanName: "testspan",
AttributeProcessorEvent: "span",
AttributeSpanRepresentativeCount: float64(1),
AttributeSpanType: "unknown",
AttributeSpanDurationUs: expectedDuration.Microseconds(),
AttributeEventOutcome: "success",
AttributeSuccessCount: int64(1),
AttributeChildIDs: []any{"0300000000000000", "0400000000000000"},
},
expectedSpanLinks: func() *ptrace.SpanLinkSlice {
spanLinks := ptrace.NewSpanLinkSlice()
// Only the span link without `is_child` or `elastic.is_child` is expected
spanLinks.AppendEmpty().SetSpanID([8]byte{2})
return &spanLinks
}(),
},
} {
t.Run(tc.name, func(t *testing.T) {
// Merge existing input attrs with the attrs added
// by enrichment to get the expected attributes.
expectedAttrs := tc.input.Attributes().AsRaw()
expectedSpan := ptrace.NewSpan()
tc.input.CopyTo(expectedSpan)

// Merge with the expected attributes and override the span links.
for k, v := range tc.enrichedAttrs {
expectedAttrs[k] = v
expectedSpan.Attributes().PutEmpty(k).FromRaw(v)
}
// Override span links
if tc.expectedSpanLinks != nil {
tc.expectedSpanLinks.CopyTo(expectedSpan.Links())
} else {
expectedSpan.Links().RemoveIf(func(_ ptrace.SpanLink) bool { return true })
}

EnrichSpan(tc.input, config.Config{
Span: tc.config,
})

assert.Empty(t, cmp.Diff(expectedAttrs, tc.input.Attributes().AsRaw()))
assert.NoError(t, ptracetest.CompareSpan(expectedSpan, tc.input))
})
}
}
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.22.0
require (
github.com/google/go-cmp v0.6.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden v0.112.0
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.112.0
github.com/stretchr/testify v1.9.0
go.opentelemetry.io/collector/pdata v1.18.0
go.opentelemetry.io/collector/semconv v0.112.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden v0.112.0 h1:+jb8oibBLgnEvhWiMomtxEf4bshEDwtnmKYTM8bf96U=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/golden v0.112.0/go.mod h1:G4KniRkewEl7JaT1EVTczTWi1nfYk2bD5GAn4aqBh4o=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.112.0 h1:WUAQfu+9bm7t86tyfqkcuz6vTCJfNAxMVocTZPLnWWs=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatatest v0.112.0/go.mod h1:dQCrspUDJRs7P6pXRALwj/yKIMzTYCvLa7XlzNycVFY=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.112.0 h1:FIQ/vt0Ulnwr2PSkLSD0SfdSyfm9dmBBnBcjAbngC7o=
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.112.0/go.mod h1:W9HkQWHB/Zc6adYHDG3FNyxfERt9eBAw2sBqNYBBBEE=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
Expand Down