Match spans against the instrumentation library and resource attributes #928

zeitlinger · 2020-05-07T12:23:36Z

Add the possibility to match spans against the instrumentation library.

This is how it works:

// version match
//  expected actual  match
//  nil      <blank> yes
//  nil      1       yes
//  <blank>  <blank> yes
//  <blank>  1       no
//  1        <blank> no
//  1        1       yes

You can decide to match against all versions (expected = nil), a specific version (e.g. expected = 1), or against "version not provided" (expected = <blank>).

match spans on resource attributes

Use case: drop attribute values (e.g. a password) for a certain version (host.image.version)

allow regexp when matching attribute value if it is a string

Regular expressions should only be prohibited if the expected value is not a string (because an implicit conversion to string would be unexpected).
If the expected value is a string, then a regular expression should be allowed.

zeitlinger · 2020-05-07T12:24:50Z

@bogdandrutu this is the prototype that we discussed on gitter.

codecov-io · 2020-05-07T12:41:35Z

Codecov Report

Merging #928 into master will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #928      +/-   ##
==========================================
+ Coverage   91.35%   91.38%   +0.03%     
==========================================
  Files         280      281       +1     
  Lines       16661    16673      +12     
==========================================
+ Hits        15220    15236      +16     
+ Misses       1009     1007       -2     
+ Partials      432      430       -2

Impacted Files	Coverage Δ
...rnal/processor/filterset/strict/strictfilterset.go	`100.00% <ø> (ø)`
processor/metrics.go	`69.62% <ø> (-2.15%)`	⬇️
internal/processor/filterconfig/config.go	`100.00% <100.00%> (ø)`
...nternal/processor/filterhelper/attributematcher.go	`100.00% <100.00%> (ø)`
internal/processor/filterlog/filterlog.go	`90.90% <100.00%> (-1.40%)`	⬇️
internal/processor/filterspan/filterspan.go	`100.00% <100.00%> (+3.33%)`	⬆️
processor/attributesprocessor/attributes.go	`90.90% <100.00%> (-1.95%)`	⬇️
processor/spanprocessor/span.go	`87.95% <100.00%> (-0.82%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64ec3f3...b78b488. Read the comment docs.

bogdandrutu · 2020-05-07T14:29:58Z

internal/processor/filterspan/config.go

@@ -108,3 +110,16 @@ type Attribute struct {
 	// If it is not set, any value will match.
 	Value interface{} `mapstructure:"value"`
 }
+
+// version match


Move this comment to the Version argument?

internal/processor/filterspan/config.go

tigrannajaryan

Unless this is somehow urgent I would like to block this until open-telemetry/opentelemetry-proto#149 is resolved. If proposals in open-telemetry/opentelemetry-proto#149 are accepted we don't need this PR.

bogdandrutu · 2020-05-07T17:51:51Z

@tigrannajarayan the link is probably wrong

tigrannajaryan · 2020-05-07T17:56:25Z

@bogdandrutu sorry, link updated.

tigrannajaryan · 2020-06-29T16:28:08Z

open-telemetry/opentelemetry-proto#149 is closed, the decision is to keep InstrumentationLibrary.

This PR can now move forward, nothing blocks it.

@zeitlinger can you please rebase and fix the conflicts?

zeitlinger · 2020-06-30T07:39:22Z

@tigrannajaryan @bogdandrutu

I see that the matching logic has changed - and this is an opportunity to revisit the design:

If we combine "library" and "version" into one string (concatenated by : as done e.g. by gradle), then the config would look like this:

    include:
      match_type: regexp
      libraries: ["bar", "foo:.*"]

This would match a library bar without a version or a library foo with any version.

I think it's also easier to understand than the previous version match matrix:

// version match
//  expected actual  match
//  nil      <blank> yes
//  nil      1       yes
//  <blank>  <blank> yes
//  <blank>  1       no
//  1        <blank> no
//  1        1       yes

Last, but not least, this would simplify the implementation, because it would be easy to use filterset.FilterSet

WDYT?

zeitlinger · 2020-07-28T12:34:27Z

@tigrannajaryan @bogdandrutu

I see that the matching logic has changed - and this is an opportunity to revisit the design:

If we combine "library" and "version" into one string (concatenated by : as done e.g. by gradle), then the config would look like this:
    include:
      match_type: regexp
      libraries: ["bar", "foo:.*"]
This would match a library bar without a version or a library foo with any version.

I think it's also easier to understand than the previous version match matrix:
// version match
//  expected actual  match
//  nil      <blank> yes
//  nil      1       yes
//  <blank>  <blank> yes
//  <blank>  1       no
//  1        <blank> no
//  1        1       yes
Last, but not least, this would simplify the implementation, because it would be easy to use filterset.FilterSet

WDYT?

I found a way to avoid this and have and use the more natural

libraries: 
  - name: n
    version: v

bogdandrutu · 2020-08-26T21:38:35Z

@zeitlinger looks like we are ready to merge this, so can you rebase and close all comments :)

zeitlinger · 2020-08-27T11:29:02Z

@zeitlinger looks like we are ready to merge this, so can you rebase and close all comments :)

@bogdandrutu done - note that this PR is based on #929

internal/processor/filterspan/config.go

bogdandrutu · 2020-09-11T20:55:40Z

internal/processor/filterspan/config.go

@@ -100,3 +104,16 @@ type Attribute struct {
 	// If it is not set, any value will match.
 	Value interface{} `mapstructure:"value"`
 }
+
+type InstrumentationLibrary struct {


Comment if exported. Does this need to be exported?

actually not - neither does Attribute. Why did you export Attribute?
It's kind of a public interface, as it has the json structure.

zeitlinger · 2020-09-15T16:58:23Z

@bogdandrutu I've tried to restart contrib-test - but it's still failing. Looks like a flaky test, but I wasn't able to figure out what it is.

zeitlinger · 2020-10-06T17:17:34Z

Please also make similar changes for Logs.

I extracted the attribute matching logic - anything else?

tigrannajaryan · 2020-10-06T18:46:02Z

internal/processor/filterconfig/config.go

@@ -71,7 +71,7 @@ type MatchProperties struct {
 	// Config configures the matching patterns used when matching span properties.
 	filterset.Config `mapstructure:",squash"`

-	// Note: For spans, one of Services, SpanNames or Attributes must be specified with a
+	// Note: For spans, one of Services, SpanNames, Attributes, Resources or Libraries must be specified with a
 	// non-empty value for a valid configuration.

 	// For logs, one of LogNames or Attributes must be specified with a


Update comment.

what exactly?

Suggested change

// For logs, one of LogNames or Attributes must be specified with a

// For logs, one of LogNames, Attributes, Resources or Libraries must be specified with a

Don't we now support Resource and Libraries for logs too?

tigrannajaryan · 2020-10-06T18:50:13Z

Mostly LGTM, just one minor comment left.

Normally we ask one PR to add one distinct piece of functionality. This PR adds filtering by resources and by library. Ideally it would be 2 separate PRs. I'll merge the PR as-is to avoid more work to split the PR. For the future please follow one PR - one feature principle.

tigrannajaryan · 2020-10-06T18:50:29Z

The build failed, make sure it passes.

zeitlinger · 2020-10-06T18:54:25Z

Ideally it would be 2 separate PRs.

it was 2 PRs initially, but it considerable effort to update both PRs with upstream

match spans on instrumentation library allow regexp on attribute value if it is a string

zeitlinger · 2020-10-07T06:42:56Z

@tigrannajaryan contrib-test seems to be flaky - other tests are working now

tigrannajaryan

LGTM, except one last comment. Can merge after that.

tigrannajaryan · 2020-10-08T00:07:46Z

internal/processor/filterconfig/config.go

@@ -71,7 +71,7 @@ type MatchProperties struct {
 	// Config configures the matching patterns used when matching span properties.
 	filterset.Config `mapstructure:",squash"`

-	// Note: For spans, one of Services, SpanNames or Attributes must be specified with a
+	// Note: For spans, one of Services, SpanNames, Attributes, Resources or Libraries must be specified with a
 	// non-empty value for a valid configuration.

 	// For logs, one of LogNames or Attributes must be specified with a


Suggested change

// For logs, one of LogNames or Attributes must be specified with a

// For logs, one of LogNames, Attributes, Resources or Libraries must be specified with a

Don't we now support Resource and Libraries for logs too?

zeitlinger · 2020-10-08T06:10:29Z

LGTM, except one last comment. Can merge after that.

I double checked logs and we can and should support resources and libraries as well.

I didn't notice this because log filtering is not fully implemented - but the #1830 seems to do it. Once that is merged, I can also include support for resources and libraries.

match logs on instrumentation library

zeitlinger · 2020-10-08T08:25:51Z

I didn't notice this because log filtering is not fully implemented - but the #1830 seems to do it. Once that is merged, I can also include support for resources and libraries.

Actually it was no problem to implement support - the only intersection with #1830 is a signature change to pass resource and library (which are also available on ResourceLogs

match logs on instrumentation library

zeitlinger · 2020-10-08T14:37:28Z

@tigrannajaryan please review again

tigrannajaryan · 2020-10-08T22:15:30Z

Thank you @zeitlinger

zeitlinger · 2020-10-09T07:41:11Z

@tigrannajaryan thanks for helping out!

zeitlinger force-pushed the extended-matcher branch from 7397254 to b59de38 Compare May 7, 2020 12:35

This was referenced May 7, 2020

match spans on resource attributes #929

Closed

Rewritable Tracer open-telemetry/opentelemetry-java-instrumentation#309

Closed

bogdandrutu reviewed May 7, 2020

View reviewed changes

tigrannajaryan requested changes May 7, 2020

View reviewed changes

tigrannajaryan mentioned this pull request May 7, 2020

Clarify and potentially remove InstrumentationLibrarySpans open-telemetry/opentelemetry-proto#149

Closed

zeitlinger force-pushed the extended-matcher branch from b59de38 to 5da1db8 Compare July 28, 2020 12:33

zeitlinger force-pushed the extended-matcher branch 2 times, most recently from bdf7a77 to 81d8a12 Compare July 28, 2020 13:30

tigrannajaryan assigned bogdandrutu Aug 26, 2020

zeitlinger force-pushed the extended-matcher branch from 81d8a12 to a2aa818 Compare August 27, 2020 11:27

zeitlinger marked this pull request as ready for review September 9, 2020 19:59

zeitlinger requested review from dmitryax, james-bebbington, nilebox, owais and pjanotti as code owners September 9, 2020 19:59

bogdandrutu reviewed Sep 11, 2020

View reviewed changes

zeitlinger force-pushed the extended-matcher branch from 3f865ac to 800c7ee Compare September 15, 2020 12:22

tigrannajaryan self-assigned this Oct 6, 2020

zeitlinger changed the title ~~match spans against the instrumentation library~~ match spans against the instrumentation library and resource attributes Oct 6, 2020

zeitlinger force-pushed the extended-matcher branch from 51c784e to 54454ff Compare October 6, 2020 17:16

tigrannajaryan changed the title ~~match spans against the instrumentation library and resource attributes~~ Match spans against the instrumentation library and resource attributes Oct 6, 2020

tigrannajaryan reviewed Oct 6, 2020

View reviewed changes

zeitlinger force-pushed the extended-matcher branch from 54454ff to f7f0d08 Compare October 6, 2020 19:00

match spans on resource attributes

711fa19

match spans on instrumentation library allow regexp on attribute value if it is a string

zeitlinger force-pushed the extended-matcher branch from f7f0d08 to 711fa19 Compare October 7, 2020 06:28

tigrannajaryan reviewed Oct 8, 2020

View reviewed changes

match logs on resource attributes

45b512b

match logs on instrumentation library

zeitlinger added 2 commits October 8, 2020 12:24

match logs on resource attributes

23d2265

match logs on instrumentation library

match logs on resource attributes

b78b488

match logs on instrumentation library

tigrannajaryan approved these changes Oct 8, 2020

View reviewed changes

tigrannajaryan merged commit 393e98f into open-telemetry:master Oct 8, 2020

zeitlinger deleted the extended-matcher branch October 9, 2020 07:36

zeitlinger mentioned this pull request Oct 9, 2020

Log Support in Attribute Processor (2/2) #1830

Closed

hughesjj pushed a commit to hughesjj/opentelemetry-collector that referenced this pull request Apr 27, 2023

Remove a duplicate line in docs/troubleshooting.md (open-telemetry#928)

ce4e310

crobert-1 mentioned this pull request Feb 29, 2024

Clarify attributes also use regex open-telemetry/opentelemetry-collector-contrib#31496

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Match spans against the instrumentation library and resource attributes #928

Match spans against the instrumentation library and resource attributes #928

zeitlinger commented May 7, 2020 •

edited

Loading

zeitlinger commented May 7, 2020

codecov-io commented May 7, 2020 •

edited by codecov bot

Loading

bogdandrutu May 7, 2020

tigrannajaryan left a comment •

edited

Loading

bogdandrutu commented May 7, 2020

tigrannajaryan commented May 7, 2020

tigrannajaryan commented Jun 29, 2020

zeitlinger commented Jun 30, 2020

zeitlinger commented Jul 28, 2020

bogdandrutu commented Aug 26, 2020

zeitlinger commented Aug 27, 2020

bogdandrutu Sep 11, 2020

zeitlinger Sep 15, 2020

zeitlinger commented Sep 15, 2020

zeitlinger commented Oct 6, 2020

tigrannajaryan Oct 6, 2020

zeitlinger Oct 6, 2020

tigrannajaryan Oct 8, 2020

tigrannajaryan commented Oct 6, 2020

tigrannajaryan commented Oct 6, 2020

zeitlinger commented Oct 6, 2020

zeitlinger commented Oct 7, 2020

tigrannajaryan left a comment

tigrannajaryan Oct 8, 2020

zeitlinger commented Oct 8, 2020

zeitlinger commented Oct 8, 2020

zeitlinger commented Oct 8, 2020

tigrannajaryan commented Oct 8, 2020

zeitlinger commented Oct 9, 2020

	// For logs, one of LogNames or Attributes must be specified with a
	// For logs, one of LogNames, Attributes, Resources or Libraries must be specified with a

Match spans against the instrumentation library and resource attributes #928

Match spans against the instrumentation library and resource attributes #928

Conversation

zeitlinger commented May 7, 2020 • edited Loading

Add the possibility to match spans against the instrumentation library.

match spans on resource attributes

allow regexp when matching attribute value if it is a string

zeitlinger commented May 7, 2020

codecov-io commented May 7, 2020 • edited by codecov bot Loading

Codecov Report

bogdandrutu May 7, 2020

Choose a reason for hiding this comment

tigrannajaryan left a comment • edited Loading

Choose a reason for hiding this comment

bogdandrutu commented May 7, 2020

tigrannajaryan commented May 7, 2020

tigrannajaryan commented Jun 29, 2020

zeitlinger commented Jun 30, 2020

zeitlinger commented Jul 28, 2020

bogdandrutu commented Aug 26, 2020

zeitlinger commented Aug 27, 2020

bogdandrutu Sep 11, 2020

Choose a reason for hiding this comment

zeitlinger Sep 15, 2020

Choose a reason for hiding this comment

zeitlinger commented Sep 15, 2020

zeitlinger commented Oct 6, 2020

tigrannajaryan Oct 6, 2020

Choose a reason for hiding this comment

zeitlinger Oct 6, 2020

Choose a reason for hiding this comment

tigrannajaryan Oct 8, 2020

Choose a reason for hiding this comment

tigrannajaryan commented Oct 6, 2020

tigrannajaryan commented Oct 6, 2020

zeitlinger commented Oct 6, 2020

zeitlinger commented Oct 7, 2020

tigrannajaryan left a comment

Choose a reason for hiding this comment

tigrannajaryan Oct 8, 2020

Choose a reason for hiding this comment

zeitlinger commented Oct 8, 2020

zeitlinger commented Oct 8, 2020

zeitlinger commented Oct 8, 2020

tigrannajaryan commented Oct 8, 2020

zeitlinger commented Oct 9, 2020

zeitlinger commented May 7, 2020 •

edited

Loading

codecov-io commented May 7, 2020 •

edited by codecov bot

Loading

tigrannajaryan left a comment •

edited

Loading