tracer: support dot notation for tags with array values

[katiehockman]

What does this PR do?

Improves array support in tag values by automatically encoding arrays with dot notation.

With the following code snippet:

span := tracer.StartSpan("somename", tracer.Tag("custom", []string{"foo", "bar"}))

Before (just a string):

After (an array):

Note that this only supports one level of array nesting. Any more, and the user will need to do this manually.

AIT-8525

Motivation

This supports more precise search queries and filtering. For example, with the previous code, a DD_APM_REPLACE_TAG with custom would have removed the entire array. Now, a DD_APM_REPLACE_TAG with custom will do nothing. A customer will instead specify the index, e.g. a DD_APM_REPLACE_TAG with custom.1, to only redact the indices they want, or do custom.* to redact them all.

Reviewer's Checklist

• Changed code has unit tests for its functionality at or near 100% coverage.
• There is a benchmark for any new code, or changes to existing code.
• If this interacts with the agent in a new way, a system test has been added.

For Datadog employees:

• If this PR touches code that handles credentials of any kind, such as Datadog API keys, I've requested a review from @DataDog/security-design-and-guidance.
• This PR doesn't touch any of that.

Unsure? Have a question? Request a review!

[pr-commenter]

Benchmarks

Benchmark execution time: 2023-11-20 16:34:14

Comparing candidate commit d97386a in PR branch katie.hockman/otel-span-arrays with baseline commit 9cf50b3 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 39 metrics, 2 unstable metrics.

[github-actions]

This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[katiehockman]

Adding a comment to un-mark this as stale. I still want to do this, but I haven't had time to manually test it yet.

[katiehockman]

Because these changes will cause reflect to be called on every tag, there is a small performance cost if the code is reached. For example, if the code is above the Stringer check, then it adds a check every time a tag is added to see if it's a slice.

benchstat old.txt new.txt
goos: darwin
goarch: arm64
pkg: gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer
                  │   old.txt   │              new.txt               │
                  │   sec/op    │   sec/op     vs base               │
SetTagMetric-10     42.37n ± 2%   42.74n ± 3%       ~ (p=0.631 n=10)
SetTagString-10     39.99n ± 3%   39.81n ± 3%       ~ (p=0.579 n=10)
SetTagStringer-10   54.71n ± 2%   56.82n ± 2%  +3.87% (p=0.000 n=10)
SetTagField-10      18.61n ± 1%   18.55n ± 2%       ~ (p=0.361 n=10)
geomean             36.24n        36.60n       +0.97%

I'm not terribly concerned by 2 nanoseconds, but it could in theory add up if someone has many tags. I moved the check to be as low down the list of checks as possible so it's hopefully not hit in most cases (I expect the majority of tags are strings, ints, floats).

[katiehockman]

The parametric tests caught a bug in the way we were encoding the value, which should now be fixed. The system-tests PR is here: DataDog/system-tests#1841

[katiehockman]

And the failing parametric test in this PR is because it relies on old behavior. DataDog/system-tests#1841 will skip that test and enable the new one.