New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement extra_services for remote config #6130
Implement extra_services for remote config #6130
Conversation
3d60edb
to
5948be8
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.31.0-SNAPSHOT~8b357abce7, baseline=1.31.0-SNAPSHOT~142c8cac77
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059423
Total [baseline] (9.367 s) : 0, 9367301
Agent [candidate] (1.065 s) : 0, 1065307
Total [candidate] (9.345 s) : 0, 9345382
section appsec
Agent [baseline] (1.168 s) : 0, 1168037
Total [baseline] (9.495 s) : 0, 9495162
Agent [candidate] (1.159 s) : 0, 1158716
Total [candidate] (9.451 s) : 0, 9450511
section iast
Agent [baseline] (1.194 s) : 0, 1194162
Total [baseline] (9.768 s) : 0, 9767988
Agent [candidate] (1.188 s) : 0, 1187828
Total [candidate] (9.722 s) : 0, 9721950
section profiling
Agent [baseline] (1.288 s) : 0, 1287750
Total [baseline] (9.611 s) : 0, 9610678
Agent [candidate] (1.28 s) : 0, 1280433
Total [candidate] (9.614 s) : 0, 9613623
gantt
title petclinic - break down per module: candidate=1.31.0-SNAPSHOT~8b357abce7, baseline=1.31.0-SNAPSHOT~142c8cac77
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (666.934 ms) : 0, 666934
BytebuddyAgent [candidate] (670.695 ms) : 0, 670695
GlobalTracer [baseline] (298.087 ms) : 0, 298087
GlobalTracer [candidate] (300.384 ms) : 0, 300384
AppSec [baseline] (51.918 ms) : 0, 51918
AppSec [candidate] (51.613 ms) : 0, 51613
Remote Config [baseline] (704.905 µs) : 0, 705
Remote Config [candidate] (709.597 µs) : 0, 710
Telemetry [baseline] (7.608 ms) : 0, 7608
Telemetry [candidate] (7.636 ms) : 0, 7636
section appsec
BytebuddyAgent [baseline] (673.171 ms) : 0, 673171
BytebuddyAgent [candidate] (667.76 ms) : 0, 667760
GlobalTracer [baseline] (301.039 ms) : 0, 301039
GlobalTracer [candidate] (298.542 ms) : 0, 298542
AppSec [baseline] (151.649 ms) : 0, 151649
AppSec [candidate] (150.746 ms) : 0, 150746
Remote Config [baseline] (643.434 µs) : 0, 643
Remote Config [candidate] (634.561 µs) : 0, 635
Telemetry [baseline] (6.901 ms) : 0, 6901
Telemetry [candidate] (6.817 ms) : 0, 6817
section iast
BytebuddyAgent [baseline] (785.538 ms) : 0, 785538
BytebuddyAgent [candidate] (780.61 ms) : 0, 780610
GlobalTracer [baseline] (292.124 ms) : 0, 292124
GlobalTracer [candidate] (290.453 ms) : 0, 290453
AppSec [baseline] (53.084 ms) : 0, 53084
AppSec [candidate] (51.932 ms) : 0, 51932
Remote Config [baseline] (672.422 µs) : 0, 672
Remote Config [candidate] (706.352 µs) : 0, 706
Telemetry [baseline] (6.646 ms) : 0, 6646
Telemetry [candidate] (6.602 ms) : 0, 6602
IAST [baseline] (21.442 ms) : 0, 21442
IAST [candidate] (23.176 ms) : 0, 23176
section profiling
ProfilingAgent [baseline] (112.417 ms) : 0, 112417
ProfilingAgent [candidate] (111.998 ms) : 0, 111998
BytebuddyAgent [baseline] (670.222 ms) : 0, 670222
BytebuddyAgent [candidate] (667.153 ms) : 0, 667153
GlobalTracer [baseline] (385.299 ms) : 0, 385299
GlobalTracer [candidate] (383.029 ms) : 0, 383029
AppSec [baseline] (52.508 ms) : 0, 52508
AppSec [candidate] (51.981 ms) : 0, 51981
Remote Config [baseline] (673.388 µs) : 0, 673
Remote Config [candidate] (662.257 µs) : 0, 662
Telemetry [baseline] (11.638 ms) : 0, 11638
Telemetry [candidate] (10.862 ms) : 0, 10862
Profiling [baseline] (112.444 ms) : 0, 112444
Profiling [candidate] (112.022 ms) : 0, 112022
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~8b357abce7, baseline=1.31.0-SNAPSHOT~142c8cac77
dateFormat X
axisFormat %s
section baseline
no_agent (1.34 ms) : 1321, 1359
. : milestone, 1340,
appsec (1.768 ms) : 1743, 1794
. : milestone, 1768,
iast (1.525 ms) : 1501, 1549
. : milestone, 1525,
profiling (1.587 ms) : 1560, 1613
. : milestone, 1587,
tracing (1.496 ms) : 1471, 1522
. : milestone, 1496,
section candidate
no_agent (1.357 ms) : 1338, 1376
. : milestone, 1357,
appsec (1.766 ms) : 1740, 1791
. : milestone, 1766,
iast (1.509 ms) : 1484, 1533
. : milestone, 1509,
profiling (1.522 ms) : 1496, 1549
. : milestone, 1522,
tracing (1.499 ms) : 1474, 1523
. : milestone, 1499,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~8b357abce7, baseline=1.31.0-SNAPSHOT~142c8cac77
dateFormat X
axisFormat %s
section baseline
no_agent (374.392 µs) : 352, 396
. : milestone, 374,
iast (460.369 µs) : 440, 481
. : milestone, 460,
iast_FULL (531.682 µs) : 511, 552
. : milestone, 532,
iast_GLOBAL (484.559 µs) : 463, 506
. : milestone, 485,
iast_HARDCODED_SECRET_DISABLED (472.252 µs) : 451, 493
. : milestone, 472,
iast_INACTIVE (436.434 µs) : 416, 457
. : milestone, 436,
iast_TELEMETRY_OFF (468.934 µs) : 448, 490
. : milestone, 469,
tracing (433.637 µs) : 413, 455
. : milestone, 434,
section candidate
no_agent (370.33 µs) : 348, 393
. : milestone, 370,
iast (471.344 µs) : 450, 493
. : milestone, 471,
iast_FULL (526.951 µs) : 506, 548
. : milestone, 527,
iast_GLOBAL (497.862 µs) : 477, 519
. : milestone, 498,
iast_HARDCODED_SECRET_DISABLED (471.663 µs) : 451, 492
. : milestone, 472,
iast_INACTIVE (448.813 µs) : 427, 470
. : milestone, 449,
iast_TELEMETRY_OFF (461.71 µs) : 441, 483
. : milestone, 462,
tracing (439.076 µs) : 418, 460
. : milestone, 439,
|
remote-config/src/main/java/datadog/remoteconfig/state/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
631d8e6
to
e36bd71
Compare
remote-config/src/main/java/datadog/remoteconfig/tuf/RemoteConfigRequest.java
Outdated
Show resolved
Hide resolved
remote-config/src/main/java/datadog/remoteconfig/state/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
967f107
to
b9096d8
Compare
b9096d8
to
0912fbf
Compare
0912fbf
to
400c162
Compare
remote-config/src/main/java/datadog/remoteconfig/state/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
remote-config/src/main/java/datadog/remoteconfig/state/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
732bc97
to
ee64e26
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this happens on every single call to finish a span this has the potential to noticeably affect performance. Have we done any performance measurements with real-world applications to gauge the impact?
Rather than compare the span's service name against the local service name on finish
(which is an extremely hot code path) is there a way we could collect this information as the naming schema(s) are accessed? There are potentially ways to do that so we only add the extra service name to the bucket once, the first time the naming schema is accessed.
695eebc
to
0ce4241
Compare
internal-api/src/main/java/datadog/trace/util/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's OK to me. You also should need to add the same on the TagInterceptor when the service name is inferred from the servlet context (the case of multi-services for webapps) or for the service name tag (will address service name set by manual instrumentation)
internal-api/src/main/java/datadog/trace/util/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
...l-api/src/test/groovy/datadog/trace/api/naming/ExtraServiceProviderNamingV0ForkedTest.groovy
Outdated
Show resolved
Hide resolved
…vider.java Co-authored-by: Andrea Marziali <andrea.marziali@datadoghq.com>
internal-api/src/main/java/datadog/trace/util/ExtraServicesProvider.java
Outdated
Show resolved
Hide resolved
@@ -1773,6 +1776,8 @@ PROFILING_DATADOG_PROFILER_ENABLED, isDatadogProfilerSafeInCurrentEnvironment()) | |||
remoteConfigTargetsKey = | |||
configProvider.getString(REMOTE_CONFIG_TARGETS_KEY, DEFAULT_REMOTE_CONFIG_TARGETS_KEY); | |||
|
|||
remoteConfigMaxExtraServices = configProvider.getInteger(REMOTE_CONFIG_MAX_EXTRA_SERVICES, 64); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be nice to extract a constant with the default value.
if (extraServices.size() >= MAX_EXTRA_SERVICE) { | ||
if (!limitReachedLogged) { | ||
log.debug( | ||
"extra service limit({}) reached: service {} can't be added", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could add the SEND_TELEMETRY
marker, so we get notified if this situation is common.
@@ -0,0 +1,61 @@ | |||
package datadog.trace.util; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not technically a utility class, perhaps datadog.trace.api.telemetry
would be a better package since it already contains similar collectors? Or datadog.trace.api.remoteconfig
?
import org.slf4j.Logger; | ||
import org.slf4j.LoggerFactory; | ||
|
||
public class ExtraServicesProvider { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest renaming this to ServiceCollector
or ExtraServiceCollector
because that's what it does - it is used to populate the extra services field in RC, but it could also be used elsewhere. (Generally I prefer naming classes based on what they do.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion!
// singleton | ||
} | ||
|
||
public void maybeAddExtraService(final String serviceName) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shorten to addService
or recordService
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just minor suggestions around naming
What Does This Do Add extraService field to RemoteConfigRequest with all the different service names in the spans New ExtraServiceProvider to manage extra services that satisfies the next conditions: Limited to 64 elements New service name can be added if its different from the global one, is not null and is not in the current list When comparing service names, it should be don in a case-insensitive manner if the list is empty it should return null It has been decided to attempt adding a new extra service when naming-schema is invoked to get a service name Motivation Knowing all service names used by a tracer instances help the UI give better feedback to the user. The agent version v7.46.0 contains an new field "extra_services" in the remote config client, that allows clients to list any additional services that are used within tracer spans.
What Does This Do
Add extraService field to RemoteConfigRequest with all the different service names in the spans
New ExtraServiceProvider to manage extra services that satisfies the next conditions:
It has been decided to attempt adding a new extra service when the span will be finished. This approach is used by other languages like dotNet. Any suggestions from @DataDog/apm-java ?
Motivation
Knowing all service names used by a tracer instances help the UI give better feedback to the user.
The agent version v7.46.0 contains an new field "extra_services" in the remote config client, that allows clients to list any additional services that are used within tracer spans.
Additional Notes
Jira ticket: APPSEC-9907
.NET PR: DataDog/dd-trace-dotnet#4419