Add x-datadog-trace-id header to ITR backend requests #6189

nikita-tkachenko-datadog · 2023-11-10T10:22:55Z

What Does This Do

Adds x-datadog-trace-id header to HTTP requests sent to CI Visibility endpoints.

Motivation

The header is needed to simplify debugging, so that different requests made within a single test session could be correlated.

Additional Notes

x-datadog-parent-id header is added with the same value as trace ID intentionally.

Jira ticket: CIVIS-7973

pr-commenter · 2023-11-10T11:04:57Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~3fb888b81b	1.24.0-SNAPSHOT~ec34ac76cd
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1036378
Total [baseline] (8.803 s) : 0, 8802687
Agent [candidate] (1.034 s) : 0, 1034171
Total [candidate] (8.781 s) : 0, 8780930
section iast
Agent [baseline] (1.149 s) : 0, 1149443
Total [baseline] (9.31 s) : 0, 9309612
Agent [candidate] (1.161 s) : 0, 1160590
Total [candidate] (9.358 s) : 0, 9358106
section iast_TELEMETRY_OFF
Agent [baseline] (1.144 s) : 0, 1144229
Total [baseline] (9.284 s) : 0, 9283924
Agent [candidate] (1.147 s) : 0, 1146900
Total [candidate] (9.342 s) : 0, 9341684

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	iast	1.149 s	113.065 ms (10.9%)
Agent	iast_TELEMETRY_OFF	1.144 s	107.85 ms (10.4%)
Total	tracing	8.803 s	-
Total	iast	9.31 s	506.926 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.284 s	481.238 ms (5.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.034 s	-
Agent	iast	1.161 s	126.419 ms (12.2%)
Agent	iast_TELEMETRY_OFF	1.147 s	112.729 ms (10.9%)
Total	tracing	8.781 s	-
Total	iast	9.358 s	577.176 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.342 s	560.753 ms (6.4%)

gantt
    title insecure-bank - break down per module: candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.578 ms) : 0, 646578
BytebuddyAgent [candidate] (645.05 ms) : 0, 645050
GlobalTracer [baseline] (294.272 ms) : 0, 294272
GlobalTracer [candidate] (294.101 ms) : 0, 294101
AppSec [baseline] (48.895 ms) : 0, 48895
AppSec [candidate] (48.59 ms) : 0, 48590
Remote Config [baseline] (692.69 µs) : 0, 693
Remote Config [candidate] (694.115 µs) : 0, 694
Telemetry [baseline] (11.423 ms) : 0, 11423
Telemetry [candidate] (11.288 ms) : 0, 11288
section iast
BytebuddyAgent [baseline] (764.157 ms) : 0, 764157
BytebuddyAgent [candidate] (771.913 ms) : 0, 771913
GlobalTracer [baseline] (273.652 ms) : 0, 273652
GlobalTracer [candidate] (276.682 ms) : 0, 276682
AppSec [baseline] (46.679 ms) : 0, 46679
AppSec [candidate] (46.942 ms) : 0, 46942
Remote Config [baseline] (575.251 µs) : 0, 575
Remote Config [candidate] (583.262 µs) : 0, 583
Telemetry [baseline] (12.65 ms) : 0, 12650
Telemetry [candidate] (12.771 ms) : 0, 12771
IAST [baseline] (17.387 ms) : 0, 17387
IAST [candidate] (16.982 ms) : 0, 16982
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (758.185 ms) : 0, 758185
BytebuddyAgent [candidate] (759.798 ms) : 0, 759798
GlobalTracer [baseline] (274.058 ms) : 0, 274058
GlobalTracer [candidate] (275.211 ms) : 0, 275211
AppSec [baseline] (46.622 ms) : 0, 46622
AppSec [candidate] (46.555 ms) : 0, 46555
Remote Config [baseline] (557.654 µs) : 0, 558
Remote Config [candidate] (566.181 µs) : 0, 566
Telemetry [baseline] (13.038 ms) : 0, 13038
Telemetry [candidate] (14.738 ms) : 0, 14738
IAST [baseline] (17.255 ms) : 0, 17255
IAST [candidate] (15.651 ms) : 0, 15651

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1033995
Total [baseline] (9.315 s) : 0, 9314767
Agent [candidate] (1.037 s) : 0, 1036595
Total [candidate] (9.28 s) : 0, 9280209
section appsec
Agent [baseline] (1.119 s) : 0, 1119174
Total [baseline] (9.413 s) : 0, 9413150
Agent [candidate] (1.122 s) : 0, 1122248
Total [candidate] (9.443 s) : 0, 9443277
section iast
Agent [baseline] (1.148 s) : 0, 1148406
Total [baseline] (9.461 s) : 0, 9460690
Agent [candidate] (1.154 s) : 0, 1154326
Total [candidate] (9.477 s) : 0, 9476866
section profiling
Agent [baseline] (1.227 s) : 0, 1226883
Total [baseline] (9.599 s) : 0, 9598844
Agent [candidate] (1.219 s) : 0, 1219216
Total [candidate] (9.565 s) : 0, 9564595

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.034 s	-
Agent	appsec	1.119 s	85.178 ms (8.2%)
Agent	iast	1.148 s	114.41 ms (11.1%)
Agent	profiling	1.227 s	192.887 ms (18.7%)
Total	tracing	9.315 s	-
Total	appsec	9.413 s	98.383 ms (1.1%)
Total	iast	9.461 s	145.923 ms (1.6%)
Total	profiling	9.599 s	284.077 ms (3.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.037 s	-
Agent	appsec	1.122 s	85.653 ms (8.3%)
Agent	iast	1.154 s	117.731 ms (11.4%)
Agent	profiling	1.219 s	182.621 ms (17.6%)
Total	tracing	9.28 s	-
Total	appsec	9.443 s	163.068 ms (1.8%)
Total	iast	9.477 s	196.657 ms (2.1%)
Total	profiling	9.565 s	284.385 ms (3.1%)

gantt
    title petclinic - break down per module: candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.132 ms) : 0, 645132
BytebuddyAgent [candidate] (646.047 ms) : 0, 646047
GlobalTracer [baseline] (293.846 ms) : 0, 293846
GlobalTracer [candidate] (295.234 ms) : 0, 295234
AppSec [baseline] (48.702 ms) : 0, 48702
AppSec [candidate] (48.877 ms) : 0, 48877
Remote Config [baseline] (689.172 µs) : 0, 689
Remote Config [candidate] (697.48 µs) : 0, 697
Telemetry [baseline] (11.294 ms) : 0, 11294
Telemetry [candidate] (11.333 ms) : 0, 11333
section appsec
BytebuddyAgent [baseline] (645.066 ms) : 0, 645066
BytebuddyAgent [candidate] (646.315 ms) : 0, 646315
GlobalTracer [baseline] (293.136 ms) : 0, 293136
GlobalTracer [candidate] (294.769 ms) : 0, 294769
AppSec [baseline] (139.18 ms) : 0, 139180
AppSec [candidate] (139.347 ms) : 0, 139347
Remote Config [baseline] (642.615 µs) : 0, 643
Remote Config [candidate] (639.798 µs) : 0, 640
Telemetry [baseline] (6.844 ms) : 0, 6844
Telemetry [candidate] (6.877 ms) : 0, 6877
section iast
BytebuddyAgent [baseline] (764.906 ms) : 0, 764906
BytebuddyAgent [candidate] (767.067 ms) : 0, 767067
GlobalTracer [baseline] (273.491 ms) : 0, 273491
GlobalTracer [candidate] (274.777 ms) : 0, 274777
AppSec [baseline] (46.453 ms) : 0, 46453
AppSec [candidate] (46.653 ms) : 0, 46653
Remote Config [baseline] (566.539 µs) : 0, 567
Remote Config [candidate] (589.163 µs) : 0, 589
Telemetry [baseline] (11.105 ms) : 0, 11105
Telemetry [candidate] (12.476 ms) : 0, 12476
IAST [baseline] (17.432 ms) : 0, 17432
IAST [candidate] (18.211 ms) : 0, 18211
section profiling
BytebuddyAgent [baseline] (661.858 ms) : 0, 661858
BytebuddyAgent [candidate] (656.975 ms) : 0, 656975
GlobalTracer [baseline] (360.279 ms) : 0, 360279
GlobalTracer [candidate] (358.201 ms) : 0, 358201
AppSec [baseline] (49.222 ms) : 0, 49222
AppSec [candidate] (49.017 ms) : 0, 49017
Remote Config [baseline] (666.065 µs) : 0, 666
Remote Config [candidate] (646.824 µs) : 0, 647
Telemetry [baseline] (11.399 ms) : 0, 11399
Telemetry [candidate] (11.347 ms) : 0, 11347
ProfilingAgent [baseline] (88.751 ms) : 0, 88751
ProfilingAgent [candidate] (88.715 ms) : 0, 88715
Profiling [baseline] (88.775 ms) : 0, 88775
Profiling [candidate] (88.74 ms) : 0, 88740

Load

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~3fb888b81b	1.24.0-SNAPSHOT~ec34ac76cd
config	baseline	candidate
end_time	2023-11-13T12:16:54	2023-11-13T12:33:22
start_time	2023-11-13T12:16:42	2023-11-13T12:33:10

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.132 µs) : 345, 385
.   : milestone, 365,
iast (457.151 µs) : 436, 478
.   : milestone, 457,
iast_FULL (523.458 µs) : 503, 544
.   : milestone, 523,
iast_INACTIVE (441.152 µs) : 420, 463
.   : milestone, 441,
iast_TELEMETRY_OFF (454.624 µs) : 434, 476
.   : milestone, 455,
tracing (441.093 µs) : 420, 462
.   : milestone, 441,
section candidate
no_agent (359.45 µs) : 340, 379
.   : milestone, 359,
iast (461.093 µs) : 440, 482
.   : milestone, 461,
iast_FULL (521.186 µs) : 500, 542
.   : milestone, 521,
iast_INACTIVE (433.268 µs) : 413, 454
.   : milestone, 433,
iast_TELEMETRY_OFF (456.073 µs) : 435, 477
.   : milestone, 456,
tracing (435.196 µs) : 414, 457
.   : milestone, 435,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.132 µs [345.05 µs, 385.214 µs]	-
iast	457.151 µs [436.374 µs, 477.928 µs]	92.019 µs (25.2%)
iast_FULL	523.458 µs [502.936 µs, 543.981 µs]	158.326 µs (43.4%)
iast_INACTIVE	441.152 µs [419.746 µs, 462.558 µs]	76.02 µs (20.8%)
iast_TELEMETRY_OFF	454.624 µs [433.616 µs, 475.633 µs]	89.492 µs (24.5%)
tracing	441.093 µs [419.689 µs, 462.497 µs]	75.961 µs (20.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	359.45 µs [339.524 µs, 379.376 µs]	-
iast	461.093 µs [439.905 µs, 482.28 µs]	101.643 µs (28.3%)
iast_FULL	521.186 µs [500.305 µs, 542.067 µs]	161.736 µs (45.0%)
iast_INACTIVE	433.268 µs [412.681 µs, 453.855 µs]	73.819 µs (20.5%)
iast_TELEMETRY_OFF	456.073 µs [434.792 µs, 477.353 µs]	96.623 µs (26.9%)
tracing	435.196 µs [413.84 µs, 456.551 µs]	75.746 µs (21.1%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~ec34ac76cd, baseline=1.24.0-SNAPSHOT~3fb888b81b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1333, 1371
.   : milestone, 1352,
appsec (1.684 ms) : 1659, 1708
.   : milestone, 1684,
iast (1.473 ms) : 1450, 1497
.   : milestone, 1473,
profiling (1.488 ms) : 1461, 1514
.   : milestone, 1488,
tracing (1.456 ms) : 1431, 1481
.   : milestone, 1456,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.712 ms) : 1687, 1738
.   : milestone, 1712,
iast (1.484 ms) : 1458, 1509
.   : milestone, 1484,
profiling (1.451 ms) : 1426, 1476
.   : milestone, 1451,
tracing (1.467 ms) : 1442, 1492
.   : milestone, 1467,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.352 ms [1.333 ms, 1.371 ms]	-
appsec	1.684 ms [1.659 ms, 1.708 ms]	331.766 µs (24.5%)
iast	1.473 ms [1.45 ms, 1.497 ms]	121.484 µs (9.0%)
profiling	1.488 ms [1.461 ms, 1.514 ms]	135.687 µs (10.0%)
tracing	1.456 ms [1.431 ms, 1.481 ms]	104.184 µs (7.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.345 ms [1.326 ms, 1.364 ms]	-
appsec	1.712 ms [1.687 ms, 1.738 ms]	367.226 µs (27.3%)
iast	1.484 ms [1.458 ms, 1.509 ms]	138.474 µs (10.3%)
profiling	1.451 ms [1.426 ms, 1.476 ms]	106.123 µs (7.9%)
tracing	1.467 ms [1.442 ms, 1.492 ms]	122.28 µs (9.1%)

Add x-datadog-trace-id header to ITR backend requests

ae49cdf

nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Nov 10, 2023

nikita-tkachenko-datadog and others added 2 commits November 10, 2023 12:07

Add x-datadog-parent-id header

ffb8740

Merge branch 'master' into nikita-tkachenko/x-datadog-trace-id

3991ae0

nikita-tkachenko-datadog marked this pull request as ready for review November 13, 2023 11:55

nikita-tkachenko-datadog requested a review from a team as a code owner November 13, 2023 11:55

nikita-tkachenko-datadog requested a review from juan-fernandez November 13, 2023 11:55

Merge branch 'master' into nikita-tkachenko/x-datadog-trace-id

ec34ac7

juan-fernandez approved these changes Nov 13, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit 5dce4fd into master Nov 13, 2023
67 of 69 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/x-datadog-trace-id branch November 13, 2023 15:28

github-actions bot added this to the 1.24.0 milestone Nov 13, 2023

jandro996 pushed a commit that referenced this pull request Nov 14, 2023

Add x-datadog-trace-id header to ITR backend requests (#6189)

2ed0b49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add x-datadog-trace-id header to ITR backend requests #6189

Add x-datadog-trace-id header to ITR backend requests #6189

nikita-tkachenko-datadog commented Nov 10, 2023 •

edited

Loading

pr-commenter bot commented Nov 10, 2023 •

edited

Loading

Add x-datadog-trace-id header to ITR backend requests #6189

Add x-datadog-trace-id header to ITR backend requests #6189

Conversation

nikita-tkachenko-datadog commented Nov 10, 2023 • edited Loading

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Nov 10, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

nikita-tkachenko-datadog commented Nov 10, 2023 •

edited

Loading

pr-commenter bot commented Nov 10, 2023 •

edited

Loading