Add custom test fingerprint tags to ITR requests #6195

nikita-tkachenko-datadog · 2023-11-10T12:56:47Z

What Does This Do

Adds custom test fingerprint extensions to ITR requests for settings and skippable tests.

Motivation

The customers can configure custom tags to extend test fingerprints.
These tags - if present - must be included into the ITR requests

Jira ticket: CIVIS-7742

pr-commenter · 2023-11-10T13:39:52Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~5dce4fdae6	1.24.0-SNAPSHOT~f32913b64c
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1033622
Total [baseline] (9.285 s) : 0, 9285413
Agent [candidate] (1.049 s) : 0, 1048584
Total [candidate] (9.324 s) : 0, 9324228
section appsec
Agent [baseline] (1.127 s) : 0, 1127404
Total [baseline] (9.456 s) : 0, 9456057
Agent [candidate] (1.123 s) : 0, 1122644
Total [candidate] (9.481 s) : 0, 9481389
section iast
Agent [baseline] (1.149 s) : 0, 1148881
Total [baseline] (9.502 s) : 0, 9502347
Agent [candidate] (1.153 s) : 0, 1152505
Total [candidate] (9.568 s) : 0, 9568044
section profiling
Agent [baseline] (1.225 s) : 0, 1225021
Total [baseline] (9.542 s) : 0, 9541829
Agent [candidate] (1.226 s) : 0, 1225825
Total [candidate] (9.612 s) : 0, 9611530

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.034 s	-
Agent	appsec	1.127 s	93.781 ms (9.1%)
Agent	iast	1.149 s	115.259 ms (11.2%)
Agent	profiling	1.225 s	191.398 ms (18.5%)
Total	tracing	9.285 s	-
Total	appsec	9.456 s	170.644 ms (1.8%)
Total	iast	9.502 s	216.934 ms (2.3%)
Total	profiling	9.542 s	256.416 ms (2.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	appsec	1.123 s	74.06 ms (7.1%)
Agent	iast	1.153 s	103.922 ms (9.9%)
Agent	profiling	1.226 s	177.241 ms (16.9%)
Total	tracing	9.324 s	-
Total	appsec	9.481 s	157.161 ms (1.7%)
Total	iast	9.568 s	243.816 ms (2.6%)
Total	profiling	9.612 s	287.302 ms (3.1%)

gantt
    title petclinic - break down per module: candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.066 ms) : 0, 645066
BytebuddyAgent [candidate] (655.236 ms) : 0, 655236
GlobalTracer [baseline] (293.601 ms) : 0, 293601
GlobalTracer [candidate] (297.134 ms) : 0, 297134
AppSec [baseline] (48.654 ms) : 0, 48654
AppSec [candidate] (49.229 ms) : 0, 49229
Remote Config [baseline] (680.158 µs) : 0, 680
Remote Config [candidate] (694.058 µs) : 0, 694
Telemetry [baseline] (11.309 ms) : 0, 11309
Telemetry [candidate] (11.37 ms) : 0, 11370
section appsec
BytebuddyAgent [baseline] (650.076 ms) : 0, 650076
BytebuddyAgent [candidate] (647.013 ms) : 0, 647013
GlobalTracer [baseline] (295.662 ms) : 0, 295662
GlobalTracer [candidate] (294.72 ms) : 0, 294720
AppSec [baseline] (139.614 ms) : 0, 139614
AppSec [candidate] (138.983 ms) : 0, 138983
Remote Config [baseline] (655.379 µs) : 0, 655
Remote Config [candidate] (652.513 µs) : 0, 653
Telemetry [baseline] (6.909 ms) : 0, 6909
Telemetry [candidate] (6.872 ms) : 0, 6872
section iast
BytebuddyAgent [baseline] (764.089 ms) : 0, 764089
BytebuddyAgent [candidate] (766.716 ms) : 0, 766716
GlobalTracer [baseline] (273.473 ms) : 0, 273473
GlobalTracer [candidate] (274.771 ms) : 0, 274771
AppSec [baseline] (47.027 ms) : 0, 47027
AppSec [candidate] (47.228 ms) : 0, 47228
Remote Config [baseline] (571.283 µs) : 0, 571
Remote Config [candidate] (569.136 µs) : 0, 569
Telemetry [baseline] (11.879 ms) : 0, 11879
Telemetry [candidate] (11.195 ms) : 0, 11195
IAST [baseline] (17.494 ms) : 0, 17494
IAST [candidate] (17.519 ms) : 0, 17519
section profiling
ProfilingAgent [baseline] (88.597 ms) : 0, 88597
ProfilingAgent [candidate] (88.382 ms) : 0, 88382
BytebuddyAgent [baseline] (659.869 ms) : 0, 659869
BytebuddyAgent [candidate] (662.486 ms) : 0, 662486
GlobalTracer [baseline] (360.392 ms) : 0, 360392
GlobalTracer [candidate] (359.21 ms) : 0, 359210
AppSec [baseline] (49.322 ms) : 0, 49322
AppSec [candidate] (49.024 ms) : 0, 49024
Remote Config [baseline] (653.14 µs) : 0, 653
Remote Config [candidate] (646.67 µs) : 0, 647
Telemetry [baseline] (11.449 ms) : 0, 11449
Telemetry [candidate] (11.312 ms) : 0, 11312
Profiling [baseline] (88.62 ms) : 0, 88620
Profiling [candidate] (88.407 ms) : 0, 88407

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1036010
Total [baseline] (8.778 s) : 0, 8777982
Agent [candidate] (1.035 s) : 0, 1035320
Total [candidate] (8.8 s) : 0, 8800270
section iast
Agent [baseline] (1.158 s) : 0, 1158095
Total [baseline] (9.337 s) : 0, 9336853
Agent [candidate] (1.159 s) : 0, 1159130
Total [candidate] (9.358 s) : 0, 9357761
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145320
Total [baseline] (9.294 s) : 0, 9294269
Agent [candidate] (1.146 s) : 0, 1145855
Total [candidate] (9.298 s) : 0, 9298209

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.036 s	-
Agent	iast	1.158 s	122.086 ms (11.8%)
Agent	iast_TELEMETRY_OFF	1.145 s	109.311 ms (10.6%)
Total	tracing	8.778 s	-
Total	iast	9.337 s	558.872 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.294 s	516.287 ms (5.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.035 s	-
Agent	iast	1.159 s	123.809 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.146 s	110.535 ms (10.7%)
Total	tracing	8.8 s	-
Total	iast	9.358 s	557.491 ms (6.3%)
Total	iast_TELEMETRY_OFF	9.298 s	497.939 ms (5.7%)

gantt
    title insecure-bank - break down per module: candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (646.378 ms) : 0, 646378
BytebuddyAgent [candidate] (646.003 ms) : 0, 646003
GlobalTracer [baseline] (294.393 ms) : 0, 294393
GlobalTracer [candidate] (294.09 ms) : 0, 294090
AppSec [baseline] (48.647 ms) : 0, 48647
AppSec [candidate] (48.774 ms) : 0, 48774
Remote Config [baseline] (689.877 µs) : 0, 690
Remote Config [candidate] (688.647 µs) : 0, 689
Telemetry [baseline] (11.304 ms) : 0, 11304
Telemetry [candidate] (11.237 ms) : 0, 11237
section iast
BytebuddyAgent [baseline] (770.326 ms) : 0, 770326
BytebuddyAgent [candidate] (770.727 ms) : 0, 770727
GlobalTracer [baseline] (275.686 ms) : 0, 275686
GlobalTracer [candidate] (275.811 ms) : 0, 275811
AppSec [baseline] (46.558 ms) : 0, 46558
AppSec [candidate] (47.028 ms) : 0, 47028
IAST [baseline] (17.596 ms) : 0, 17596
IAST [candidate] (16.274 ms) : 0, 16274
Remote Config [baseline] (571.99 µs) : 0, 572
Remote Config [candidate] (566.186 µs) : 0, 566
Telemetry [baseline] (12.669 ms) : 0, 12669
Telemetry [candidate] (14.016 ms) : 0, 14016
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (758.566 ms) : 0, 758566
BytebuddyAgent [candidate] (759.467 ms) : 0, 759467
GlobalTracer [baseline] (274.088 ms) : 0, 274088
GlobalTracer [candidate] (274.851 ms) : 0, 274851
AppSec [baseline] (46.367 ms) : 0, 46367
AppSec [candidate] (46.35 ms) : 0, 46350
IAST [baseline] (17.39 ms) : 0, 17390
IAST [candidate] (16.849 ms) : 0, 16849
Remote Config [baseline] (563.486 µs) : 0, 563
Remote Config [candidate] (572.388 µs) : 0, 572
Telemetry [baseline] (13.805 ms) : 0, 13805
Telemetry [candidate] (13.311 ms) : 0, 13311

Load

Parameters

	Baseline	Candidate
commit	1.24.0-SNAPSHOT~5dce4fdae6	1.24.0-SNAPSHOT~f32913b64c
config	baseline	candidate
end_time	2023-11-13T15:52:01	2023-11-13T16:08:30
start_time	2023-11-13T15:51:48	2023-11-13T16:08:17

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1322, 1362
.   : milestone, 1342,
appsec (1.708 ms) : 1683, 1734
.   : milestone, 1708,
iast (1.485 ms) : 1461, 1509
.   : milestone, 1485,
profiling (1.45 ms) : 1424, 1475
.   : milestone, 1450,
tracing (1.444 ms) : 1420, 1469
.   : milestone, 1444,
section candidate
no_agent (1.34 ms) : 1320, 1359
.   : milestone, 1340,
appsec (1.722 ms) : 1697, 1747
.   : milestone, 1722,
iast (1.48 ms) : 1455, 1504
.   : milestone, 1480,
profiling (1.486 ms) : 1459, 1512
.   : milestone, 1486,
tracing (1.457 ms) : 1433, 1482
.   : milestone, 1457,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.342 ms [1.322 ms, 1.362 ms]	-
appsec	1.708 ms [1.683 ms, 1.734 ms]	366.627 µs (27.3%)
iast	1.485 ms [1.461 ms, 1.509 ms]	143.1 µs (10.7%)
profiling	1.45 ms [1.424 ms, 1.475 ms]	107.876 µs (8.0%)
tracing	1.444 ms [1.42 ms, 1.469 ms]	102.691 µs (7.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.34 ms [1.32 ms, 1.359 ms]	-
appsec	1.722 ms [1.697 ms, 1.747 ms]	382.232 µs (28.5%)
iast	1.48 ms [1.455 ms, 1.504 ms]	140.074 µs (10.5%)
profiling	1.486 ms [1.459 ms, 1.512 ms]	146.069 µs (10.9%)
tracing	1.457 ms [1.433 ms, 1.482 ms]	117.806 µs (8.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.24.0-SNAPSHOT~f32913b64c, baseline=1.24.0-SNAPSHOT~5dce4fdae6
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.555 µs) : 340, 381
.   : milestone, 361,
iast (462.291 µs) : 441, 483
.   : milestone, 462,
iast_FULL (519.86 µs) : 499, 541
.   : milestone, 520,
iast_INACTIVE (441.359 µs) : 420, 463
.   : milestone, 441,
iast_TELEMETRY_OFF (461.559 µs) : 441, 482
.   : milestone, 462,
tracing (434.141 µs) : 413, 455
.   : milestone, 434,
section candidate
no_agent (361.41 µs) : 340, 383
.   : milestone, 361,
iast (463.794 µs) : 443, 484
.   : milestone, 464,
iast_FULL (519.819 µs) : 499, 540
.   : milestone, 520,
iast_INACTIVE (438.304 µs) : 417, 459
.   : milestone, 438,
iast_TELEMETRY_OFF (454.22 µs) : 434, 475
.   : milestone, 454,
tracing (435.631 µs) : 415, 456
.   : milestone, 436,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.555 µs [339.679 µs, 381.431 µs]	-
iast	462.291 µs [441.465 µs, 483.116 µs]	101.736 µs (28.2%)
iast_FULL	519.86 µs [499.049 µs, 540.672 µs]	159.305 µs (44.2%)
iast_INACTIVE	441.359 µs [419.995 µs, 462.723 µs]	80.804 µs (22.4%)
iast_TELEMETRY_OFF	461.559 µs [440.916 µs, 482.202 µs]	101.004 µs (28.0%)
tracing	434.141 µs [412.817 µs, 455.465 µs]	73.586 µs (20.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	361.41 µs [340.118 µs, 382.703 µs]	-
iast	463.794 µs [443.099 µs, 484.489 µs]	102.384 µs (28.3%)
iast_FULL	519.819 µs [499.139 µs, 540.499 µs]	158.409 µs (43.8%)
iast_INACTIVE	438.304 µs [417.163 µs, 459.445 µs]	76.894 µs (21.3%)
iast_TELEMETRY_OFF	454.22 µs [433.797 µs, 474.643 µs]	92.81 µs (25.7%)
tracing	435.631 µs [414.875 µs, 456.388 µs]	74.221 µs (20.5%)

…quests

Add custom test fingerprint tags to ITR requests

6fc89b8

nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Nov 10, 2023

Merge branch 'master' into nikita-tkachenko/add-custom-tags-to-itr-re…

f32913b

…quests

nikita-tkachenko-datadog marked this pull request as ready for review November 13, 2023 16:54

nikita-tkachenko-datadog requested a review from a team as a code owner November 13, 2023 16:54

nikita-tkachenko-datadog requested a review from ericlaz November 13, 2023 16:54

juan-fernandez approved these changes Nov 14, 2023

View reviewed changes

nikita-tkachenko-datadog merged commit 80d0355 into master Nov 14, 2023
66 of 69 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/add-custom-tags-to-itr-requests branch November 14, 2023 11:39

github-actions bot added this to the 1.24.0 milestone Nov 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom test fingerprint tags to ITR requests #6195

Add custom test fingerprint tags to ITR requests #6195

nikita-tkachenko-datadog commented Nov 10, 2023 •

edited by jira bot

Loading

pr-commenter bot commented Nov 10, 2023 •

edited

Loading

Add custom test fingerprint tags to ITR requests #6195

Add custom test fingerprint tags to ITR requests #6195

Conversation

nikita-tkachenko-datadog commented Nov 10, 2023 • edited by jira bot Loading

What Does This Do

Motivation

pr-commenter bot commented Nov 10, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

nikita-tkachenko-datadog commented Nov 10, 2023 •

edited by jira bot

Loading

pr-commenter bot commented Nov 10, 2023 •

edited

Loading