Limit the number of verified tag vectors in CI Visibility metric tests #9905

nikita-tkachenko-datadog · 2025-11-06T10:00:07Z

What Does This Do

Updates datadog.trace.civisibility.telemetry.CiVisibilityMetricCollectorTest to limit the number of tested metric/tags pairs.

Motivation

The test is verifying all the possible metric/tag pairs, which is redundant. As the number of metrics grew over time, the test execution time increased non-linearly. It now takes more than 10 minutes to run.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-datadog-prod-us1 · 2025-11-06T10:06:47Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 63.87% (+4.19%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 57ec2c7 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-11-06T10:40:07Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/metric-test-update
git_commit_date	1762297540	1762421698
git_commit_sha	`8db72c0`	`57ec2c7`
release_version	1.56.0-SNAPSHOT~8db72c0988	1.56.0-SNAPSHOT~57ec2c774e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1762424554	1762424554
ci_job_id	1219141881	1219141881
ci_pipeline_id	81428674	81428674
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-2vkldycp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-2vkldycp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 10 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1054131
Total [baseline] (8.644 s) : 0, 8643960
Agent [candidate] (1.047 s) : 0, 1046580
Total [candidate] (8.607 s) : 0, 8607225
section iast
Agent [baseline] (1.18 s) : 0, 1179545
Total [baseline] (9.243 s) : 0, 9242643
Agent [candidate] (1.184 s) : 0, 1184194
Total [candidate] (9.258 s) : 0, 9258378

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	iast	1.18 s	125.414 ms (11.9%)
Total	tracing	8.644 s	-
Total	iast	9.243 s	598.682 ms (6.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.184 s	137.615 ms (13.1%)
Total	tracing	8.607 s	-
Total	iast	9.258 s	651.153 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (709.343 ms) : 0, 709343
BytebuddyAgent [candidate] (704.439 ms) : 0, 704439
GlobalTracer [baseline] (247.506 ms) : 0, 247506
GlobalTracer [candidate] (246.114 ms) : 0, 246114
AppSec [baseline] (32.64 ms) : 0, 32640
AppSec [candidate] (32.129 ms) : 0, 32129
Debugger [baseline] (6.451 ms) : 0, 6451
Debugger [candidate] (6.381 ms) : 0, 6381
Remote Config [baseline] (725.67 µs) : 0, 726
Remote Config [candidate] (708.924 µs) : 0, 709
Telemetry [baseline] (12.999 ms) : 0, 12999
Telemetry [candidate] (12.761 ms) : 0, 12761
Flare Poller [baseline] (8.184 ms) : 0, 8184
Flare Poller [candidate] (7.908 ms) : 0, 7908
section iast
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.469 ms) : 0, 1469
BytebuddyAgent [baseline] (828.706 ms) : 0, 828706
BytebuddyAgent [candidate] (833.245 ms) : 0, 833245
GlobalTracer [baseline] (234.615 ms) : 0, 234615
GlobalTracer [candidate] (234.887 ms) : 0, 234887
IAST [baseline] (34.22 ms) : 0, 34220
IAST [candidate] (34.977 ms) : 0, 34977
AppSec [baseline] (26.943 ms) : 0, 26943
AppSec [candidate] (25.887 ms) : 0, 25887
Debugger [baseline] (5.974 ms) : 0, 5974
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (581.76 µs) : 0, 582
Remote Config [candidate] (585.02 µs) : 0, 585
Telemetry [baseline] (8.293 ms) : 0, 8293
Telemetry [candidate] (8.208 ms) : 0, 8208
Flare Poller [baseline] (4.127 ms) : 0, 4127
Flare Poller [candidate] (4.088 ms) : 0, 4088

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1045582
Total [baseline] (10.668 s) : 0, 10668452
Agent [candidate] (1.046 s) : 0, 1045962
Total [candidate] (10.794 s) : 0, 10793910
section appsec
Agent [baseline] (1.23 s) : 0, 1229661
Total [baseline] (10.809 s) : 0, 10809384
Agent [candidate] (1.218 s) : 0, 1217954
Total [candidate] (10.906 s) : 0, 10906138
section iast
Agent [baseline] (1.196 s) : 0, 1195783
Total [baseline] (11.239 s) : 0, 11238529
Agent [candidate] (1.185 s) : 0, 1184727
Total [candidate] (11.105 s) : 0, 11105402
section profiling
Agent [baseline] (1.188 s) : 0, 1188468
Total [baseline] (10.896 s) : 0, 10896479
Agent [candidate] (1.198 s) : 0, 1198301
Total [candidate] (10.945 s) : 0, 10945246

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.23 s	184.079 ms (17.6%)
Agent	iast	1.196 s	150.201 ms (14.4%)
Agent	profiling	1.188 s	142.886 ms (13.7%)
Total	tracing	10.668 s	-
Total	appsec	10.809 s	140.932 ms (1.3%)
Total	iast	11.239 s	570.076 ms (5.3%)
Total	profiling	10.896 s	228.027 ms (2.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.218 s	171.992 ms (16.4%)
Agent	iast	1.185 s	138.765 ms (13.3%)
Agent	profiling	1.198 s	152.339 ms (14.6%)
Total	tracing	10.794 s	-
Total	appsec	10.906 s	112.228 ms (1.0%)
Total	iast	11.105 s	311.492 ms (2.9%)
Total	profiling	10.945 s	151.336 ms (1.4%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.454 ms) : 0, 1454
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (703.366 ms) : 0, 703366
BytebuddyAgent [candidate] (704.079 ms) : 0, 704079
GlobalTracer [baseline] (245.819 ms) : 0, 245819
GlobalTracer [candidate] (245.975 ms) : 0, 245975
AppSec [baseline] (32.299 ms) : 0, 32299
AppSec [candidate] (32.253 ms) : 0, 32253
Debugger [baseline] (6.382 ms) : 0, 6382
Debugger [candidate] (6.37 ms) : 0, 6370
Remote Config [baseline] (710.152 µs) : 0, 710
Remote Config [candidate] (710.199 µs) : 0, 710
Telemetry [baseline] (15.055 ms) : 0, 15055
Telemetry [candidate] (14.88 ms) : 0, 14880
Flare Poller [baseline] (5.791 ms) : 0, 5791
Flare Poller [candidate] (5.674 ms) : 0, 5674
section appsec
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (734.979 ms) : 0, 734979
BytebuddyAgent [candidate] (726.901 ms) : 0, 726901
GlobalTracer [baseline] (239.116 ms) : 0, 239116
GlobalTracer [candidate] (236.74 ms) : 0, 236740
IAST [baseline] (25.053 ms) : 0, 25053
IAST [candidate] (24.692 ms) : 0, 24692
AppSec [baseline] (174.777 ms) : 0, 174777
AppSec [candidate] (174.281 ms) : 0, 174281
Debugger [baseline] (6.002 ms) : 0, 6002
Debugger [candidate] (6.005 ms) : 0, 6005
Remote Config [baseline] (654.844 µs) : 0, 655
Remote Config [candidate] (646.741 µs) : 0, 647
Telemetry [baseline] (8.588 ms) : 0, 8588
Telemetry [candidate] (8.574 ms) : 0, 8574
Flare Poller [baseline] (4.044 ms) : 0, 4044
Flare Poller [candidate] (3.94 ms) : 0, 3940
section iast
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.467 ms) : 0, 1467
BytebuddyAgent [baseline] (842.188 ms) : 0, 842188
BytebuddyAgent [candidate] (832.668 ms) : 0, 832668
GlobalTracer [baseline] (235.75 ms) : 0, 235750
GlobalTracer [candidate] (235.479 ms) : 0, 235479
IAST [baseline] (32.883 ms) : 0, 32883
IAST [candidate] (33.549 ms) : 0, 33549
AppSec [baseline] (29.037 ms) : 0, 29037
AppSec [candidate] (27.761 ms) : 0, 27761
Debugger [baseline] (6.096 ms) : 0, 6096
Debugger [candidate] (6.018 ms) : 0, 6018
Remote Config [baseline] (601.355 µs) : 0, 601
Remote Config [candidate] (592.666 µs) : 0, 593
Telemetry [baseline] (8.558 ms) : 0, 8558
Telemetry [candidate] (8.368 ms) : 0, 8368
Flare Poller [baseline] (4.139 ms) : 0, 4139
Flare Poller [candidate] (4.105 ms) : 0, 4105
section profiling
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.473 ms) : 0, 1473
BytebuddyAgent [baseline] (727.425 ms) : 0, 727425
BytebuddyAgent [candidate] (733.743 ms) : 0, 733743
GlobalTracer [baseline] (221.103 ms) : 0, 221103
GlobalTracer [candidate] (222.803 ms) : 0, 222803
AppSec [baseline] (31.801 ms) : 0, 31801
AppSec [candidate] (32.243 ms) : 0, 32243
Debugger [baseline] (8.329 ms) : 0, 8329
Debugger [candidate] (7.657 ms) : 0, 7657
Remote Config [baseline] (672.771 µs) : 0, 673
Remote Config [candidate] (675.126 µs) : 0, 675
Telemetry [baseline] (14.35 ms) : 0, 14350
Telemetry [candidate] (14.498 ms) : 0, 14498
Flare Poller [baseline] (4.088 ms) : 0, 4088
Flare Poller [candidate] (4.157 ms) : 0, 4157
ProfilingAgent [baseline] (110.246 ms) : 0, 110246
ProfilingAgent [candidate] (110.819 ms) : 0, 110819
Profiling [baseline] (110.888 ms) : 0, 110888
Profiling [candidate] (111.507 ms) : 0, 111507

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/metric-test-update
git_commit_date	1762297540	1762421698
git_commit_sha	`8db72c0`	`57ec2c7`
release_version	1.56.0-SNAPSHOT~8db72c0988	1.56.0-SNAPSHOT~57ec2c774e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1762425046	1762425046
ci_job_id	1219141882	1219141882
ci_pipeline_id	81428674	81428674
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-e0smxtnn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-e0smxtnn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	worse [+149.785µs; +238.394µs] or [+4.859%; +7.734%]	unstable [-235.426op/s; +64.801op/s] or [-15.884%; +4.372%]	3.277ms	1396.875op/s	3.082ms	1482.188op/s
scenario:load:insecure-bank:profiling:high_load	better [-172.877µs; -132.980µs] or [-8.212%; -6.317%]	unstable [-92.354op/s; +418.417op/s] or [-4.301%; +19.487%]	1.952ms	2310.188op/s	2.105ms	2147.156op/s
scenario:load:petclinic:profiling:high_load	worse [+0.884ms; +1.294ms] or [+4.785%; +7.007%]	unstable [-39.143op/s; +11.080op/s] or [-15.494%; +4.386%]	19.562ms	238.594op/s	18.473ms	252.625op/s
scenario:load:petclinic:iast:high_load	worse [+781.552µs; +1176.975µs] or [+4.437%; +6.681%]	unstable [-40.334op/s; +12.647op/s] or [-15.229%; +4.775%]	18.595ms	251.000op/s	17.616ms	264.844op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.224 ms) : 1212, 1236
.   : milestone, 1224,
iast (3.082 ms) : 3047, 3118
.   : milestone, 3082,
iast_FULL (5.874 ms) : 5815, 5933
.   : milestone, 5874,
iast_GLOBAL (3.629 ms) : 3576, 3682
.   : milestone, 3629,
profiling (2.105 ms) : 2085, 2126
.   : milestone, 2105,
tracing (1.81 ms) : 1795, 1825
.   : milestone, 1810,
section candidate
no_agent (1.223 ms) : 1210, 1235
.   : milestone, 1223,
iast (3.277 ms) : 3231, 3323
.   : milestone, 3277,
iast_FULL (5.83 ms) : 5772, 5889
.   : milestone, 5830,
iast_GLOBAL (3.687 ms) : 3632, 3742
.   : milestone, 3687,
profiling (1.952 ms) : 1936, 1969
.   : milestone, 1952,
tracing (1.861 ms) : 1843, 1879
.   : milestone, 1861,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.224 ms [1.212 ms, 1.236 ms]	-
iast	3.082 ms [3.047 ms, 3.118 ms]	1.859 ms (151.9%)
iast_FULL	5.874 ms [5.815 ms, 5.933 ms]	4.65 ms (379.9%)
iast_GLOBAL	3.629 ms [3.576 ms, 3.682 ms]	2.406 ms (196.5%)
profiling	2.105 ms [2.085 ms, 2.126 ms]	881.164 µs (72.0%)
tracing	1.81 ms [1.795 ms, 1.825 ms]	585.611 µs (47.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.223 ms [1.21 ms, 1.235 ms]	-
iast	3.277 ms [3.231 ms, 3.323 ms]	2.054 ms (168.0%)
iast_FULL	5.83 ms [5.772 ms, 5.889 ms]	4.608 ms (376.9%)
iast_GLOBAL	3.687 ms [3.632 ms, 3.742 ms]	2.464 ms (201.6%)
profiling	1.952 ms [1.936 ms, 1.969 ms]	729.63 µs (59.7%)
tracing	1.861 ms [1.843 ms, 1.879 ms]	638.391 µs (52.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.073 ms) : 17885, 18260
.   : milestone, 18073,
appsec (18.905 ms) : 18711, 19098
.   : milestone, 18905,
code_origins (17.591 ms) : 17416, 17766
.   : milestone, 17591,
iast (17.616 ms) : 17439, 17792
.   : milestone, 17616,
profiling (18.473 ms) : 18290, 18656
.   : milestone, 18473,
tracing (17.594 ms) : 17419, 17768
.   : milestone, 17594,
section candidate
no_agent (18.502 ms) : 18307, 18697
.   : milestone, 18502,
appsec (18.572 ms) : 18381, 18763
.   : milestone, 18572,
code_origins (17.994 ms) : 17815, 18173
.   : milestone, 17994,
iast (18.595 ms) : 18404, 18785
.   : milestone, 18595,
profiling (19.562 ms) : 19364, 19760
.   : milestone, 19562,
tracing (17.546 ms) : 17370, 17721
.   : milestone, 17546,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.073 ms [17.885 ms, 18.26 ms]	-
appsec	18.905 ms [18.711 ms, 19.098 ms]	832.131 µs (4.6%)
code_origins	17.591 ms [17.416 ms, 17.766 ms]	-481.721 µs (-2.7%)
iast	17.616 ms [17.439 ms, 17.792 ms]	-457.099 µs (-2.5%)
profiling	18.473 ms [18.29 ms, 18.656 ms]	400.289 µs (2.2%)
tracing	17.594 ms [17.419 ms, 17.768 ms]	-478.951 µs (-2.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.502 ms [18.307 ms, 18.697 ms]	-
appsec	18.572 ms [18.381 ms, 18.763 ms]	70.42 µs (0.4%)
code_origins	17.994 ms [17.815 ms, 18.173 ms]	-507.633 µs (-2.7%)
iast	18.595 ms [18.404 ms, 18.785 ms]	93.051 µs (0.5%)
profiling	19.562 ms [19.364 ms, 19.76 ms]	1.06 ms (5.7%)
tracing	17.546 ms [17.37 ms, 17.721 ms]	-956.115 µs (-5.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/metric-test-update
git_commit_date	1762297540	1762421698
git_commit_sha	`8db72c0`	`57ec2c7`
release_version	1.56.0-SNAPSHOT~8db72c0988	1.56.0-SNAPSHOT~57ec2c774e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1762424677	1762424677
ci_job_id	1219141883	1219141883
ci_pipeline_id	81428674	81428674
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-e7b81dsl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-e7b81dsl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.952 s) : 14952000, 14952000
.   : milestone, 14952000,
appsec (15.162 s) : 15162000, 15162000
.   : milestone, 15162000,
iast (18.536 s) : 18536000, 18536000
.   : milestone, 18536000,
iast_GLOBAL (18.261 s) : 18261000, 18261000
.   : milestone, 18261000,
profiling (15.064 s) : 15064000, 15064000
.   : milestone, 15064000,
tracing (14.854 s) : 14854000, 14854000
.   : milestone, 14854000,
section candidate
no_agent (15.398 s) : 15398000, 15398000
.   : milestone, 15398000,
appsec (15.5 s) : 15500000, 15500000
.   : milestone, 15500000,
iast (18.457 s) : 18457000, 18457000
.   : milestone, 18457000,
iast_GLOBAL (17.973 s) : 17973000, 17973000
.   : milestone, 17973000,
profiling (15.172 s) : 15172000, 15172000
.   : milestone, 15172000,
tracing (15.194 s) : 15194000, 15194000
.   : milestone, 15194000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.952 s [14.952 s, 14.952 s]	-
appsec	15.162 s [15.162 s, 15.162 s]	210.0 ms (1.4%)
iast	18.536 s [18.536 s, 18.536 s]	3.584 s (24.0%)
iast_GLOBAL	18.261 s [18.261 s, 18.261 s]	3.309 s (22.1%)
profiling	15.064 s [15.064 s, 15.064 s]	112.0 ms (0.7%)
tracing	14.854 s [14.854 s, 14.854 s]	-98.0 ms (-0.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.398 s [15.398 s, 15.398 s]	-
appsec	15.5 s [15.5 s, 15.5 s]	102.0 ms (0.7%)
iast	18.457 s [18.457 s, 18.457 s]	3.059 s (19.9%)
iast_GLOBAL	17.973 s [17.973 s, 17.973 s]	2.575 s (16.7%)
profiling	15.172 s [15.172 s, 15.172 s]	-226.0 ms (-1.5%)
tracing	15.194 s [15.194 s, 15.194 s]	-204.0 ms (-1.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~57ec2c774e, baseline=1.56.0-SNAPSHOT~8db72c0988
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.451 ms) : 2400, 2502
.   : milestone, 2451,
iast (2.209 ms) : 2145, 2272
.   : milestone, 2209,
iast_GLOBAL (2.248 ms) : 2184, 2312
.   : milestone, 2248,
profiling (2.493 ms) : 2324, 2662
.   : milestone, 2493,
tracing (2.023 ms) : 1973, 2072
.   : milestone, 2023,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (3.684 ms) : 3467, 3901
.   : milestone, 3684,
iast (2.21 ms) : 2146, 2273
.   : milestone, 2210,
iast_GLOBAL (2.258 ms) : 2193, 2322
.   : milestone, 2258,
profiling (2.047 ms) : 1996, 2098
.   : milestone, 2047,
tracing (2.027 ms) : 1978, 2077
.   : milestone, 2027,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	2.451 ms [2.4 ms, 2.502 ms]	976.505 µs (66.2%)
iast	2.209 ms [2.145 ms, 2.272 ms]	733.897 µs (49.8%)
iast_GLOBAL	2.248 ms [2.184 ms, 2.312 ms]	773.312 µs (52.4%)
profiling	2.493 ms [2.324 ms, 2.662 ms]	1.019 ms (69.1%)
tracing	2.023 ms [1.973 ms, 2.072 ms]	548.127 µs (37.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	3.684 ms [3.467 ms, 3.901 ms]	2.209 ms (149.8%)
iast	2.21 ms [2.146 ms, 2.273 ms]	734.742 µs (49.8%)
iast_GLOBAL	2.258 ms [2.193 ms, 2.322 ms]	782.618 µs (53.1%)
profiling	2.047 ms [1.996 ms, 2.098 ms]	572.188 µs (38.8%)
tracing	2.027 ms [1.978 ms, 2.077 ms]	552.497 µs (37.5%)

Limit the number of verified tag vectors in CI Visibility metric tests

57ec2c7

nikita-tkachenko-datadog added the comp: testing Testing label Nov 6, 2025

nikita-tkachenko-datadog requested a review from a team as a code owner November 6, 2025 10:00

nikita-tkachenko-datadog added the tag: no release notes Changes to exclude from release notes label Nov 6, 2025

nikita-tkachenko-datadog requested a review from Mariovido November 6, 2025 10:00

nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Nov 6, 2025

daniel-mohedano approved these changes Nov 6, 2025

View reviewed changes

nikita-tkachenko-datadog merged commit 59d7f4e into master Nov 6, 2025
545 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/metric-test-update branch November 6, 2025 10:49

github-actions bot added this to the 1.56.0 milestone Nov 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Limit the number of verified tag vectors in CI Visibility metric tests #9905

Limit the number of verified tag vectors in CI Visibility metric tests #9905

Uh oh!

nikita-tkachenko-datadog commented Nov 6, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 6, 2025

Uh oh!

pr-commenter bot commented Nov 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Limit the number of verified tag vectors in CI Visibility metric tests #9905

Limit the number of verified tag vectors in CI Visibility metric tests #9905

Uh oh!

Conversation

nikita-tkachenko-datadog commented Nov 6, 2025

What Does This Do

Motivation

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 6, 2025

Uh oh!

pr-commenter bot commented Nov 6, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants