Skip to content

Update system-tests to 1de04c42cbd9783432258004e46eb5982bbc9fe5 #8631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
smola merged 1 commit into from
Mar 31, 2025
Merged

Update system-tests to 1de04c42cbd9783432258004e46eb5982bbc9fe5 #8631

smola merged 1 commit into from
Mar 31, 2025

Conversation

@smola
Copy link
Member

@smola smola commented Mar 28, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 28, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 28, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-update
git_commit_date 1743418147 1743418161
git_commit_sha c79bf57 4047d54
release_version 1.48.0-SNAPSHOT~c79bf572e7 1.48.0-SNAPSHOT~4047d54bdf
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743421178 1743421178
ci_job_id 872096263 872096263
ci_pipeline_id 60492222 60492222
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-si1bmdmz-project-304-concurrent-0-567amyws 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-si1bmdmz-project-304-concurrent-0-567amyws 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 67 metrics, 4 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058761
Total [baseline] (8.724 s) : 0, 8724174
Agent [candidate] (1.053 s) : 0, 1052570
Total [candidate] (8.657 s) : 0, 8657077
section iast
Agent [baseline] (1.186 s) : 0, 1185685
Total [baseline] (9.246 s) : 0, 9246240
Agent [candidate] (1.188 s) : 0, 1187977
Total [candidate] (9.278 s) : 0, 9277547
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.178 s) : 0, 1178321
Total [baseline] (9.214 s) : 0, 9214049
Agent [candidate] (1.18 s) : 0, 1180000
Total [candidate] (9.183 s) : 0, 9182831
section iast_TELEMETRY_OFF
Agent [baseline] (1.176 s) : 0, 1176175
Total [baseline] (9.201 s) : 0, 9201293
Agent [candidate] (1.179 s) : 0, 1179269
Total [candidate] (9.248 s) : 0, 9248074
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent iast 1.186 s 126.923 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.178 s 119.559 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.176 s 117.413 ms (11.1%)
Total tracing 8.724 s -
Total iast 9.246 s 522.066 ms (6.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.214 s 489.874 ms (5.6%)
Total iast_TELEMETRY_OFF 9.201 s 477.118 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.188 s 135.407 ms (12.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.18 s 127.43 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.179 s 126.698 ms (12.0%)
Total tracing 8.657 s -
Total iast 9.278 s 620.47 ms (7.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.183 s 525.753 ms (6.1%)
Total iast_TELEMETRY_OFF 9.248 s 590.996 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.82 ms) : 0, 723820
BytebuddyAgent [candidate] (719.784 ms) : 0, 719784
GlobalTracer [baseline] (240.49 ms) : 0, 240490
GlobalTracer [candidate] (239.568 ms) : 0, 239568
AppSec [baseline] (55.605 ms) : 0, 55605
AppSec [candidate] (55.125 ms) : 0, 55125
Debugger [baseline] (4.505 ms) : 0, 4505
Debugger [candidate] (4.458 ms) : 0, 4458
Remote Config [baseline] (694.734 µs) : 0, 695
Remote Config [candidate] (689.404 µs) : 0, 689
Telemetry [baseline] (12.861 ms) : 0, 12861
Telemetry [candidate] (12.313 ms) : 0, 12313
section iast
BytebuddyAgent [baseline] (843.219 ms) : 0, 843219
BytebuddyAgent [candidate] (843.754 ms) : 0, 843754
GlobalTracer [baseline] (229.774 ms) : 0, 229774
GlobalTracer [candidate] (230.585 ms) : 0, 230585
AppSec [baseline] (55.558 ms) : 0, 55558
AppSec [candidate] (56.11 ms) : 0, 56110
Debugger [baseline] (4.186 ms) : 0, 4186
Debugger [candidate] (4.207 ms) : 0, 4207
Remote Config [baseline] (608.754 µs) : 0, 609
Remote Config [candidate] (619.6 µs) : 0, 620
Telemetry [baseline] (8.892 ms) : 0, 8892
Telemetry [candidate] (8.966 ms) : 0, 8966
IAST [baseline] (22.851 ms) : 0, 22851
IAST [candidate] (23.141 ms) : 0, 23141
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.959 ms) : 0, 835959
BytebuddyAgent [candidate] (836.986 ms) : 0, 836986
GlobalTracer [baseline] (228.782 ms) : 0, 228782
GlobalTracer [candidate] (229.327 ms) : 0, 229327
AppSec [baseline] (56.364 ms) : 0, 56364
AppSec [candidate] (56.301 ms) : 0, 56301
Debugger [baseline] (4.246 ms) : 0, 4246
Debugger [candidate] (4.27 ms) : 0, 4270
Remote Config [baseline] (613.163 µs) : 0, 613
Remote Config [candidate] (614.212 µs) : 0, 614
Telemetry [baseline] (8.884 ms) : 0, 8884
Telemetry [candidate] (8.981 ms) : 0, 8981
IAST [baseline] (23.01 ms) : 0, 23010
IAST [candidate] (23.036 ms) : 0, 23036
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (835.521 ms) : 0, 835521
BytebuddyAgent [candidate] (837.027 ms) : 0, 837027
GlobalTracer [baseline] (228.429 ms) : 0, 228429
GlobalTracer [candidate] (229.105 ms) : 0, 229105
AppSec [baseline] (55.865 ms) : 0, 55865
AppSec [candidate] (56.457 ms) : 0, 56457
Debugger [baseline] (4.155 ms) : 0, 4155
Debugger [candidate] (4.236 ms) : 0, 4236
Remote Config [baseline] (610.402 µs) : 0, 610
Remote Config [candidate] (615.215 µs) : 0, 615
Telemetry [baseline] (8.706 ms) : 0, 8706
Telemetry [candidate] (8.801 ms) : 0, 8801
IAST [baseline] (22.469 ms) : 0, 22469
IAST [candidate] (22.464 ms) : 0, 22464
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1054794
Total [baseline] (10.425 s) : 0, 10425378
Agent [candidate] (1.051 s) : 0, 1051114
Total [candidate] (10.453 s) : 0, 10452653
section appsec
Agent [baseline] (1.197 s) : 0, 1196711
Total [baseline] (10.758 s) : 0, 10757939
Agent [candidate] (1.195 s) : 0, 1195461
Total [candidate] (10.799 s) : 0, 10798869
section iast
Agent [baseline] (1.178 s) : 0, 1178297
Total [baseline] (10.981 s) : 0, 10980761
Agent [candidate] (1.189 s) : 0, 1188615
Total [candidate] (11.144 s) : 0, 11143883
section profiling
Agent [baseline] (1.28 s) : 0, 1279890
Total [baseline] (10.832 s) : 0, 10832441
Agent [candidate] (1.275 s) : 0, 1275024
Total [candidate] (10.877 s) : 0, 10876538
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.197 s 141.917 ms (13.5%)
Agent iast 1.178 s 123.503 ms (11.7%)
Agent profiling 1.28 s 225.096 ms (21.3%)
Total tracing 10.425 s -
Total appsec 10.758 s 332.561 ms (3.2%)
Total iast 10.981 s 555.383 ms (5.3%)
Total profiling 10.832 s 407.063 ms (3.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent appsec 1.195 s 144.347 ms (13.7%)
Agent iast 1.189 s 137.5 ms (13.1%)
Agent profiling 1.275 s 223.909 ms (21.3%)
Total tracing 10.453 s -
Total appsec 10.799 s 346.215 ms (3.3%)
Total iast 11.144 s 691.23 ms (6.6%)
Total profiling 10.877 s 423.885 ms (4.1%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.823 ms) : 0, 719823
BytebuddyAgent [candidate] (717.579 ms) : 0, 717579
GlobalTracer [baseline] (240.011 ms) : 0, 240011
GlobalTracer [candidate] (238.735 ms) : 0, 238735
AppSec [baseline] (54.659 ms) : 0, 54659
AppSec [candidate] (54.336 ms) : 0, 54336
Debugger [baseline] (4.488 ms) : 0, 4488
Debugger [candidate] (5.095 ms) : 0, 5095
Remote Config [baseline] (688.96 µs) : 0, 689
Remote Config [candidate] (680.886 µs) : 0, 681
Telemetry [baseline] (14.456 ms) : 0, 14456
Telemetry [candidate] (14.124 ms) : 0, 14124
section appsec
BytebuddyAgent [baseline] (739.029 ms) : 0, 739029
BytebuddyAgent [candidate] (739.03 ms) : 0, 739030
GlobalTracer [baseline] (236.242 ms) : 0, 236242
GlobalTracer [candidate] (235.428 ms) : 0, 235428
AppSec [baseline] (176.308 ms) : 0, 176308
AppSec [candidate] (176.192 ms) : 0, 176192
Debugger [baseline] (4.288 ms) : 0, 4288
Debugger [candidate] (4.298 ms) : 0, 4298
Remote Config [baseline] (644.149 µs) : 0, 644
Remote Config [candidate] (644.753 µs) : 0, 645
Telemetry [baseline] (8.545 ms) : 0, 8545
Telemetry [candidate] (8.56 ms) : 0, 8560
IAST [baseline] (22.093 ms) : 0, 22093
IAST [candidate] (21.536 ms) : 0, 21536
section iast
BytebuddyAgent [baseline] (836.912 ms) : 0, 836912
BytebuddyAgent [candidate] (844.446 ms) : 0, 844446
GlobalTracer [baseline] (228.552 ms) : 0, 228552
GlobalTracer [candidate] (230.226 ms) : 0, 230226
AppSec [baseline] (55.997 ms) : 0, 55997
AppSec [candidate] (56.354 ms) : 0, 56354
Debugger [baseline] (4.194 ms) : 0, 4194
Debugger [candidate] (4.264 ms) : 0, 4264
Remote Config [baseline] (602.124 µs) : 0, 602
Remote Config [candidate] (624.319 µs) : 0, 624
Telemetry [baseline] (8.83 ms) : 0, 8830
Telemetry [candidate] (8.897 ms) : 0, 8897
IAST [baseline] (22.749 ms) : 0, 22749
IAST [candidate] (23.187 ms) : 0, 23187
section profiling
BytebuddyAgent [baseline] (712.972 ms) : 0, 712972
BytebuddyAgent [candidate] (711.436 ms) : 0, 711436
GlobalTracer [baseline] (350.188 ms) : 0, 350188
GlobalTracer [candidate] (347.721 ms) : 0, 347721
AppSec [baseline] (54.018 ms) : 0, 54018
AppSec [candidate] (53.239 ms) : 0, 53239
Debugger [baseline] (4.38 ms) : 0, 4380
Debugger [candidate] (4.309 ms) : 0, 4309
Remote Config [baseline] (707.009 µs) : 0, 707
Remote Config [candidate] (703.38 µs) : 0, 703
Telemetry [baseline] (9.075 ms) : 0, 9075
Telemetry [candidate] (9.045 ms) : 0, 9045
ProfilingAgent [baseline] (102.761 ms) : 0, 102761
ProfilingAgent [candidate] (102.719 ms) : 0, 102719
Profiling [baseline] (102.786 ms) : 0, 102786
Profiling [candidate] (102.745 ms) : 0, 102745
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-31T11:07:49 2025-03-31T11:15:36
git_branch master smola/system-tests-update
git_commit_date 1743418147 1743418161
git_commit_sha c79bf57 4047d54
release_version 1.48.0-SNAPSHOT~c79bf572e7 1.48.0-SNAPSHOT~4047d54bdf
start_time 2025-03-31T11:07:35 2025-03-31T11:15:22
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743420137 1743420137
ci_job_id 872096264 872096264
ci_pipeline_id 60492222 60492222
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-si1bmdmz-project-304-concurrent-1-hnvairu2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-si1bmdmz-project-304-concurrent-1-hnvairu2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7
    dateFormat X
    axisFormat %s
section baseline
no_agent (392.483 µs) : 373, 412
.   : milestone, 392,
iast (522.419 µs) : 501, 544
.   : milestone, 522,
iast_FULL (738.604 µs) : 716, 761
.   : milestone, 739,
iast_GLOBAL (570.151 µs) : 547, 593
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (521.261 µs) : 500, 543
.   : milestone, 521,
iast_INACTIVE (470.798 µs) : 450, 492
.   : milestone, 471,
iast_TELEMETRY_OFF (505.2 µs) : 483, 527
.   : milestone, 505,
tracing (464.913 µs) : 444, 486
.   : milestone, 465,
section candidate
no_agent (386.524 µs) : 367, 406
.   : milestone, 387,
iast (515.276 µs) : 493, 537
.   : milestone, 515,
iast_FULL (740.993 µs) : 719, 763
.   : milestone, 741,
iast_GLOBAL (564.598 µs) : 543, 587
.   : milestone, 565,
iast_HARDCODED_SECRET_DISABLED (522.864 µs) : 501, 545
.   : milestone, 523,
iast_INACTIVE (471.832 µs) : 450, 493
.   : milestone, 472,
iast_TELEMETRY_OFF (513.16 µs) : 491, 535
.   : milestone, 513,
tracing (471.666 µs) : 450, 493
.   : milestone, 472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 392.483 µs [372.813 µs, 412.153 µs] -
iast 522.419 µs [500.976 µs, 543.862 µs] 129.936 µs (33.1%)
iast_FULL 738.604 µs [716.292 µs, 760.916 µs] 346.121 µs (88.2%)
iast_GLOBAL 570.151 µs [547.458 µs, 592.843 µs] 177.667 µs (45.3%)
iast_HARDCODED_SECRET_DISABLED 521.261 µs [499.578 µs, 542.945 µs] 128.778 µs (32.8%)
iast_INACTIVE 470.798 µs [449.666 µs, 491.931 µs] 78.315 µs (20.0%)
iast_TELEMETRY_OFF 505.2 µs [483.082 µs, 527.318 µs] 112.716 µs (28.7%)
tracing 464.913 µs [443.969 µs, 485.857 µs] 72.43 µs (18.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.524 µs [366.695 µs, 406.353 µs] -
iast 515.276 µs [493.451 µs, 537.102 µs] 128.753 µs (33.3%)
iast_FULL 740.993 µs [718.85 µs, 763.136 µs] 354.469 µs (91.7%)
iast_GLOBAL 564.598 µs [542.686 µs, 586.51 µs] 178.074 µs (46.1%)
iast_HARDCODED_SECRET_DISABLED 522.864 µs [500.759 µs, 544.97 µs] 136.34 µs (35.3%)
iast_INACTIVE 471.832 µs [450.478 µs, 493.185 µs] 85.308 µs (22.1%)
iast_TELEMETRY_OFF 513.16 µs [491.034 µs, 535.286 µs] 126.636 µs (32.8%)
tracing 471.666 µs [450.37 µs, 492.962 µs] 85.142 µs (22.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.361 ms) : 1340, 1381
.   : milestone, 1361,
appsec (1.741 ms) : 1718, 1765
.   : milestone, 1741,
appsec_no_iast (1.744 ms) : 1720, 1768
.   : milestone, 1744,
code_origins (1.693 ms) : 1667, 1720
.   : milestone, 1693,
iast (1.52 ms) : 1495, 1545
.   : milestone, 1520,
profiling (1.536 ms) : 1513, 1559
.   : milestone, 1536,
tracing (1.51 ms) : 1483, 1537
.   : milestone, 1510,
section candidate
no_agent (1.369 ms) : 1350, 1388
.   : milestone, 1369,
appsec (1.723 ms) : 1699, 1747
.   : milestone, 1723,
appsec_no_iast (1.732 ms) : 1708, 1756
.   : milestone, 1732,
code_origins (1.702 ms) : 1675, 1729
.   : milestone, 1702,
iast (1.527 ms) : 1502, 1552
.   : milestone, 1527,
profiling (1.572 ms) : 1547, 1596
.   : milestone, 1572,
tracing (1.525 ms) : 1500, 1549
.   : milestone, 1525,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.361 ms [1.34 ms, 1.381 ms] -
appsec 1.741 ms [1.718 ms, 1.765 ms] 380.826 µs (28.0%)
appsec_no_iast 1.744 ms [1.72 ms, 1.768 ms] 383.516 µs (28.2%)
code_origins 1.693 ms [1.667 ms, 1.72 ms] 332.803 µs (24.5%)
iast 1.52 ms [1.495 ms, 1.545 ms] 159.546 µs (11.7%)
profiling 1.536 ms [1.513 ms, 1.559 ms] 175.106 µs (12.9%)
tracing 1.51 ms [1.483 ms, 1.537 ms] 149.145 µs (11.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.369 ms [1.35 ms, 1.388 ms] -
appsec 1.723 ms [1.699 ms, 1.747 ms] 354.133 µs (25.9%)
appsec_no_iast 1.732 ms [1.708 ms, 1.756 ms] 363.345 µs (26.5%)
code_origins 1.702 ms [1.675 ms, 1.729 ms] 332.708 µs (24.3%)
iast 1.527 ms [1.502 ms, 1.552 ms] 157.972 µs (11.5%)
profiling 1.572 ms [1.547 ms, 1.596 ms] 202.664 µs (14.8%)
tracing 1.525 ms [1.5 ms, 1.549 ms] 155.761 µs (11.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-update
git_commit_date 1743418147 1743418161
git_commit_sha c79bf57 4047d54
release_version 1.48.0-SNAPSHOT~c79bf572e7 1.48.0-SNAPSHOT~4047d54bdf
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1743420619 1743420619
ci_job_id 872096265 872096265
ci_pipeline_id 60492222 60492222
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-si1bmdmz-project-304-concurrent-2-tyfg61zi 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-si1bmdmz-project-304-concurrent-2-tyfg61zi 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.351 ms) : 2307, 2395
.   : milestone, 2351,
iast (2.116 ms) : 2061, 2171
.   : milestone, 2116,
iast_GLOBAL (2.156 ms) : 2101, 2212
.   : milestone, 2156,
profiling (1.97 ms) : 1926, 2014
.   : milestone, 1970,
tracing (1.957 ms) : 1914, 2000
.   : milestone, 1957,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.34 ms) : 2296, 2384
.   : milestone, 2340,
iast (2.111 ms) : 2056, 2167
.   : milestone, 2111,
iast_GLOBAL (2.158 ms) : 2102, 2213
.   : milestone, 2158,
profiling (1.993 ms) : 1948, 2038
.   : milestone, 1993,
tracing (1.947 ms) : 1904, 1989
.   : milestone, 1947,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.351 ms [2.307 ms, 2.395 ms] 876.136 µs (59.4%)
iast 2.116 ms [2.061 ms, 2.171 ms] 640.788 µs (43.4%)
iast_GLOBAL 2.156 ms [2.101 ms, 2.212 ms] 681.223 µs (46.2%)
profiling 1.97 ms [1.926 ms, 2.014 ms] 494.916 µs (33.5%)
tracing 1.957 ms [1.914 ms, 2.0 ms] 481.461 µs (32.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.463 ms, 1.486 ms] -
appsec 2.34 ms [2.296 ms, 2.384 ms] 865.767 µs (58.7%)
iast 2.111 ms [2.056 ms, 2.167 ms] 636.985 µs (43.2%)
iast_GLOBAL 2.158 ms [2.102 ms, 2.213 ms] 683.195 µs (46.3%)
profiling 1.993 ms [1.948 ms, 2.038 ms] 518.226 µs (35.1%)
tracing 1.947 ms [1.904 ms, 1.989 ms] 472.128 µs (32.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~4047d54bdf, baseline=1.48.0-SNAPSHOT~c79bf572e7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.224 s) : 15224000, 15224000
.   : milestone, 15224000,
appsec (14.762 s) : 14762000, 14762000
.   : milestone, 14762000,
iast (18.028 s) : 18028000, 18028000
.   : milestone, 18028000,
iast_GLOBAL (18.039 s) : 18039000, 18039000
.   : milestone, 18039000,
profiling (15.173 s) : 15173000, 15173000
.   : milestone, 15173000,
tracing (15.252 s) : 15252000, 15252000
.   : milestone, 15252000,
section candidate
no_agent (15.306 s) : 15306000, 15306000
.   : milestone, 15306000,
appsec (15.049 s) : 15049000, 15049000
.   : milestone, 15049000,
iast (18.886 s) : 18886000, 18886000
.   : milestone, 18886000,
iast_GLOBAL (17.92 s) : 17920000, 17920000
.   : milestone, 17920000,
profiling (15.651 s) : 15651000, 15651000
.   : milestone, 15651000,
tracing (15.322 s) : 15322000, 15322000
.   : milestone, 15322000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.224 s [15.224 s, 15.224 s] -
appsec 14.762 s [14.762 s, 14.762 s] -462.0 ms (-3.0%)
iast 18.028 s [18.028 s, 18.028 s] 2.804 s (18.4%)
iast_GLOBAL 18.039 s [18.039 s, 18.039 s] 2.815 s (18.5%)
profiling 15.173 s [15.173 s, 15.173 s] -51.0 ms (-0.3%)
tracing 15.252 s [15.252 s, 15.252 s] 28.0 ms (0.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.306 s [15.306 s, 15.306 s] -
appsec 15.049 s [15.049 s, 15.049 s] -257.0 ms (-1.7%)
iast 18.886 s [18.886 s, 18.886 s] 3.58 s (23.4%)
iast_GLOBAL 17.92 s [17.92 s, 17.92 s] 2.614 s (17.1%)
profiling 15.651 s [15.651 s, 15.651 s] 345.0 ms (2.3%)
tracing 15.322 s [15.322 s, 15.322 s] 16.0 ms (0.1%)

@smola smola marked this pull request as ready for review March 28, 2025 10:25
@smola smola requested a review from a team as a code owner March 28, 2025 10:25
@smola smola requested a review from dougqh March 28, 2025 10:25
@smola smola enabled auto-merge (squash) March 28, 2025 10:26
@smola smola added tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Mar 28, 2025
@smola smola force-pushed the smola/system-tests-update branch from 365b5ae to 4047d54 Compare March 31, 2025 10:49
@smola smola changed the title Update system-tests to 096aef2c742b9b4ef16d81ba4d6e55e4c8c87ea4 Update system-tests to 1de04c42cbd9783432258004e46eb5982bbc9fe5 Mar 31, 2025
@smola smola merged commit 8e29e20 into master Mar 31, 2025
236 of 252 checks passed
@smola smola deleted the smola/system-tests-update branch March 31, 2025 11:48
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.48.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @mcculls