Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update system-tests to 4d979bdcab25754d38a8d190525bef87dad66a6e #7095

Merged
merged 3 commits into from
Jun 4, 2024
Merged

Update system-tests to 4d979bdcab25754d38a8d190525bef87dad66a6e #7095

merged 3 commits into from
Jun 4, 2024

Conversation

smola
Copy link
Member

@smola smola commented May 30, 2024

What Does This Do

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels May 30, 2024
@smola smola requested a review from a team as a code owner May 30, 2024 12:47
@smola smola requested review from dougqh and nayeem-kamal May 30, 2024 12:47
@pr-commenter
Copy link

pr-commenter bot commented May 31, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-20240530
git_commit_date 1717427688 1717427714
git_commit_sha bf9fd09 03ad506
release_version 1.35.0-SNAPSHOT~bf9fd09c02 1.35.0-SNAPSHOT~03ad50604c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1717430679 1717430679
ci_job_id 530556730 530556730
ci_pipeline_id 35707484 35707484
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062159
Total [baseline] (8.571 s) : 0, 8570607
Agent [candidate] (1.058 s) : 0, 1057873
Total [candidate] (8.54 s) : 0, 8539673
section iast
Agent [baseline] (1.173 s) : 0, 1172817
Total [baseline] (8.994 s) : 0, 8994269
Agent [candidate] (1.172 s) : 0, 1171869
Total [candidate] (9.014 s) : 0, 9014317
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.166 s) : 0, 1165977
Total [baseline] (8.985 s) : 0, 8985141
Agent [candidate] (1.168 s) : 0, 1168209
Total [candidate] (8.964 s) : 0, 8964337
section iast_TELEMETRY_OFF
Agent [baseline] (1.159 s) : 0, 1159420
Total [baseline] (8.99 s) : 0, 8990433
Agent [candidate] (1.164 s) : 0, 1164225
Total [candidate] (8.982 s) : 0, 8982071
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.173 s 110.658 ms (10.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.166 s 103.818 ms (9.8%)
Agent iast_TELEMETRY_OFF 1.159 s 97.261 ms (9.2%)
Total tracing 8.571 s -
Total iast 8.994 s 423.662 ms (4.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.985 s 414.534 ms (4.8%)
Total iast_TELEMETRY_OFF 8.99 s 419.825 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.172 s 113.996 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.168 s 110.336 ms (10.4%)
Agent iast_TELEMETRY_OFF 1.164 s 106.352 ms (10.1%)
Total tracing 8.54 s -
Total iast 9.014 s 474.644 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 8.964 s 424.664 ms (5.0%)
Total iast_TELEMETRY_OFF 8.982 s 442.398 ms (5.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (655.576 ms) : 0, 655576
BytebuddyAgent [candidate] (652.705 ms) : 0, 652705
GlobalTracer [baseline] (314.238 ms) : 0, 314238
GlobalTracer [candidate] (312.736 ms) : 0, 312736
AppSec [baseline] (49.542 ms) : 0, 49542
AppSec [candidate] (49.621 ms) : 0, 49621
Remote Config [baseline] (664.645 µs) : 0, 665
Remote Config [candidate] (662.121 µs) : 0, 662
Telemetry [baseline] (7.587 ms) : 0, 7587
Telemetry [candidate] (7.609 ms) : 0, 7609
section iast
BytebuddyAgent [baseline] (784.301 ms) : 0, 784301
BytebuddyAgent [candidate] (781.997 ms) : 0, 781997
GlobalTracer [baseline] (294.295 ms) : 0, 294295
GlobalTracer [candidate] (293.617 ms) : 0, 293617
AppSec [baseline] (47.738 ms) : 0, 47738
AppSec [candidate] (49.608 ms) : 0, 49608
IAST [baseline] (25.623 ms) : 0, 25623
IAST [candidate] (24.955 ms) : 0, 24955
Remote Config [baseline] (586.926 µs) : 0, 587
Remote Config [candidate] (608.256 µs) : 0, 608
Telemetry [baseline] (6.917 ms) : 0, 6917
Telemetry [candidate] (7.738 ms) : 0, 7738
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (777.35 ms) : 0, 777350
BytebuddyAgent [candidate] (779.107 ms) : 0, 779107
GlobalTracer [baseline] (292.384 ms) : 0, 292384
GlobalTracer [candidate] (293.353 ms) : 0, 293353
AppSec [baseline] (47.47 ms) : 0, 47470
AppSec [candidate] (48.515 ms) : 0, 48515
IAST [baseline] (27.302 ms) : 0, 27302
IAST [candidate] (25.665 ms) : 0, 25665
Remote Config [baseline] (608.246 µs) : 0, 608
Remote Config [candidate] (605.802 µs) : 0, 606
Telemetry [baseline] (7.579 ms) : 0, 7579
Telemetry [candidate] (7.578 ms) : 0, 7578
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (774.194 ms) : 0, 774194
BytebuddyAgent [candidate] (776.049 ms) : 0, 776049
GlobalTracer [baseline] (292.281 ms) : 0, 292281
GlobalTracer [candidate] (292.569 ms) : 0, 292569
AppSec [baseline] (48.69 ms) : 0, 48690
AppSec [candidate] (47.886 ms) : 0, 47886
IAST [baseline] (22.754 ms) : 0, 22754
IAST [candidate] (24.66 ms) : 0, 24660
Remote Config [baseline] (595.846 µs) : 0, 596
Remote Config [candidate] (604.042 µs) : 0, 604
Telemetry [baseline] (7.61 ms) : 0, 7610
Telemetry [candidate] (9.126 ms) : 0, 9126
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.056 s) : 0, 1055563
Total [baseline] (10.286 s) : 0, 10285842
Agent [candidate] (1.056 s) : 0, 1055678
Total [candidate] (10.35 s) : 0, 10350328
section appsec
Agent [baseline] (1.175 s) : 0, 1174639
Total [baseline] (10.454 s) : 0, 10454498
Agent [candidate] (1.175 s) : 0, 1174830
Total [candidate] (10.446 s) : 0, 10446460
section iast
Agent [baseline] (1.168 s) : 0, 1168073
Total [baseline] (10.79 s) : 0, 10789538
Agent [candidate] (1.164 s) : 0, 1163968
Total [candidate] (10.694 s) : 0, 10693967
section profiling
Agent [baseline] (1.257 s) : 0, 1257179
Total [baseline] (10.556 s) : 0, 10556122
Agent [candidate] (1.256 s) : 0, 1256246
Total [candidate] (10.614 s) : 0, 10614022
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent appsec 1.175 s 119.076 ms (11.3%)
Agent iast 1.168 s 112.51 ms (10.7%)
Agent profiling 1.257 s 201.616 ms (19.1%)
Total tracing 10.286 s -
Total appsec 10.454 s 168.655 ms (1.6%)
Total iast 10.79 s 503.696 ms (4.9%)
Total profiling 10.556 s 270.279 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent appsec 1.175 s 119.152 ms (11.3%)
Agent iast 1.164 s 108.289 ms (10.3%)
Agent profiling 1.256 s 200.568 ms (19.0%)
Total tracing 10.35 s -
Total appsec 10.446 s 96.132 ms (0.9%)
Total iast 10.694 s 343.639 ms (3.3%)
Total profiling 10.614 s 263.694 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (651.356 ms) : 0, 651356
BytebuddyAgent [candidate] (651.757 ms) : 0, 651757
GlobalTracer [baseline] (312.042 ms) : 0, 312042
GlobalTracer [candidate] (312.294 ms) : 0, 312294
AppSec [baseline] (49.538 ms) : 0, 49538
AppSec [candidate] (49.132 ms) : 0, 49132
Remote Config [baseline] (659.581 µs) : 0, 660
Remote Config [candidate] (656.473 µs) : 0, 656
Telemetry [baseline] (7.554 ms) : 0, 7554
Telemetry [candidate] (7.486 ms) : 0, 7486
section appsec
BytebuddyAgent [baseline] (673.972 ms) : 0, 673972
BytebuddyAgent [candidate] (674.039 ms) : 0, 674039
GlobalTracer [baseline] (296.096 ms) : 0, 296096
GlobalTracer [candidate] (295.798 ms) : 0, 295798
AppSec [baseline] (152.054 ms) : 0, 152054
AppSec [candidate] (151.89 ms) : 0, 151890
Remote Config [baseline] (618.806 µs) : 0, 619
Remote Config [candidate] (615.65 µs) : 0, 616
Telemetry [baseline] (8.093 ms) : 0, 8093
Telemetry [candidate] (8.747 ms) : 0, 8747
IAST [baseline] (18.652 ms) : 0, 18652
IAST [candidate] (18.696 ms) : 0, 18696
section iast
BytebuddyAgent [baseline] (782.055 ms) : 0, 782055
BytebuddyAgent [candidate] (778.021 ms) : 0, 778021
GlobalTracer [baseline] (292.171 ms) : 0, 292171
GlobalTracer [candidate] (292.137 ms) : 0, 292137
AppSec [baseline] (49.699 ms) : 0, 49699
AppSec [candidate] (51.101 ms) : 0, 51101
Remote Config [baseline] (580.142 µs) : 0, 580
Remote Config [candidate] (581.129 µs) : 0, 581
Telemetry [baseline] (6.765 ms) : 0, 6765
Telemetry [candidate] (6.813 ms) : 0, 6813
IAST [baseline] (23.48 ms) : 0, 23480
IAST [candidate] (22.065 ms) : 0, 22065
section profiling
BytebuddyAgent [baseline] (661.797 ms) : 0, 661797
BytebuddyAgent [candidate] (661.2 ms) : 0, 661200
GlobalTracer [baseline] (385.447 ms) : 0, 385447
GlobalTracer [candidate] (384.988 ms) : 0, 384988
AppSec [baseline] (50.577 ms) : 0, 50577
AppSec [candidate] (50.109 ms) : 0, 50109
Remote Config [baseline] (794.489 µs) : 0, 794
Remote Config [candidate] (817.542 µs) : 0, 818
Telemetry [baseline] (7.435 ms) : 0, 7435
Telemetry [candidate] (7.456 ms) : 0, 7456
ProfilingAgent [baseline] (94.529 ms) : 0, 94529
ProfilingAgent [candidate] (95.176 ms) : 0, 95176
Profiling [baseline] (94.554 ms) : 0, 94554
Profiling [candidate] (95.201 ms) : 0, 95201
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-03T15:35:36 2024-06-03T15:42:25
git_branch master smola/system-tests-20240530
git_commit_date 1717427688 1717427714
git_commit_sha bf9fd09 03ad506
release_version 1.35.0-SNAPSHOT~bf9fd09c02 1.35.0-SNAPSHOT~03ad50604c
start_time 2024-06-03T15:35:22 2024-06-03T15:42:12
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1717429690 1717429690
ci_job_id 530556732 530556732
ci_pipeline_id 35707484 35707484
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 15 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-114.000µs; -60.490µs] or [-7.303%; -3.875%]		 unstable
[-335.714op/s; +809.788op/s] or [-11.330%; +27.330%]		 1.474ms 3200.000op/s 1.561ms 2962.963op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.336 ms) : 1317, 1356
.   : milestone, 1336,
appsec (1.726 ms) : 1701, 1752
.   : milestone, 1726,
appsec_no_iast (1.724 ms) : 1700, 1748
.   : milestone, 1724,
iast (1.489 ms) : 1467, 1512
.   : milestone, 1489,
profiling (1.561 ms) : 1536, 1586
.   : milestone, 1561,
tracing (1.467 ms) : 1442, 1492
.   : milestone, 1467,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.728 ms) : 1705, 1752
.   : milestone, 1728,
appsec_no_iast (1.723 ms) : 1699, 1748
.   : milestone, 1723,
iast (1.471 ms) : 1448, 1494
.   : milestone, 1471,
profiling (1.474 ms) : 1449, 1499
.   : milestone, 1474,
tracing (1.481 ms) : 1457, 1505
.   : milestone, 1481,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.336 ms [1.317 ms, 1.356 ms] -
appsec 1.726 ms [1.701 ms, 1.752 ms] 390.435 µs (29.2%)
appsec_no_iast 1.724 ms [1.7 ms, 1.748 ms] 388.083 µs (29.0%)
iast 1.489 ms [1.467 ms, 1.512 ms] 153.064 µs (11.5%)
profiling 1.561 ms [1.536 ms, 1.586 ms] 224.943 µs (16.8%)
tracing 1.467 ms [1.442 ms, 1.492 ms] 130.613 µs (9.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.728 ms [1.705 ms, 1.752 ms] 383.588 µs (28.5%)
appsec_no_iast 1.723 ms [1.699 ms, 1.748 ms] 378.426 µs (28.1%)
iast 1.471 ms [1.448 ms, 1.494 ms] 125.96 µs (9.4%)
profiling 1.474 ms [1.449 ms, 1.499 ms] 128.926 µs (9.6%)
tracing 1.481 ms [1.457 ms, 1.505 ms] 136.334 µs (10.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.395 µs) : 349, 392
.   : milestone, 370,
iast (479.094 µs) : 457, 501
.   : milestone, 479,
iast_FULL (548.441 µs) : 527, 570
.   : milestone, 548,
iast_GLOBAL (500.201 µs) : 479, 521
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (483.27 µs) : 462, 505
.   : milestone, 483,
iast_INACTIVE (449.836 µs) : 429, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (463.173 µs) : 442, 484
.   : milestone, 463,
tracing (439.981 µs) : 419, 461
.   : milestone, 440,
section candidate
no_agent (369.646 µs) : 349, 390
.   : milestone, 370,
iast (480.874 µs) : 459, 503
.   : milestone, 481,
iast_FULL (549.01 µs) : 528, 570
.   : milestone, 549,
iast_GLOBAL (501.863 µs) : 481, 523
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (479.592 µs) : 458, 501
.   : milestone, 480,
iast_INACTIVE (447.199 µs) : 427, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (467.205 µs) : 446, 488
.   : milestone, 467,
tracing (438.134 µs) : 418, 458
.   : milestone, 438,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.395 µs [348.76 µs, 392.03 µs] -
iast 479.094 µs [457.478 µs, 500.711 µs] 108.699 µs (29.3%)
iast_FULL 548.441 µs [527.302 µs, 569.579 µs] 178.045 µs (48.1%)
iast_GLOBAL 500.201 µs [479.281 µs, 521.12 µs] 129.806 µs (35.0%)
iast_HARDCODED_SECRET_DISABLED 483.27 µs [461.926 µs, 504.613 µs] 112.874 µs (30.5%)
iast_INACTIVE 449.836 µs [428.788 µs, 470.883 µs] 79.441 µs (21.4%)
iast_TELEMETRY_OFF 463.173 µs [442.481 µs, 483.865 µs] 92.778 µs (25.0%)
tracing 439.981 µs [419.21 µs, 460.752 µs] 69.586 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.646 µs [349.306 µs, 389.986 µs] -
iast 480.874 µs [459.039 µs, 502.71 µs] 111.228 µs (30.1%)
iast_FULL 549.01 µs [528.016 µs, 570.004 µs] 179.365 µs (48.5%)
iast_GLOBAL 501.863 µs [480.858 µs, 522.868 µs] 132.217 µs (35.8%)
iast_HARDCODED_SECRET_DISABLED 479.592 µs [458.25 µs, 500.933 µs] 109.946 µs (29.7%)
iast_INACTIVE 447.199 µs [426.614 µs, 467.784 µs] 77.553 µs (21.0%)
iast_TELEMETRY_OFF 467.205 µs [446.184 µs, 488.227 µs] 97.559 µs (26.4%)
tracing 438.134 µs [418.03 µs, 458.239 µs] 68.488 µs (18.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-20240530
git_commit_date 1717427688 1717427714
git_commit_sha bf9fd09 03ad506
release_version 1.35.0-SNAPSHOT~bf9fd09c02 1.35.0-SNAPSHOT~03ad50604c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1717430186 1717430186
ci_job_id 530556736 530556736
ci_pipeline_id 35707484 35707484
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1470
.   : milestone, 1458,
appsec (2.201 ms) : 2166, 2235
.   : milestone, 2201,
iast (1.969 ms) : 1928, 2011
.   : milestone, 1969,
iast_GLOBAL (2.014 ms) : 1972, 2055
.   : milestone, 2014,
profiling (2.31 ms) : 2132, 2488
.   : milestone, 2310,
tracing (1.829 ms) : 1797, 1861
.   : milestone, 1829,
section candidate
no_agent (1.463 ms) : 1452, 1475
.   : milestone, 1463,
appsec (2.204 ms) : 2169, 2238
.   : milestone, 2204,
iast (1.965 ms) : 1924, 2006
.   : milestone, 1965,
iast_GLOBAL (1.994 ms) : 1954, 2035
.   : milestone, 1994,
profiling (1.839 ms) : 1807, 1872
.   : milestone, 1839,
tracing (1.823 ms) : 1791, 1855
.   : milestone, 1823,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.458 ms [1.447 ms, 1.47 ms] -
appsec 2.201 ms [2.166 ms, 2.235 ms] 742.453 µs (50.9%)
iast 1.969 ms [1.928 ms, 2.011 ms] 511.069 µs (35.0%)
iast_GLOBAL 2.014 ms [1.972 ms, 2.055 ms] 555.22 µs (38.1%)
profiling 2.31 ms [2.132 ms, 2.488 ms] 851.386 µs (58.4%)
tracing 1.829 ms [1.797 ms, 1.861 ms] 370.512 µs (25.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.475 ms] -
appsec 2.204 ms [2.169 ms, 2.238 ms] 740.41 µs (50.6%)
iast 1.965 ms [1.924 ms, 2.006 ms] 501.169 µs (34.2%)
iast_GLOBAL 1.994 ms [1.954 ms, 2.035 ms] 531.028 µs (36.3%)
profiling 1.839 ms [1.807 ms, 1.872 ms] 375.629 µs (25.7%)
tracing 1.823 ms [1.791 ms, 1.855 ms] 359.534 µs (24.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~03ad50604c, baseline=1.35.0-SNAPSHOT~bf9fd09c02
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.746 s) : 14746000, 14746000
.   : milestone, 14746000,
appsec (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,
iast (18.745 s) : 18745000, 18745000
.   : milestone, 18745000,
iast_GLOBAL (17.948 s) : 17948000, 17948000
.   : milestone, 17948000,
profiling (15.005 s) : 15005000, 15005000
.   : milestone, 15005000,
tracing (15.065 s) : 15065000, 15065000
.   : milestone, 15065000,
section candidate
no_agent (14.833 s) : 14833000, 14833000
.   : milestone, 14833000,
appsec (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
iast (18.764 s) : 18764000, 18764000
.   : milestone, 18764000,
iast_GLOBAL (17.688 s) : 17688000, 17688000
.   : milestone, 17688000,
profiling (15.289 s) : 15289000, 15289000
.   : milestone, 15289000,
tracing (14.929 s) : 14929000, 14929000
.   : milestone, 14929000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.746 s [14.746 s, 14.746 s] -
appsec 14.942 s [14.942 s, 14.942 s] 196.0 ms (1.3%)
iast 18.745 s [18.745 s, 18.745 s] 3.999 s (27.1%)
iast_GLOBAL 17.948 s [17.948 s, 17.948 s] 3.202 s (21.7%)
profiling 15.005 s [15.005 s, 15.005 s] 259.0 ms (1.8%)
tracing 15.065 s [15.065 s, 15.065 s] 319.0 ms (2.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.833 s [14.833 s, 14.833 s] -
appsec 15.022 s [15.022 s, 15.022 s] 189.0 ms (1.3%)
iast 18.764 s [18.764 s, 18.764 s] 3.931 s (26.5%)
iast_GLOBAL 17.688 s [17.688 s, 17.688 s] 2.855 s (19.2%)
profiling 15.289 s [15.289 s, 15.289 s] 456.0 ms (3.1%)
tracing 14.929 s [14.929 s, 14.929 s] 96.0 ms (0.6%)

@smola smola changed the title Update system-tests to ad338b7d065a2f698e355e9e87891d70247780aa Update system-tests to 4d979bdcab25754d38a8d190525bef87dad66a6e May 31, 2024
@smola smola force-pushed the smola/system-tests-20240530 branch from ef51f7d to 4a3fa79 Compare May 31, 2024 12:51
@smola smola merged commit 70f9058 into master Jun 4, 2024
80 of 81 checks passed
@smola smola deleted the smola/system-tests-20240530 branch June 4, 2024 07:21
@github-actions github-actions bot added this to the 1.35.0 milestone Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dougqh dougqh dougqh approved these changes

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.35.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@smola @dougqh