Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to AppSec rules v1.11.0 #6754

Merged
merged 1 commit into from Mar 5, 2024
Merged

Upgrade to AppSec rules v1.11.0 #6754

merged 1 commit into from Mar 5, 2024

Conversation

smola
Copy link
Member

@smola smola commented Feb 28, 2024 •
edited

What Does This Do

Upgrade to the latest Application Security rules.

Additional Notes

The new rules produce a small fixed increase in startup time for users with DD_APPSEC_ENABLED=true. This is 2-4ms in our benchmarks. We will try to improve this in future versions.

@smola smola added the comp: asm waf Application Security Management (WAF) label Feb 28, 2024
@pr-commenter
Copy link

pr-commenter bot commented Feb 28, 2024 •
edited

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/appsec-rules-1.11.0
git_commit_date 1709558898 1709631287
git_commit_sha aeb7876 edc0cec
release_version 1.31.0~aeb7876f22 1.32.0-SNAPSHOT~edc0ceceed
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709634472 1709634472
ci_job_id 450680068 450680068
ci_pipeline_id 29490470 29490470
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 49 metrics, 13 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:appsec:AppSec worse
[+4.028ms; +5.285ms] or [+2.663%; +3.493%]		 155.958ms 151.301ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1082529
Total [baseline] (9.158 s) : 0, 9157910
Agent [candidate] (1.093 s) : 0, 1092590
Total [candidate] (9.201 s) : 0, 9200707
section appsec
Agent [baseline] (1.198 s) : 0, 1198446
Total [baseline] (9.337 s) : 0, 9336610
Agent [candidate] (1.203 s) : 0, 1203114
Total [candidate] (9.297 s) : 0, 9297409
section iast
Agent [baseline] (1.204 s) : 0, 1204246
Total [baseline] (9.299 s) : 0, 9299252
Agent [candidate] (1.207 s) : 0, 1206566
Total [candidate] (9.29 s) : 0, 9289924
section profiling
Agent [baseline] (1.272 s) : 0, 1272260
Total [baseline] (9.333 s) : 0, 9333110
Agent [candidate] (1.284 s) : 0, 1283888
Total [candidate] (9.354 s) : 0, 9354089
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.198 s 115.918 ms (10.7%)
Agent iast 1.204 s 121.717 ms (11.2%)
Agent profiling 1.272 s 189.731 ms (17.5%)
Total tracing 9.158 s -
Total appsec 9.337 s 178.7 ms (2.0%)
Total iast 9.299 s 141.342 ms (1.5%)
Total profiling 9.333 s 175.199 ms (1.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent appsec 1.203 s 110.525 ms (10.1%)
Agent iast 1.207 s 113.976 ms (10.4%)
Agent profiling 1.284 s 191.298 ms (17.5%)
Total tracing 9.201 s -
Total appsec 9.297 s 96.702 ms (1.1%)
Total iast 9.29 s 89.217 ms (1.0%)
Total profiling 9.354 s 153.382 ms (1.7%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (696.688 ms) : 0, 696688
BytebuddyAgent [candidate] (703.408 ms) : 0, 703408
GlobalTracer [baseline] (291.378 ms) : 0, 291378
GlobalTracer [candidate] (294.886 ms) : 0, 294886
AppSec [baseline] (51.554 ms) : 0, 51554
AppSec [candidate] (51.04 ms) : 0, 51040
Remote Config [baseline] (746.106 µs) : 0, 746
Remote Config [candidate] (724.879 µs) : 0, 725
Telemetry [baseline] (7.77 ms) : 0, 7770
Telemetry [candidate] (7.839 ms) : 0, 7839
section appsec
BytebuddyAgent [baseline] (696.305 ms) : 0, 696305
BytebuddyAgent [candidate] (696.108 ms) : 0, 696108
GlobalTracer [baseline] (291.259 ms) : 0, 291259
GlobalTracer [candidate] (291.431 ms) : 0, 291431
AppSec [baseline] (151.301 ms) : 0, 151301
AppSec [candidate] (155.958 ms) : 0, 155958
Remote Config [baseline] (607.147 µs) : 0, 607
Remote Config [candidate] (606.682 µs) : 0, 607
Telemetry [baseline] (6.869 ms) : 0, 6869
Telemetry [candidate] (6.867 ms) : 0, 6867
IAST [baseline] (17.823 ms) : 0, 17823
IAST [candidate] (17.774 ms) : 0, 17774
section iast
BytebuddyAgent [baseline] (799.012 ms) : 0, 799012
BytebuddyAgent [candidate] (799.995 ms) : 0, 799995
GlobalTracer [baseline] (287.546 ms) : 0, 287546
GlobalTracer [candidate] (288.766 ms) : 0, 288766
AppSec [baseline] (54.284 ms) : 0, 54284
AppSec [candidate] (50.843 ms) : 0, 50843
Remote Config [baseline] (587.088 µs) : 0, 587
Remote Config [candidate] (587.507 µs) : 0, 588
Telemetry [baseline] (6.678 ms) : 0, 6678
Telemetry [candidate] (6.689 ms) : 0, 6689
IAST [baseline] (22.0 ms) : 0, 22000
IAST [candidate] (25.341 ms) : 0, 25341
section profiling
BytebuddyAgent [baseline] (688.227 ms) : 0, 688227
BytebuddyAgent [candidate] (694.378 ms) : 0, 694378
GlobalTracer [baseline] (373.962 ms) : 0, 373962
GlobalTracer [candidate] (377.484 ms) : 0, 377484
AppSec [baseline] (52.691 ms) : 0, 52691
AppSec [candidate] (53.259 ms) : 0, 53259
Remote Config [baseline] (777.59 µs) : 0, 778
Remote Config [candidate] (798.42 µs) : 0, 798
Telemetry [baseline] (7.444 ms) : 0, 7444
Telemetry [candidate] (7.425 ms) : 0, 7425
ProfilingAgent [baseline] (93.133 ms) : 0, 93133
ProfilingAgent [candidate] (93.993 ms) : 0, 93993
Profiling [baseline] (93.156 ms) : 0, 93156
Profiling [candidate] (94.017 ms) : 0, 94017
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.099 s) : 0, 1099344
Total [baseline] (8.637 s) : 0, 8636555
Agent [candidate] (1.083 s) : 0, 1083009
Total [candidate] (8.57 s) : 0, 8570468
section iast
Agent [baseline] (1.222 s) : 0, 1221527
Total [baseline] (9.076 s) : 0, 9076255
Agent [candidate] (1.209 s) : 0, 1209001
Total [candidate] (9.079 s) : 0, 9078707
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.202 s) : 0, 1201766
Total [baseline] (9.036 s) : 0, 9036197
Agent [candidate] (1.209 s) : 0, 1208951
Total [candidate] (9.028 s) : 0, 9028106
section iast_TELEMETRY_OFF
Agent [baseline] (1.208 s) : 0, 1208302
Total [baseline] (9.063 s) : 0, 9062667
Agent [candidate] (1.199 s) : 0, 1198681
Total [candidate] (9.03 s) : 0, 9029713
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.099 s -
Agent iast 1.222 s 122.183 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.202 s 102.423 ms (9.3%)
Agent iast_TELEMETRY_OFF 1.208 s 108.958 ms (9.9%)
Total tracing 8.637 s -
Total iast 9.076 s 439.701 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.036 s 399.642 ms (4.6%)
Total iast_TELEMETRY_OFF 9.063 s 426.113 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent iast 1.209 s 125.992 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.209 s 125.942 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.199 s 115.673 ms (10.7%)
Total tracing 8.57 s -
Total iast 9.079 s 508.238 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.028 s 457.638 ms (5.3%)
Total iast_TELEMETRY_OFF 9.03 s 459.244 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (708.157 ms) : 0, 708157
BytebuddyAgent [candidate] (697.103 ms) : 0, 697103
GlobalTracer [baseline] (295.734 ms) : 0, 295734
GlobalTracer [candidate] (292.258 ms) : 0, 292258
AppSec [baseline] (51.993 ms) : 0, 51993
AppSec [candidate] (50.985 ms) : 0, 50985
Remote Config [baseline] (736.312 µs) : 0, 736
Remote Config [candidate] (737.98 µs) : 0, 738
Telemetry [baseline] (7.79 ms) : 0, 7790
Telemetry [candidate] (7.592 ms) : 0, 7592
section iast
BytebuddyAgent [baseline] (810.846 ms) : 0, 810846
BytebuddyAgent [candidate] (801.361 ms) : 0, 801361
GlobalTracer [baseline] (291.585 ms) : 0, 291585
GlobalTracer [candidate] (288.865 ms) : 0, 288865
AppSec [baseline] (52.477 ms) : 0, 52477
AppSec [candidate] (52.329 ms) : 0, 52329
IAST [baseline] (24.75 ms) : 0, 24750
IAST [candidate] (24.047 ms) : 0, 24047
Remote Config [baseline] (584.784 µs) : 0, 585
Remote Config [candidate] (591.18 µs) : 0, 591
Telemetry [baseline] (6.715 ms) : 0, 6715
Telemetry [candidate] (7.449 ms) : 0, 7449
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (796.986 ms) : 0, 796986
BytebuddyAgent [candidate] (802.206 ms) : 0, 802206
GlobalTracer [baseline] (287.524 ms) : 0, 287524
GlobalTracer [candidate] (288.781 ms) : 0, 288781
AppSec [baseline] (50.888 ms) : 0, 50888
AppSec [candidate] (51.258 ms) : 0, 51258
IAST [baseline] (23.978 ms) : 0, 23978
IAST [candidate] (24.413 ms) : 0, 24413
Remote Config [baseline] (592.976 µs) : 0, 593
Remote Config [candidate] (578.095 µs) : 0, 578
Telemetry [baseline] (7.516 ms) : 0, 7516
Telemetry [candidate] (7.399 ms) : 0, 7399
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.48 ms) : 0, 799480
BytebuddyAgent [candidate] (792.887 ms) : 0, 792887
GlobalTracer [baseline] (291.278 ms) : 0, 291278
GlobalTracer [candidate] (289.849 ms) : 0, 289849
AppSec [baseline] (54.043 ms) : 0, 54043
AppSec [candidate] (52.813 ms) : 0, 52813
IAST [baseline] (20.174 ms) : 0, 20174
IAST [candidate] (21.623 ms) : 0, 21623
Remote Config [baseline] (591.682 µs) : 0, 592
Remote Config [candidate] (587.678 µs) : 0, 588
Telemetry [baseline] (8.172 ms) : 0, 8172
Telemetry [candidate] (6.545 ms) : 0, 6545

Load

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.338 ms) : 1319, 1357
.   : milestone, 1338,
appsec (1.771 ms) : 1748, 1794
.   : milestone, 1771,
iast (1.512 ms) : 1489, 1536
.   : milestone, 1512,
profiling (1.519 ms) : 1495, 1542
.   : milestone, 1519,
tracing (1.497 ms) : 1474, 1520
.   : milestone, 1497,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.769 ms) : 1745, 1793
.   : milestone, 1769,
iast (1.515 ms) : 1492, 1538
.   : milestone, 1515,
profiling (1.517 ms) : 1493, 1541
.   : milestone, 1517,
tracing (1.499 ms) : 1475, 1523
.   : milestone, 1499,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.319 ms, 1.357 ms] -
appsec 1.771 ms [1.748 ms, 1.794 ms] 433.109 µs (32.4%)
iast 1.512 ms [1.489 ms, 1.536 ms] 173.95 µs (13.0%)
profiling 1.519 ms [1.495 ms, 1.542 ms] 180.358 µs (13.5%)
tracing 1.497 ms [1.474 ms, 1.52 ms] 158.643 µs (11.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.769 ms [1.745 ms, 1.793 ms] 414.237 µs (30.6%)
iast 1.515 ms [1.492 ms, 1.538 ms] 159.982 µs (11.8%)
profiling 1.517 ms [1.493 ms, 1.541 ms] 161.985 µs (12.0%)
tracing 1.499 ms [1.475 ms, 1.523 ms] 144.069 µs (10.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~edc0ceceed, baseline=1.31.0~aeb7876f22
    dateFormat X
    axisFormat %s
section baseline
no_agent (359.883 µs) : 340, 380
.   : milestone, 360,
iast (466.826 µs) : 447, 487
.   : milestone, 467,
iast_FULL (529.777 µs) : 509, 550
.   : milestone, 530,
iast_GLOBAL (484.918 µs) : 465, 505
.   : milestone, 485,
iast_HARDCODED_SECRET_DISABLED (470.301 µs) : 449, 491
.   : milestone, 470,
iast_INACTIVE (449.107 µs) : 427, 471
.   : milestone, 449,
iast_TELEMETRY_OFF (461.532 µs) : 441, 482
.   : milestone, 462,
tracing (438.866 µs) : 418, 459
.   : milestone, 439,
section candidate
no_agent (359.666 µs) : 339, 380
.   : milestone, 360,
iast (467.508 µs) : 447, 488
.   : milestone, 468,
iast_FULL (533.37 µs) : 513, 554
.   : milestone, 533,
iast_GLOBAL (492.695 µs) : 471, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (475.248 µs) : 455, 496
.   : milestone, 475,
iast_INACTIVE (439.997 µs) : 420, 460
.   : milestone, 440,
iast_TELEMETRY_OFF (470.064 µs) : 449, 491
.   : milestone, 470,
tracing (444.516 µs) : 423, 466
.   : milestone, 445,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.883 µs [340.184 µs, 379.582 µs] -
iast 466.826 µs [446.594 µs, 487.058 µs] 106.943 µs (29.7%)
iast_FULL 529.777 µs [509.347 µs, 550.206 µs] 169.894 µs (47.2%)
iast_GLOBAL 484.918 µs [464.657 µs, 505.178 µs] 125.035 µs (34.7%)
iast_HARDCODED_SECRET_DISABLED 470.301 µs [449.463 µs, 491.139 µs] 110.418 µs (30.7%)
iast_INACTIVE 449.107 µs [426.779 µs, 471.435 µs] 89.224 µs (24.8%)
iast_TELEMETRY_OFF 461.532 µs [440.841 µs, 482.222 µs] 101.649 µs (28.2%)
tracing 438.866 µs [418.481 µs, 459.252 µs] 78.983 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.666 µs [339.164 µs, 380.168 µs] -
iast 467.508 µs [446.633 µs, 488.382 µs] 107.842 µs (30.0%)
iast_FULL 533.37 µs [512.873 µs, 553.866 µs] 173.704 µs (48.3%)
iast_GLOBAL 492.695 µs [471.373 µs, 514.018 µs] 133.03 µs (37.0%)
iast_HARDCODED_SECRET_DISABLED 475.248 µs [454.961 µs, 495.534 µs] 115.582 µs (32.1%)
iast_INACTIVE 439.997 µs [419.611 µs, 460.383 µs] 80.331 µs (22.3%)
iast_TELEMETRY_OFF 470.064 µs [449.273 µs, 490.856 µs] 110.399 µs (30.7%)
tracing 444.516 µs [423.392 µs, 465.639 µs] 84.85 µs (23.6%)

@smola smola marked this pull request as ready for review February 28, 2024 17:35
@smola smola requested a review from a team as a code owner February 28, 2024 17:35
@smola smola enabled auto-merge (squash) February 28, 2024 17:46
@smola smola disabled auto-merge February 28, 2024 17:47
@smola smola force-pushed the smola/appsec-rules-1.11.0 branch from 23518b2 to 25fcf3b Compare March 1, 2024 07:50
@smola smola force-pushed the smola/appsec-rules-1.11.0 branch from 6c3dbe0 to edc0cec Compare March 5, 2024 09:35
@smola smola merged commit 7e58261 into master Mar 5, 2024
79 checks passed
@smola smola deleted the smola/appsec-rules-1.11.0 branch March 5, 2024 11:59
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 5, 2024
@smola smola mentioned this pull request Mar 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

@jandro996 jandro996 jandro996 approved these changes

Assignees
No one assigned
Labels
comp: asm waf Application Security Management (WAF)
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@smola @manuel-alvarez-alvarez @jandro996