Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Minify AppSec rules #6773

Merged
merged 1 commit into from Mar 11, 2024
Merged

Minify AppSec rules #6773

merged 1 commit into from Mar 11, 2024

Conversation

smola
Copy link
Member

@smola smola commented Mar 5, 2024 •
edited

What Does This Do

Minify AppSec rules.

Motivation

Reduce JAR size and load time.

Additional Notes

This partially offsets the startup time regression introduced by #6754 when DD_APPSEC_ENABLED=true.

@smola smola added the comp: asm waf Application Security Management (WAF) label Mar 5, 2024
@smola smola requested a review from a team as a code owner March 5, 2024 12:01
@pr-commenter
Copy link

pr-commenter bot commented Mar 5, 2024 •
edited

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/minify-appsec-rules
git_commit_date 1709909278 1710143400
git_commit_sha e9c489f a3bbdb9
release_version 1.32.0-SNAPSHOT~e9c489fd8c 1.32.0-SNAPSHOT~a3bbdb9ded
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710146587 1710146587
ci_job_id 455977785 455977785
ci_pipeline_id 29873437 29873437
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 15 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:tracing:AppSec better
[-1.899ms; -1.320ms] or [-3.740%; -2.600%]		 49.158ms 50.768ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1083853
Total [baseline] (9.137 s) : 0, 9136546
Agent [candidate] (1.082 s) : 0, 1082182
Total [candidate] (9.2 s) : 0, 9200442
section appsec
Agent [baseline] (1.212 s) : 0, 1212190
Total [baseline] (9.319 s) : 0, 9319461
Agent [candidate] (1.213 s) : 0, 1213465
Total [candidate] (9.422 s) : 0, 9422027
section iast
Agent [baseline] (1.217 s) : 0, 1217122
Total [baseline] (9.371 s) : 0, 9371297
Agent [candidate] (1.207 s) : 0, 1206504
Total [candidate] (9.383 s) : 0, 9383103
section profiling
Agent [baseline] (1.279 s) : 0, 1278519
Total [baseline] (9.375 s) : 0, 9374574
Agent [candidate] (1.29 s) : 0, 1289931
Total [candidate] (9.424 s) : 0, 9424361
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent appsec 1.212 s 128.337 ms (11.8%)
Agent iast 1.217 s 133.269 ms (12.3%)
Agent profiling 1.279 s 194.666 ms (18.0%)
Total tracing 9.137 s -
Total appsec 9.319 s 182.915 ms (2.0%)
Total iast 9.371 s 234.751 ms (2.6%)
Total profiling 9.375 s 238.028 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent appsec 1.213 s 131.283 ms (12.1%)
Agent iast 1.207 s 124.323 ms (11.5%)
Agent profiling 1.29 s 207.75 ms (19.2%)
Total tracing 9.2 s -
Total appsec 9.422 s 221.586 ms (2.4%)
Total iast 9.383 s 182.662 ms (2.0%)
Total profiling 9.424 s 223.92 ms (2.4%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.272 ms) : 0, 698272
BytebuddyAgent [candidate] (697.484 ms) : 0, 697484
GlobalTracer [baseline] (291.876 ms) : 0, 291876
GlobalTracer [candidate] (292.591 ms) : 0, 292591
AppSec [baseline] (50.768 ms) : 0, 50768
AppSec [candidate] (49.158 ms) : 0, 49158
Remote Config [baseline] (733.955 µs) : 0, 734
Remote Config [candidate] (722.76 µs) : 0, 723
Telemetry [baseline] (7.784 ms) : 0, 7784
Telemetry [candidate] (7.914 ms) : 0, 7914
section appsec
BytebuddyAgent [baseline] (702.473 ms) : 0, 702473
BytebuddyAgent [candidate] (703.862 ms) : 0, 703862
GlobalTracer [baseline] (293.025 ms) : 0, 293025
GlobalTracer [candidate] (294.788 ms) : 0, 294788
AppSec [baseline] (156.434 ms) : 0, 156434
AppSec [candidate] (154.642 ms) : 0, 154642
IAST [baseline] (18.011 ms) : 0, 18011
IAST [candidate] (17.955 ms) : 0, 17955
Remote Config [baseline] (616.993 µs) : 0, 617
Remote Config [candidate] (615.311 µs) : 0, 615
Telemetry [baseline] (6.983 ms) : 0, 6983
Telemetry [candidate] (6.945 ms) : 0, 6945
section iast
BytebuddyAgent [baseline] (808.255 ms) : 0, 808255
BytebuddyAgent [candidate] (801.996 ms) : 0, 801996
GlobalTracer [baseline] (290.569 ms) : 0, 290569
GlobalTracer [candidate] (289.877 ms) : 0, 289877
AppSec [baseline] (54.009 ms) : 0, 54009
AppSec [candidate] (48.609 ms) : 0, 48609
IAST [baseline] (22.323 ms) : 0, 22323
IAST [candidate] (23.779 ms) : 0, 23779
Remote Config [baseline] (590.916 µs) : 0, 591
Remote Config [candidate] (574.485 µs) : 0, 574
Telemetry [baseline] (6.684 ms) : 0, 6684
Telemetry [candidate] (7.356 ms) : 0, 7356
section profiling
BytebuddyAgent [baseline] (690.58 ms) : 0, 690580
BytebuddyAgent [candidate] (696.876 ms) : 0, 696876
GlobalTracer [baseline] (375.719 ms) : 0, 375719
GlobalTracer [candidate] (381.042 ms) : 0, 381042
AppSec [baseline] (52.371 ms) : 0, 52371
AppSec [candidate] (50.312 ms) : 0, 50312
Remote Config [baseline] (801.922 µs) : 0, 802
Remote Config [candidate] (784.447 µs) : 0, 784
Telemetry [baseline] (7.449 ms) : 0, 7449
Telemetry [candidate] (7.484 ms) : 0, 7484
ProfilingAgent [baseline] (95.135 ms) : 0, 95135
ProfilingAgent [candidate] (96.472 ms) : 0, 96472
Profiling [baseline] (95.158 ms) : 0, 95158
Profiling [candidate] (96.496 ms) : 0, 96496
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.089 s) : 0, 1088964
Total [baseline] (8.572 s) : 0, 8571659
Agent [candidate] (1.086 s) : 0, 1085641
Total [candidate] (8.582 s) : 0, 8581513
section iast
Agent [baseline] (1.215 s) : 0, 1215483
Total [baseline] (9.068 s) : 0, 9068149
Agent [candidate] (1.205 s) : 0, 1205246
Total [candidate] (9.069 s) : 0, 9069343
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.209 s) : 0, 1209181
Total [baseline] (8.998 s) : 0, 8998253
Agent [candidate] (1.208 s) : 0, 1207816
Total [candidate] (9.017 s) : 0, 9016861
section iast_TELEMETRY_OFF
Agent [baseline] (1.208 s) : 0, 1207849
Total [baseline] (9.09 s) : 0, 9090259
Agent [candidate] (1.217 s) : 0, 1216523
Total [candidate] (9.075 s) : 0, 9074748
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.089 s -
Agent iast 1.215 s 126.52 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.209 s 120.217 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.208 s 118.886 ms (10.9%)
Total tracing 8.572 s -
Total iast 9.068 s 496.491 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.998 s 426.595 ms (5.0%)
Total iast_TELEMETRY_OFF 9.09 s 518.6 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent iast 1.205 s 119.606 ms (11.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 122.176 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.217 s 130.883 ms (12.1%)
Total tracing 8.582 s -
Total iast 9.069 s 487.83 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.017 s 435.347 ms (5.1%)
Total iast_TELEMETRY_OFF 9.075 s 493.234 ms (5.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (701.507 ms) : 0, 701507
BytebuddyAgent [candidate] (699.766 ms) : 0, 699766
GlobalTracer [baseline] (293.211 ms) : 0, 293211
GlobalTracer [candidate] (293.427 ms) : 0, 293427
AppSec [baseline] (51.089 ms) : 0, 51089
AppSec [candidate] (49.478 ms) : 0, 49478
Remote Config [baseline] (736.541 µs) : 0, 737
Remote Config [candidate] (737.731 µs) : 0, 738
Telemetry [baseline] (7.86 ms) : 0, 7860
Telemetry [candidate] (7.803 ms) : 0, 7803
section iast
BytebuddyAgent [baseline] (807.231 ms) : 0, 807231
BytebuddyAgent [candidate] (801.244 ms) : 0, 801244
GlobalTracer [baseline] (289.453 ms) : 0, 289453
GlobalTracer [candidate] (289.2 ms) : 0, 289200
AppSec [baseline] (53.853 ms) : 0, 53853
AppSec [candidate] (49.215 ms) : 0, 49215
IAST [baseline] (22.324 ms) : 0, 22324
IAST [candidate] (23.067 ms) : 0, 23067
Remote Config [baseline] (575.82 µs) : 0, 576
Remote Config [candidate] (581.761 µs) : 0, 582
Telemetry [baseline] (7.421 ms) : 0, 7421
Telemetry [candidate] (7.386 ms) : 0, 7386
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (802.99 ms) : 0, 802990
BytebuddyAgent [candidate] (802.702 ms) : 0, 802702
GlobalTracer [baseline] (289.065 ms) : 0, 289065
GlobalTracer [candidate] (290.178 ms) : 0, 290178
AppSec [baseline] (54.641 ms) : 0, 54641
AppSec [candidate] (50.269 ms) : 0, 50269
IAST [baseline] (20.656 ms) : 0, 20656
IAST [candidate] (23.183 ms) : 0, 23183
Remote Config [baseline] (593.344 µs) : 0, 593
Remote Config [candidate] (589.964 µs) : 0, 590
Telemetry [baseline] (6.67 ms) : 0, 6670
Telemetry [candidate] (6.652 ms) : 0, 6652
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.29 ms) : 0, 799290
BytebuddyAgent [candidate] (806.509 ms) : 0, 806509
GlobalTracer [baseline] (290.798 ms) : 0, 290798
GlobalTracer [candidate] (293.79 ms) : 0, 293790
AppSec [baseline] (53.531 ms) : 0, 53531
AppSec [candidate] (50.557 ms) : 0, 50557
IAST [baseline] (21.678 ms) : 0, 21678
IAST [candidate] (22.745 ms) : 0, 22745
Remote Config [baseline] (586.531 µs) : 0, 587
Remote Config [candidate] (612.168 µs) : 0, 612
Telemetry [baseline] (7.357 ms) : 0, 7357
Telemetry [candidate] (7.402 ms) : 0, 7402

Load

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1333, 1372
.   : milestone, 1352,
appsec (1.773 ms) : 1748, 1797
.   : milestone, 1773,
iast (1.535 ms) : 1512, 1558
.   : milestone, 1535,
profiling (1.547 ms) : 1524, 1571
.   : milestone, 1547,
tracing (1.49 ms) : 1467, 1514
.   : milestone, 1490,
section candidate
no_agent (1.356 ms) : 1336, 1375
.   : milestone, 1356,
appsec (1.774 ms) : 1750, 1798
.   : milestone, 1774,
iast (1.528 ms) : 1505, 1551
.   : milestone, 1528,
profiling (1.521 ms) : 1497, 1545
.   : milestone, 1521,
tracing (1.509 ms) : 1485, 1532
.   : milestone, 1509,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.372 ms] -
appsec 1.773 ms [1.748 ms, 1.797 ms] 420.137 µs (31.1%)
iast 1.535 ms [1.512 ms, 1.558 ms] 182.368 µs (13.5%)
profiling 1.547 ms [1.524 ms, 1.571 ms] 194.753 µs (14.4%)
tracing 1.49 ms [1.467 ms, 1.514 ms] 137.685 µs (10.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.336 ms, 1.375 ms] -
appsec 1.774 ms [1.75 ms, 1.798 ms] 418.255 µs (30.9%)
iast 1.528 ms [1.505 ms, 1.551 ms] 172.443 µs (12.7%)
profiling 1.521 ms [1.497 ms, 1.545 ms] 165.239 µs (12.2%)
tracing 1.509 ms [1.485 ms, 1.532 ms] 153.239 µs (11.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~a3bbdb9ded, baseline=1.32.0-SNAPSHOT~e9c489fd8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.386 µs) : 342, 381
.   : milestone, 361,
iast (467.967 µs) : 448, 488
.   : milestone, 468,
iast_FULL (541.413 µs) : 521, 562
.   : milestone, 541,
iast_GLOBAL (486.411 µs) : 466, 507
.   : milestone, 486,
iast_HARDCODED_SECRET_DISABLED (475.601 µs) : 455, 497
.   : milestone, 476,
iast_INACTIVE (444.694 µs) : 424, 465
.   : milestone, 445,
iast_TELEMETRY_OFF (469.866 µs) : 449, 491
.   : milestone, 470,
tracing (446.79 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (360.014 µs) : 340, 380
.   : milestone, 360,
iast (469.234 µs) : 449, 490
.   : milestone, 469,
iast_FULL (538.721 µs) : 518, 560
.   : milestone, 539,
iast_GLOBAL (500.454 µs) : 479, 522
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (481.197 µs) : 461, 502
.   : milestone, 481,
iast_INACTIVE (443.545 µs) : 423, 464
.   : milestone, 444,
iast_TELEMETRY_OFF (467.79 µs) : 447, 488
.   : milestone, 468,
tracing (444.619 µs) : 424, 465
.   : milestone, 445,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.386 µs [341.772 µs, 381.0 µs] -
iast 467.967 µs [447.708 µs, 488.227 µs] 106.581 µs (29.5%)
iast_FULL 541.413 µs [520.968 µs, 561.858 µs] 180.027 µs (49.8%)
iast_GLOBAL 486.411 µs [466.188 µs, 506.634 µs] 125.025 µs (34.6%)
iast_HARDCODED_SECRET_DISABLED 475.601 µs [454.592 µs, 496.611 µs] 114.215 µs (31.6%)
iast_INACTIVE 444.694 µs [424.269 µs, 465.119 µs] 83.308 µs (23.1%)
iast_TELEMETRY_OFF 469.866 µs [448.689 µs, 491.042 µs] 108.48 µs (30.0%)
tracing 446.79 µs [425.673 µs, 467.907 µs] 85.404 µs (23.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.014 µs [340.353 µs, 379.675 µs] -
iast 469.234 µs [448.899 µs, 489.568 µs] 109.22 µs (30.3%)
iast_FULL 538.721 µs [517.941 µs, 559.501 µs] 178.707 µs (49.6%)
iast_GLOBAL 500.454 µs [478.549 µs, 522.359 µs] 140.44 µs (39.0%)
iast_HARDCODED_SECRET_DISABLED 481.197 µs [460.506 µs, 501.888 µs] 121.183 µs (33.7%)
iast_INACTIVE 443.545 µs [423.235 µs, 463.855 µs] 83.531 µs (23.2%)
iast_TELEMETRY_OFF 467.79 µs [447.364 µs, 488.216 µs] 107.776 µs (29.9%)
tracing 444.619 µs [424.453 µs, 464.786 µs] 84.605 µs (23.5%)

manuel-alvarez-alvarez
manuel-alvarez-alvarez reviewed Mar 5, 2024
View reviewed changes
Copy link
Contributor

@manuel-alvarez-alvarez manuel-alvarez-alvarez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't it make more sense to minify during at build time?, it might make it easier to spot diffs when upgrading the rules.

@smola
Copy link
Member Author

smola commented Mar 5, 2024 •
edited

@manuel-alvarez-alvarez Yeah, I guess that would also help extending this to a few other JSONs we have around. We could try https://github.com/gradle-webtools/gradle-minify-plugin.

@smola smola force-pushed the smola/minify-appsec-rules branch from 5cb411c to a3bbdb9 Compare March 11, 2024 07:50
@smola
Copy link
Member Author

smola commented Mar 11, 2024

@manuel-alvarez-alvarez Here's an attempt at doing it at build time. I did not extend to other modules at the moment.

@smola smola merged commit 40a62c5 into master Mar 11, 2024
79 checks passed
@smola smola deleted the smola/minify-appsec-rules branch March 11, 2024 11:01
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

@ValentinZakharov ValentinZakharov Awaiting requested review from ValentinZakharov ValentinZakharov is a code owner automatically assigned from DataDog/asm-java

@mcculls mcculls Awaiting requested review from mcculls

Assignees
No one assigned
Labels
comp: asm waf Application Security Management (WAF)
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@smola @mcculls @manuel-alvarez-alvarez