Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure no IAST advices are added unless appsec is fully enabled #6813

Merged
merged 1 commit into from Mar 21, 2024
Merged

Ensure no IAST advices are added unless appsec is fully enabled #6813

merged 1 commit into from Mar 21, 2024

Conversation

manuel-alvarez-alvarez
Copy link
Contributor

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Mar 14, 2024 •
edited

What Does This Do

Removes IAST from triggering inactive opt-out advices when appsec starts in inactive mode, only when appsec is fully enabled IAST opt-out advices will be triggered (also fully enabled).

Motivation

Since there is no benefit from starting IAST opt-out advices in inactive mode besides testing stability (in fact the advices are no-ops so no vulnerabilities will be discovered), it's better simply to skip them.

@manuel-alvarez-alvarez manuel-alvarez-alvarez added the comp: asm iast Application Security Management (IAST) label Mar 14, 2024
@smola smola changed the title Ensure no advices are added unless appsec is fully enabled Ensure no IAST advices are added unless appsec is fully enabled Mar 14, 2024
@pr-commenter
Copy link

pr-commenter bot commented Mar 14, 2024 •
edited

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/iast-remove-inst-appsec-inactive
git_commit_date 1710879299 1710924629
git_commit_sha 97283c4 0924f0f
release_version 1.32.0-SNAPSHOT~97283c4020 1.32.0-SNAPSHOT~0924f0f242
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710927836 1710927836
ci_job_id 464147977 464147977
ci_pipeline_id 30428763 30428763
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 46 metrics, 16 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:appsec:IAST worse
[+1.097ms; +1.730ms] or [+6.119%; +9.644%]		 19.349ms 17.935ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.078 s) : 0, 1078480
Total [baseline] (8.582 s) : 0, 8582137
Agent [candidate] (1.086 s) : 0, 1086110
Total [candidate] (8.601 s) : 0, 8601432
section iast
Agent [baseline] (1.21 s) : 0, 1210329
Total [baseline] (9.05 s) : 0, 9050050
Agent [candidate] (1.202 s) : 0, 1202475
Total [candidate] (9.071 s) : 0, 9071197
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.207 s) : 0, 1207028
Total [baseline] (9.023 s) : 0, 9022754
Agent [candidate] (1.201 s) : 0, 1200614
Total [candidate] (8.993 s) : 0, 8993374
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1196796
Total [baseline] (9.024 s) : 0, 9023504
Agent [candidate] (1.201 s) : 0, 1200646
Total [candidate] (9.045 s) : 0, 9044550
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.078 s -
Agent iast 1.21 s 131.849 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.207 s 128.547 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.197 s 118.316 ms (11.0%)
Total tracing 8.582 s -
Total iast 9.05 s 467.913 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.023 s 440.617 ms (5.1%)
Total iast_TELEMETRY_OFF 9.024 s 441.367 ms (5.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent iast 1.202 s 116.365 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.201 s 114.504 ms (10.5%)
Agent iast_TELEMETRY_OFF 1.201 s 114.535 ms (10.5%)
Total tracing 8.601 s -
Total iast 9.071 s 469.765 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 8.993 s 391.942 ms (4.6%)
Total iast_TELEMETRY_OFF 9.045 s 443.118 ms (5.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (695.76 ms) : 0, 695760
BytebuddyAgent [candidate] (693.065 ms) : 0, 693065
GlobalTracer [baseline] (290.89 ms) : 0, 290890
GlobalTracer [candidate] (299.983 ms) : 0, 299983
AppSec [baseline] (48.905 ms) : 0, 48905
AppSec [candidate] (49.594 ms) : 0, 49594
Remote Config [baseline] (1.147 ms) : 0, 1147
Remote Config [candidate] (1.044 ms) : 0, 1044
Telemetry [baseline] (7.482 ms) : 0, 7482
Telemetry [candidate] (7.636 ms) : 0, 7636
section iast
BytebuddyAgent [baseline] (804.713 ms) : 0, 804713
BytebuddyAgent [candidate] (796.686 ms) : 0, 796686
GlobalTracer [baseline] (289.334 ms) : 0, 289334
GlobalTracer [candidate] (289.156 ms) : 0, 289156
AppSec [baseline] (51.23 ms) : 0, 51230
AppSec [candidate] (48.859 ms) : 0, 48859
IAST [baseline] (23.242 ms) : 0, 23242
IAST [candidate] (25.527 ms) : 0, 25527
Remote Config [baseline] (585.157 µs) : 0, 585
Remote Config [candidate] (560.395 µs) : 0, 560
Telemetry [baseline] (6.723 ms) : 0, 6723
Telemetry [candidate] (7.313 ms) : 0, 7313
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (802.603 ms) : 0, 802603
BytebuddyAgent [candidate] (795.924 ms) : 0, 795924
GlobalTracer [baseline] (289.249 ms) : 0, 289249
GlobalTracer [candidate] (289.023 ms) : 0, 289023
AppSec [baseline] (50.941 ms) : 0, 50941
AppSec [candidate] (50.346 ms) : 0, 50346
IAST [baseline] (21.998 ms) : 0, 21998
IAST [candidate] (23.255 ms) : 0, 23255
Remote Config [baseline] (560.673 µs) : 0, 561
Remote Config [candidate] (572.347 µs) : 0, 572
Telemetry [baseline] (7.358 ms) : 0, 7358
Telemetry [candidate] (7.206 ms) : 0, 7206
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.531 ms) : 0, 792531
BytebuddyAgent [candidate] (795.342 ms) : 0, 795342
GlobalTracer [baseline] (288.779 ms) : 0, 288779
GlobalTracer [candidate] (289.997 ms) : 0, 289997
AppSec [baseline] (50.32 ms) : 0, 50320
AppSec [candidate] (47.367 ms) : 0, 47367
IAST [baseline] (23.787 ms) : 0, 23787
IAST [candidate] (24.815 ms) : 0, 24815
Remote Config [baseline] (571.113 µs) : 0, 571
Remote Config [candidate] (576.475 µs) : 0, 576
Telemetry [baseline] (6.534 ms) : 0, 6534
Telemetry [candidate] (8.139 ms) : 0, 8139
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1082196
Total [baseline] (9.216 s) : 0, 9215561
Agent [candidate] (1.078 s) : 0, 1077589
Total [candidate] (9.215 s) : 0, 9214512
section appsec
Agent [baseline] (1.206 s) : 0, 1205605
Total [baseline] (9.326 s) : 0, 9326232
Agent [candidate] (1.204 s) : 0, 1204107
Total [candidate] (9.265 s) : 0, 9264836
section iast
Agent [baseline] (1.206 s) : 0, 1206000
Total [baseline] (9.36 s) : 0, 9360260
Agent [candidate] (1.221 s) : 0, 1221072
Total [candidate] (9.405 s) : 0, 9404923
section profiling
Agent [baseline] (1.274 s) : 0, 1274126
Total [baseline] (9.336 s) : 0, 9336387
Agent [candidate] (1.274 s) : 0, 1274083
Total [candidate] (9.422 s) : 0, 9421811
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent appsec 1.206 s 123.409 ms (11.4%)
Agent iast 1.206 s 123.803 ms (11.4%)
Agent profiling 1.274 s 191.93 ms (17.7%)
Total tracing 9.216 s -
Total appsec 9.326 s 110.672 ms (1.2%)
Total iast 9.36 s 144.7 ms (1.6%)
Total profiling 9.336 s 120.826 ms (1.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.078 s -
Agent appsec 1.204 s 126.519 ms (11.7%)
Agent iast 1.221 s 143.483 ms (13.3%)
Agent profiling 1.274 s 196.495 ms (18.2%)
Total tracing 9.215 s -
Total appsec 9.265 s 50.324 ms (0.5%)
Total iast 9.405 s 190.411 ms (2.1%)
Total profiling 9.422 s 207.299 ms (2.2%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (696.89 ms) : 0, 696890
BytebuddyAgent [candidate] (687.296 ms) : 0, 687296
GlobalTracer [baseline] (292.827 ms) : 0, 292827
GlobalTracer [candidate] (297.838 ms) : 0, 297838
AppSec [baseline] (49.547 ms) : 0, 49547
AppSec [candidate] (49.484 ms) : 0, 49484
Remote Config [baseline] (1.126 ms) : 0, 1126
Remote Config [candidate] (1.061 ms) : 0, 1061
Telemetry [baseline] (7.436 ms) : 0, 7436
Telemetry [candidate] (7.514 ms) : 0, 7514
section appsec
BytebuddyAgent [baseline] (699.758 ms) : 0, 699758
BytebuddyAgent [candidate] (697.138 ms) : 0, 697138
GlobalTracer [baseline] (292.556 ms) : 0, 292556
GlobalTracer [candidate] (291.857 ms) : 0, 291857
AppSec [baseline] (153.446 ms) : 0, 153446
AppSec [candidate] (154.077 ms) : 0, 154077
Remote Config [baseline] (616.504 µs) : 0, 617
Remote Config [candidate] (611.739 µs) : 0, 612
Telemetry [baseline] (6.87 ms) : 0, 6870
Telemetry [candidate] (6.807 ms) : 0, 6807
IAST [baseline] (17.935 ms) : 0, 17935
IAST [candidate] (19.349 ms) : 0, 19349
section iast
BytebuddyAgent [baseline] (801.758 ms) : 0, 801758
BytebuddyAgent [candidate] (810.321 ms) : 0, 810321
GlobalTracer [baseline] (288.473 ms) : 0, 288473
GlobalTracer [candidate] (292.892 ms) : 0, 292892
AppSec [baseline] (51.165 ms) : 0, 51165
AppSec [candidate] (48.067 ms) : 0, 48067
Remote Config [baseline] (567.184 µs) : 0, 567
Remote Config [candidate] (595.361 µs) : 0, 595
Telemetry [baseline] (8.118 ms) : 0, 8118
Telemetry [candidate] (7.319 ms) : 0, 7319
IAST [baseline] (21.428 ms) : 0, 21428
IAST [candidate] (27.029 ms) : 0, 27029
section profiling
BytebuddyAgent [baseline] (689.831 ms) : 0, 689831
BytebuddyAgent [candidate] (680.753 ms) : 0, 680753
GlobalTracer [baseline] (376.14 ms) : 0, 376140
GlobalTracer [candidate] (383.139 ms) : 0, 383139
AppSec [baseline] (49.91 ms) : 0, 49910
AppSec [candidate] (49.798 ms) : 0, 49798
Remote Config [baseline] (852.123 µs) : 0, 852
Remote Config [candidate] (863.448 µs) : 0, 863
Telemetry [baseline] (7.278 ms) : 0, 7278
Telemetry [candidate] (7.465 ms) : 0, 7465
ProfilingAgent [baseline] (93.93 ms) : 0, 93930
ProfilingAgent [candidate] (95.755 ms) : 0, 95755
Profiling [baseline] (93.953 ms) : 0, 93953
Profiling [candidate] (95.779 ms) : 0, 95779

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.441 µs) : 340, 381
.   : milestone, 360,
iast (473.081 µs) : 453, 493
.   : milestone, 473,
iast_FULL (533.931 µs) : 513, 554
.   : milestone, 534,
iast_GLOBAL (491.733 µs) : 471, 513
.   : milestone, 492,
iast_HARDCODED_SECRET_DISABLED (464.444 µs) : 444, 485
.   : milestone, 464,
iast_INACTIVE (443.909 µs) : 424, 464
.   : milestone, 444,
iast_TELEMETRY_OFF (465.701 µs) : 445, 486
.   : milestone, 466,
tracing (440.226 µs) : 420, 461
.   : milestone, 440,
section candidate
no_agent (370.851 µs) : 350, 391
.   : milestone, 371,
iast (472.349 µs) : 451, 493
.   : milestone, 472,
iast_FULL (536.617 µs) : 516, 557
.   : milestone, 537,
iast_GLOBAL (490.016 µs) : 470, 510
.   : milestone, 490,
iast_HARDCODED_SECRET_DISABLED (477.987 µs) : 457, 499
.   : milestone, 478,
iast_INACTIVE (448.704 µs) : 428, 469
.   : milestone, 449,
iast_TELEMETRY_OFF (467.491 µs) : 447, 488
.   : milestone, 467,
tracing (436.546 µs) : 416, 457
.   : milestone, 437,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.441 µs [340.107 µs, 380.775 µs] -
iast 473.081 µs [452.721 µs, 493.44 µs] 112.64 µs (31.3%)
iast_FULL 533.931 µs [513.425 µs, 554.438 µs] 173.49 µs (48.1%)
iast_GLOBAL 491.733 µs [470.729 µs, 512.738 µs] 131.292 µs (36.4%)
iast_HARDCODED_SECRET_DISABLED 464.444 µs [444.356 µs, 484.533 µs] 104.003 µs (28.9%)
iast_INACTIVE 443.909 µs [423.816 µs, 464.001 µs] 83.468 µs (23.2%)
iast_TELEMETRY_OFF 465.701 µs [445.345 µs, 486.057 µs] 105.26 µs (29.2%)
tracing 440.226 µs [419.569 µs, 460.883 µs] 79.785 µs (22.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.851 µs [350.317 µs, 391.385 µs] -
iast 472.349 µs [451.427 µs, 493.27 µs] 101.497 µs (27.4%)
iast_FULL 536.617 µs [515.843 µs, 557.39 µs] 165.765 µs (44.7%)
iast_GLOBAL 490.016 µs [469.762 µs, 510.27 µs] 119.165 µs (32.1%)
iast_HARDCODED_SECRET_DISABLED 477.987 µs [456.884 µs, 499.09 µs] 107.135 µs (28.9%)
iast_INACTIVE 448.704 µs [428.054 µs, 469.354 µs] 77.852 µs (21.0%)
iast_TELEMETRY_OFF 467.491 µs [447.279 µs, 487.703 µs] 96.64 µs (26.1%)
tracing 436.546 µs [415.748 µs, 457.344 µs] 65.694 µs (17.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~0924f0f242, baseline=1.32.0-SNAPSHOT~97283c4020
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1361
.   : milestone, 1342,
appsec (1.751 ms) : 1728, 1775
.   : milestone, 1751,
iast (1.534 ms) : 1511, 1556
.   : milestone, 1534,
profiling (1.555 ms) : 1531, 1580
.   : milestone, 1555,
tracing (1.512 ms) : 1489, 1536
.   : milestone, 1512,
section candidate
no_agent (1.349 ms) : 1330, 1368
.   : milestone, 1349,
appsec (1.753 ms) : 1729, 1776
.   : milestone, 1753,
iast (1.512 ms) : 1488, 1535
.   : milestone, 1512,
profiling (1.544 ms) : 1519, 1569
.   : milestone, 1544,
tracing (1.516 ms) : 1493, 1539
.   : milestone, 1516,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.361 ms] -
appsec 1.751 ms [1.728 ms, 1.775 ms] 409.381 µs (30.5%)
iast 1.534 ms [1.511 ms, 1.556 ms] 191.91 µs (14.3%)
profiling 1.555 ms [1.531 ms, 1.58 ms] 213.429 µs (15.9%)
tracing 1.512 ms [1.489 ms, 1.536 ms] 170.581 µs (12.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.33 ms, 1.368 ms] -
appsec 1.753 ms [1.729 ms, 1.776 ms] 403.562 µs (29.9%)
iast 1.512 ms [1.488 ms, 1.535 ms] 162.412 µs (12.0%)
profiling 1.544 ms [1.519 ms, 1.569 ms] 194.927 µs (14.4%)
tracing 1.516 ms [1.493 ms, 1.539 ms] 166.548 µs (12.3%)

@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-remove-inst-appsec-inactive branch from 02bd708 to ec14c24 Compare March 14, 2024 17:16
@manuel-alvarez-alvarez manuel-alvarez-alvarez marked this pull request as ready for review March 14, 2024 17:16
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-remove-inst-appsec-inactive branch from 5736332 to a9bf7f4 Compare March 19, 2024 08:16
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-remove-inst-appsec-inactive branch from a9bf7f4 to 0924f0f Compare March 20, 2024 08:50
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-remove-inst-appsec-inactive branch from 0924f0f to 1eb9bab Compare March 21, 2024 09:00
@manuel-alvarez-alvarez manuel-alvarez-alvarez merged commit 7160117 into master Mar 21, 2024
80 checks passed
@manuel-alvarez-alvarez manuel-alvarez-alvarez deleted the malvarez/iast-remove-inst-appsec-inactive branch March 21, 2024 11:47
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@jandro996 jandro996 jandro996 approved these changes

@ValentinZakharov ValentinZakharov Awaiting requested review from ValentinZakharov ValentinZakharov is a code owner automatically assigned from DataDog/asm-java

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

@am312 am312 Awaiting requested review from am312 am312 is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST)
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@manuel-alvarez-alvarez @smola @jandro996