Improve IAST metric unwrapping logic #6831

manuel-alvarez-alvarez · 2024-03-21T12:36:11Z

What Does This Do

Improves the handling of wrapped metrics from two fronts:

Removes the Function to check if a metric is wrapped that was causing unnecessary wrapped types conversion
Removes the wrapping logic from the happy path so most tags are not affected by the wrapping

pr-commenter · 2024-03-21T13:28:54Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	malvarez/iast-metric-perf
git_commit_date	1711131884	1711357104
git_commit_sha	`b856443`	`005d600`
release_version	1.32.0-SNAPSHOT~b85644360c	1.32.0-SNAPSHOT~005d600627

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1711360544	1711360544
ci_job_id	468021088	468021088
ci_pipeline_id	30714160	30714160
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 14 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:petclinic:appsec:IAST	better [-818.252µs; -562.204µs] or [-4.242%; -2.915%]	18.598ms	19.288ms

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.075 s) : 0, 1075087
Total [baseline] (10.411 s) : 0, 10411114
Agent [candidate] (1.083 s) : 0, 1083123
Total [candidate] (10.41 s) : 0, 10409598
section appsec
Agent [baseline] (1.199 s) : 0, 1198923
Total [baseline] (10.5 s) : 0, 10500096
Agent [candidate] (1.201 s) : 0, 1200594
Total [candidate] (10.51 s) : 0, 10510025
section iast
Agent [baseline] (1.206 s) : 0, 1205764
Total [baseline] (10.896 s) : 0, 10896065
Agent [candidate] (1.2 s) : 0, 1199994
Total [candidate] (10.959 s) : 0, 10958972
section profiling
Agent [baseline] (1.264 s) : 0, 1263875
Total [baseline] (10.554 s) : 0, 10553740
Agent [candidate] (1.269 s) : 0, 1268658
Total [candidate] (10.598 s) : 0, 10597672

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.075 s	-
Agent	appsec	1.199 s	123.836 ms (11.5%)
Agent	iast	1.206 s	130.677 ms (12.2%)
Agent	profiling	1.264 s	188.787 ms (17.6%)
Total	tracing	10.411 s	-
Total	appsec	10.5 s	88.981 ms (0.9%)
Total	iast	10.896 s	484.95 ms (4.7%)
Total	profiling	10.554 s	142.626 ms (1.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.083 s	-
Agent	appsec	1.201 s	117.471 ms (10.8%)
Agent	iast	1.2 s	116.872 ms (10.8%)
Agent	profiling	1.269 s	185.535 ms (17.1%)
Total	tracing	10.41 s	-
Total	appsec	10.51 s	100.427 ms (1.0%)
Total	iast	10.959 s	549.374 ms (5.3%)
Total	profiling	10.598 s	188.075 ms (1.8%)

gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.665 ms) : 0, 685665
BytebuddyAgent [candidate] (690.363 ms) : 0, 690363
GlobalTracer [baseline] (297.076 ms) : 0, 297076
GlobalTracer [candidate] (300.049 ms) : 0, 300049
AppSec [baseline] (49.416 ms) : 0, 49416
AppSec [candidate] (49.605 ms) : 0, 49605
Remote Config [baseline] (1.073 ms) : 0, 1073
Remote Config [candidate] (1.075 ms) : 0, 1075
Telemetry [baseline] (7.504 ms) : 0, 7504
Telemetry [candidate] (7.417 ms) : 0, 7417
section appsec
BytebuddyAgent [baseline] (693.437 ms) : 0, 693437
BytebuddyAgent [candidate] (694.403 ms) : 0, 694403
GlobalTracer [baseline] (291.063 ms) : 0, 291063
GlobalTracer [candidate] (291.963 ms) : 0, 291963
AppSec [baseline] (153.401 ms) : 0, 153401
AppSec [candidate] (153.863 ms) : 0, 153863
IAST [baseline] (19.288 ms) : 0, 19288
IAST [candidate] (18.598 ms) : 0, 18598
Remote Config [baseline] (609.5 µs) : 0, 610
Remote Config [candidate] (600.545 µs) : 0, 601
Telemetry [baseline] (6.769 ms) : 0, 6769
Telemetry [candidate] (6.769 ms) : 0, 6769
section iast
BytebuddyAgent [baseline] (799.174 ms) : 0, 799174
BytebuddyAgent [candidate] (794.341 ms) : 0, 794341
GlobalTracer [baseline] (290.395 ms) : 0, 290395
GlobalTracer [candidate] (289.303 ms) : 0, 289303
AppSec [baseline] (49.509 ms) : 0, 49509
AppSec [candidate] (48.821 ms) : 0, 48821
IAST [baseline] (25.069 ms) : 0, 25069
IAST [candidate] (25.981 ms) : 0, 25981
Remote Config [baseline] (588.94 µs) : 0, 589
Remote Config [candidate] (577.801 µs) : 0, 578
Telemetry [baseline] (6.512 ms) : 0, 6512
Telemetry [candidate] (6.609 ms) : 0, 6609
section profiling
BytebuddyAgent [baseline] (675.172 ms) : 0, 675172
BytebuddyAgent [candidate] (676.855 ms) : 0, 676855
GlobalTracer [baseline] (379.089 ms) : 0, 379089
GlobalTracer [candidate] (382.344 ms) : 0, 382344
AppSec [baseline] (49.438 ms) : 0, 49438
AppSec [candidate] (49.622 ms) : 0, 49622
Remote Config [baseline] (869.681 µs) : 0, 870
Remote Config [candidate] (897.706 µs) : 0, 898
Telemetry [baseline] (7.406 ms) : 0, 7406
Telemetry [candidate] (7.456 ms) : 0, 7456
ProfilingAgent [baseline] (95.745 ms) : 0, 95745
ProfilingAgent [candidate] (95.222 ms) : 0, 95222
Profiling [baseline] (95.769 ms) : 0, 95769
Profiling [candidate] (95.246 ms) : 0, 95246

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1076463
Total [baseline] (8.608 s) : 0, 8608291
Agent [candidate] (1.082 s) : 0, 1082355
Total [candidate] (8.569 s) : 0, 8569274
section iast
Agent [baseline] (1.201 s) : 0, 1201143
Total [baseline] (9.065 s) : 0, 9065066
Agent [candidate] (1.201 s) : 0, 1200564
Total [candidate] (9.124 s) : 0, 9124007
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.196 s) : 0, 1195894
Total [baseline] (9.06 s) : 0, 9059960
Agent [candidate] (1.201 s) : 0, 1200734
Total [candidate] (9.073 s) : 0, 9073178
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1196544
Total [baseline] (9.05 s) : 0, 9050092
Agent [candidate] (1.206 s) : 0, 1205888
Total [candidate] (9.12 s) : 0, 9119541

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.076 s	-
Agent	iast	1.201 s	124.68 ms (11.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.196 s	119.43 ms (11.1%)
Agent	iast_TELEMETRY_OFF	1.197 s	120.08 ms (11.2%)
Total	tracing	8.608 s	-
Total	iast	9.065 s	456.775 ms (5.3%)
Total	iast_HARDCODED_SECRET_DISABLED	9.06 s	451.669 ms (5.2%)
Total	iast_TELEMETRY_OFF	9.05 s	441.801 ms (5.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.082 s	-
Agent	iast	1.201 s	118.208 ms (10.9%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.201 s	118.379 ms (10.9%)
Agent	iast_TELEMETRY_OFF	1.206 s	123.533 ms (11.4%)
Total	tracing	8.569 s	-
Total	iast	9.124 s	554.733 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.073 s	503.904 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.12 s	550.267 ms (6.4%)

gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.22 ms) : 0, 685220
BytebuddyAgent [candidate] (689.603 ms) : 0, 689603
GlobalTracer [baseline] (298.651 ms) : 0, 298651
GlobalTracer [candidate] (299.914 ms) : 0, 299914
AppSec [baseline] (49.621 ms) : 0, 49621
AppSec [candidate] (49.518 ms) : 0, 49518
Remote Config [baseline] (1.082 ms) : 0, 1082
Remote Config [candidate] (1.056 ms) : 0, 1056
Telemetry [baseline] (7.544 ms) : 0, 7544
Telemetry [candidate] (7.482 ms) : 0, 7482
section iast
BytebuddyAgent [baseline] (797.215 ms) : 0, 797215
BytebuddyAgent [candidate] (794.264 ms) : 0, 794264
GlobalTracer [baseline] (287.84 ms) : 0, 287840
GlobalTracer [candidate] (289.693 ms) : 0, 289693
AppSec [baseline] (49.014 ms) : 0, 49014
AppSec [candidate] (51.235 ms) : 0, 51235
IAST [baseline] (24.602 ms) : 0, 24602
IAST [candidate] (23.764 ms) : 0, 23764
Remote Config [baseline] (568.571 µs) : 0, 569
Remote Config [candidate] (576.054 µs) : 0, 576
Telemetry [baseline] (7.288 ms) : 0, 7288
Telemetry [candidate] (6.623 ms) : 0, 6623
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (792.215 ms) : 0, 792215
BytebuddyAgent [candidate] (794.566 ms) : 0, 794566
GlobalTracer [baseline] (287.265 ms) : 0, 287265
GlobalTracer [candidate] (290.046 ms) : 0, 290046
AppSec [baseline] (48.758 ms) : 0, 48758
AppSec [candidate] (49.755 ms) : 0, 49755
IAST [baseline] (24.566 ms) : 0, 24566
IAST [candidate] (24.775 ms) : 0, 24775
Remote Config [baseline] (582.304 µs) : 0, 582
Remote Config [candidate] (576.513 µs) : 0, 577
Telemetry [baseline] (8.208 ms) : 0, 8208
Telemetry [candidate] (6.635 ms) : 0, 6635
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (791.19 ms) : 0, 791190
BytebuddyAgent [candidate] (797.346 ms) : 0, 797346
GlobalTracer [baseline] (288.826 ms) : 0, 288826
GlobalTracer [candidate] (292.021 ms) : 0, 292021
AppSec [baseline] (49.095 ms) : 0, 49095
AppSec [candidate] (49.965 ms) : 0, 49965
IAST [baseline] (25.867 ms) : 0, 25867
IAST [candidate] (24.738 ms) : 0, 24738
Remote Config [baseline] (576.498 µs) : 0, 576
Remote Config [candidate] (582.843 µs) : 0, 583
Telemetry [baseline] (6.578 ms) : 0, 6578
Telemetry [candidate] (6.602 ms) : 0, 6602

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-03-25T09:27:57	2024-03-25T09:49:59
git_branch	master	malvarez/iast-metric-perf
git_commit_date	1711131884	1711357104
git_commit_sha	`b856443`	`005d600`
release_version	1.32.0-SNAPSHOT~b85644360c	1.32.0-SNAPSHOT~005d600627
start_time	2024-03-25T09:27:44	2024-03-25T09:49:46

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1711360544	1711360544
ci_job_id	468021088	468021088
ci_pipeline_id	30714160	30714160
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 17 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	worse [+36.548µs; +92.262µs] or [+2.421%; +6.112%]	unstable [-662.753op/s; +434.833op/s] or [-21.539%; +14.132%]	1.574ms	2962.963op/s	1.510ms	3076.923op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.328 ms) : 1309, 1348
.   : milestone, 1328,
appsec (1.724 ms) : 1700, 1749
.   : milestone, 1724,
appsec_no_iast (1.73 ms) : 1705, 1754
.   : milestone, 1730,
iast (1.482 ms) : 1459, 1505
.   : milestone, 1482,
profiling (1.51 ms) : 1483, 1536
.   : milestone, 1510,
tracing (1.476 ms) : 1452, 1499
.   : milestone, 1476,
section candidate
no_agent (1.324 ms) : 1305, 1343
.   : milestone, 1324,
appsec (1.724 ms) : 1699, 1749
.   : milestone, 1724,
appsec_no_iast (1.723 ms) : 1698, 1748
.   : milestone, 1723,
iast (1.479 ms) : 1457, 1502
.   : milestone, 1479,
profiling (1.574 ms) : 1549, 1599
.   : milestone, 1574,
tracing (1.476 ms) : 1452, 1500
.   : milestone, 1476,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.328 ms [1.309 ms, 1.348 ms]	-
appsec	1.724 ms [1.7 ms, 1.749 ms]	395.788 µs (29.8%)
appsec_no_iast	1.73 ms [1.705 ms, 1.754 ms]	401.221 µs (30.2%)
iast	1.482 ms [1.459 ms, 1.505 ms]	153.251 µs (11.5%)
profiling	1.51 ms [1.483 ms, 1.536 ms]	181.05 µs (13.6%)
tracing	1.476 ms [1.452 ms, 1.499 ms]	147.095 µs (11.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.324 ms [1.305 ms, 1.343 ms]	-
appsec	1.724 ms [1.699 ms, 1.749 ms]	399.997 µs (30.2%)
appsec_no_iast	1.723 ms [1.698 ms, 1.748 ms]	399.059 µs (30.1%)
iast	1.479 ms [1.457 ms, 1.502 ms]	155.312 µs (11.7%)
profiling	1.574 ms [1.549 ms, 1.599 ms]	249.86 µs (18.9%)
tracing	1.476 ms [1.452 ms, 1.5 ms]	151.837 µs (11.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~005d600627, baseline=1.32.0-SNAPSHOT~b85644360c
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.159 µs) : 345, 389
.   : milestone, 367,
iast (470.482 µs) : 450, 491
.   : milestone, 470,
iast_FULL (533.753 µs) : 513, 555
.   : milestone, 534,
iast_GLOBAL (488.567 µs) : 468, 509
.   : milestone, 489,
iast_HARDCODED_SECRET_DISABLED (477.114 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (447.567 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (474.407 µs) : 453, 496
.   : milestone, 474,
tracing (433.724 µs) : 414, 454
.   : milestone, 434,
section candidate
no_agent (365.453 µs) : 346, 385
.   : milestone, 365,
iast (470.068 µs) : 450, 491
.   : milestone, 470,
iast_FULL (537.094 µs) : 517, 558
.   : milestone, 537,
iast_GLOBAL (487.249 µs) : 467, 508
.   : milestone, 487,
iast_HARDCODED_SECRET_DISABLED (475.686 µs) : 455, 497
.   : milestone, 476,
iast_INACTIVE (448.664 µs) : 427, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (469.567 µs) : 448, 491
.   : milestone, 470,
tracing (442.938 µs) : 422, 464
.   : milestone, 443,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	367.159 µs [345.462 µs, 388.856 µs]	-
iast	470.482 µs [449.681 µs, 491.284 µs]	103.323 µs (28.1%)
iast_FULL	533.753 µs [512.851 µs, 554.655 µs]	166.594 µs (45.4%)
iast_GLOBAL	488.567 µs [467.872 µs, 509.263 µs]	121.408 µs (33.1%)
iast_HARDCODED_SECRET_DISABLED	477.114 µs [456.076 µs, 498.152 µs]	109.955 µs (29.9%)
iast_INACTIVE	447.567 µs [426.563 µs, 468.571 µs]	80.408 µs (21.9%)
iast_TELEMETRY_OFF	474.407 µs [453.106 µs, 495.708 µs]	107.249 µs (29.2%)
tracing	433.724 µs [413.584 µs, 453.864 µs]	66.565 µs (18.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	365.453 µs [345.659 µs, 385.247 µs]	-
iast	470.068 µs [449.506 µs, 490.63 µs]	104.615 µs (28.6%)
iast_FULL	537.094 µs [516.668 µs, 557.521 µs]	171.641 µs (47.0%)
iast_GLOBAL	487.249 µs [466.881 µs, 507.617 µs]	121.796 µs (33.3%)
iast_HARDCODED_SECRET_DISABLED	475.686 µs [454.833 µs, 496.539 µs]	110.233 µs (30.2%)
iast_INACTIVE	448.664 µs [427.158 µs, 470.17 µs]	83.211 µs (22.8%)
iast_TELEMETRY_OFF	469.567 µs [448.444 µs, 490.69 µs]	104.114 µs (28.5%)
tracing	442.938 µs [422.37 µs, 463.505 µs]	77.485 µs (21.2%)

manuel-alvarez-alvarez added tag: performance Performance related changes comp: asm iast Application Security Management (IAST) labels Mar 21, 2024

manuel-alvarez-alvarez force-pushed the malvarez/iast-metric-perf branch from 3c47a15 to 9bd5ede Compare March 22, 2024 10:49

manuel-alvarez-alvarez changed the title ~~Delay IAST metric unwrapping to improve perf~~ Improve IAST metric unwrapping logic Mar 22, 2024

manuel-alvarez-alvarez force-pushed the malvarez/iast-metric-perf branch from 9bd5ede to f27e538 Compare March 22, 2024 10:54

manuel-alvarez-alvarez marked this pull request as ready for review March 22, 2024 10:54

manuel-alvarez-alvarez requested a review from a team as a code owner March 22, 2024 10:54

manuel-alvarez-alvarez requested review from smola and ValentinZakharov March 22, 2024 10:54

manuel-alvarez-alvarez force-pushed the malvarez/iast-metric-perf branch from f27e538 to f3b71b7 Compare March 22, 2024 10:56

jandro996 approved these changes Mar 22, 2024

View reviewed changes

manuel-alvarez-alvarez added the run-tests: all Run all tests label Mar 22, 2024

Improve IAST metric unwrapping logic

005d600

manuel-alvarez-alvarez force-pushed the malvarez/iast-metric-perf branch from f3b71b7 to 005d600 Compare March 25, 2024 09:00

manuel-alvarez-alvarez merged commit bfd3a15 into master Mar 25, 2024
130 checks passed

manuel-alvarez-alvarez deleted the malvarez/iast-metric-perf branch March 25, 2024 11:42

github-actions bot added this to the 1.32.0 milestone Mar 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve IAST metric unwrapping logic #6831

Improve IAST metric unwrapping logic #6831

manuel-alvarez-alvarez commented Mar 21, 2024 •

edited

pr-commenter bot commented Mar 21, 2024 •

edited

Improve IAST metric unwrapping logic #6831

Improve IAST metric unwrapping logic #6831

Conversation

manuel-alvarez-alvarez commented Mar 21, 2024 • edited

What Does This Do

pr-commenter bot commented Mar 21, 2024 • edited

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

manuel-alvarez-alvarez commented Mar 21, 2024 •

edited

pr-commenter bot commented Mar 21, 2024 •

edited