-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize Redaction detection when capturing values #6947
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081327
Total [baseline] (10.34 s) : 0, 10340164
Agent [candidate] (1.082 s) : 0, 1082403
Total [candidate] (10.495 s) : 0, 10495290
section appsec
Agent [baseline] (1.198 s) : 0, 1198191
Total [baseline] (10.479 s) : 0, 10479265
Agent [candidate] (1.188 s) : 0, 1188447
Total [candidate] (10.467 s) : 0, 10466756
section iast
Agent [baseline] (1.206 s) : 0, 1205996
Total [baseline] (10.719 s) : 0, 10718758
Agent [candidate] (1.198 s) : 0, 1197656
Total [candidate] (10.78 s) : 0, 10780241
section profiling
Agent [baseline] (1.267 s) : 0, 1266693
Total [baseline] (10.683 s) : 0, 10683190
Agent [candidate] (1.275 s) : 0, 1274673
Total [candidate] (10.597 s) : 0, 10597286
gantt
title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.404 ms) : 0, 677404
BytebuddyAgent [candidate] (677.653 ms) : 0, 677653
GlobalTracer [baseline] (311.3 ms) : 0, 311300
GlobalTracer [candidate] (312.04 ms) : 0, 312040
AppSec [baseline] (49.651 ms) : 0, 49651
AppSec [candidate] (49.763 ms) : 0, 49763
Remote Config [baseline] (662.545 µs) : 0, 663
Remote Config [candidate] (667.708 µs) : 0, 668
Telemetry [baseline] (7.668 ms) : 0, 7668
Telemetry [candidate] (7.631 ms) : 0, 7631
section appsec
BytebuddyAgent [baseline] (701.49 ms) : 0, 701490
BytebuddyAgent [candidate] (694.818 ms) : 0, 694818
GlobalTracer [baseline] (293.546 ms) : 0, 293546
GlobalTracer [candidate] (291.238 ms) : 0, 291238
AppSec [baseline] (150.134 ms) : 0, 150134
AppSec [candidate] (149.586 ms) : 0, 149586
IAST [baseline] (19.536 ms) : 0, 19536
IAST [candidate] (19.306 ms) : 0, 19306
Remote Config [baseline] (635.925 µs) : 0, 636
Remote Config [candidate] (623.219 µs) : 0, 623
Telemetry [baseline] (7.918 ms) : 0, 7918
Telemetry [candidate] (8.155 ms) : 0, 8155
section iast
BytebuddyAgent [baseline] (799.261 ms) : 0, 799261
BytebuddyAgent [candidate] (793.043 ms) : 0, 793043
GlobalTracer [baseline] (289.839 ms) : 0, 289839
GlobalTracer [candidate] (287.747 ms) : 0, 287747
AppSec [baseline] (50.221 ms) : 0, 50221
AppSec [candidate] (48.258 ms) : 0, 48258
IAST [baseline] (24.186 ms) : 0, 24186
IAST [candidate] (25.454 ms) : 0, 25454
Remote Config [baseline] (569.155 µs) : 0, 569
Remote Config [candidate] (575.815 µs) : 0, 576
Telemetry [baseline] (7.379 ms) : 0, 7379
Telemetry [candidate] (8.126 ms) : 0, 8126
section profiling
BytebuddyAgent [baseline] (676.388 ms) : 0, 676388
BytebuddyAgent [candidate] (680.635 ms) : 0, 680635
GlobalTracer [baseline] (380.14 ms) : 0, 380140
GlobalTracer [candidate] (381.729 ms) : 0, 381729
AppSec [baseline] (50.321 ms) : 0, 50321
AppSec [candidate] (50.499 ms) : 0, 50499
Remote Config [baseline] (704.402 µs) : 0, 704
Remote Config [candidate] (706.717 µs) : 0, 707
Telemetry [baseline] (7.517 ms) : 0, 7517
Telemetry [candidate] (7.532 ms) : 0, 7532
ProfilingAgent [baseline] (95.288 ms) : 0, 95288
ProfilingAgent [candidate] (96.819 ms) : 0, 96819
Profiling [baseline] (95.312 ms) : 0, 95312
Profiling [candidate] (96.843 ms) : 0, 96843
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.077 s) : 0, 1076720
Total [baseline] (8.546 s) : 0, 8545932
Agent [candidate] (1.083 s) : 0, 1082635
Total [candidate] (8.594 s) : 0, 8593926
section iast
Agent [baseline] (1.194 s) : 0, 1194307
Total [baseline] (9.003 s) : 0, 9002857
Agent [candidate] (1.198 s) : 0, 1198149
Total [candidate] (9.03 s) : 0, 9030079
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.208 s) : 0, 1207683
Total [baseline] (8.966 s) : 0, 8965782
Agent [candidate] (1.199 s) : 0, 1199471
Total [candidate] (8.98 s) : 0, 8980458
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1197456
Total [baseline] (8.991 s) : 0, 8991375
Agent [candidate] (1.198 s) : 0, 1197682
Total [candidate] (9.078 s) : 0, 9078130
gantt
title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.915 ms) : 0, 674915
BytebuddyAgent [candidate] (678.278 ms) : 0, 678278
GlobalTracer [baseline] (309.529 ms) : 0, 309529
GlobalTracer [candidate] (311.788 ms) : 0, 311788
AppSec [baseline] (49.569 ms) : 0, 49569
AppSec [candidate] (49.663 ms) : 0, 49663
Remote Config [baseline] (676.978 µs) : 0, 677
Remote Config [candidate] (665.483 µs) : 0, 665
Telemetry [baseline] (7.613 ms) : 0, 7613
Telemetry [candidate] (7.58 ms) : 0, 7580
section iast
BytebuddyAgent [baseline] (791.388 ms) : 0, 791388
BytebuddyAgent [candidate] (794.26 ms) : 0, 794260
GlobalTracer [baseline] (286.994 ms) : 0, 286994
GlobalTracer [candidate] (287.489 ms) : 0, 287489
AppSec [baseline] (49.486 ms) : 0, 49486
AppSec [candidate] (50.006 ms) : 0, 50006
Remote Config [baseline] (1.325 ms) : 0, 1325
Remote Config [candidate] (575.347 µs) : 0, 575
Telemetry [baseline] (7.397 ms) : 0, 7397
Telemetry [candidate] (6.632 ms) : 0, 6632
IAST [baseline] (23.423 ms) : 0, 23423
IAST [candidate] (24.924 ms) : 0, 24924
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.746 ms) : 0, 800746
BytebuddyAgent [candidate] (794.145 ms) : 0, 794145
GlobalTracer [baseline] (289.801 ms) : 0, 289801
GlobalTracer [candidate] (288.364 ms) : 0, 288364
AppSec [baseline] (49.618 ms) : 0, 49618
AppSec [candidate] (50.319 ms) : 0, 50319
Remote Config [baseline] (568.006 µs) : 0, 568
Remote Config [candidate] (576.911 µs) : 0, 577
Telemetry [baseline] (8.136 ms) : 0, 8136
Telemetry [candidate] (8.088 ms) : 0, 8088
IAST [baseline] (24.227 ms) : 0, 24227
IAST [candidate] (23.505 ms) : 0, 23505
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (791.791 ms) : 0, 791791
BytebuddyAgent [candidate] (792.411 ms) : 0, 792411
GlobalTracer [baseline] (288.909 ms) : 0, 288909
GlobalTracer [candidate] (288.41 ms) : 0, 288410
AppSec [baseline] (48.248 ms) : 0, 48248
AppSec [candidate] (48.631 ms) : 0, 48631
Remote Config [baseline] (592.781 µs) : 0, 593
Remote Config [candidate] (595.646 µs) : 0, 596
Telemetry [baseline] (6.576 ms) : 0, 6576
Telemetry [candidate] (6.684 ms) : 0, 6684
IAST [baseline] (26.974 ms) : 0, 26974
IAST [candidate] (26.556 ms) : 0, 26556
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section baseline
no_agent (1.337 ms) : 1318, 1356
. : milestone, 1337,
appsec (1.695 ms) : 1670, 1720
. : milestone, 1695,
appsec_no_iast (1.719 ms) : 1696, 1743
. : milestone, 1719,
iast (1.485 ms) : 1462, 1507
. : milestone, 1485,
profiling (1.473 ms) : 1448, 1498
. : milestone, 1473,
tracing (1.469 ms) : 1445, 1493
. : milestone, 1469,
section candidate
no_agent (1.343 ms) : 1323, 1362
. : milestone, 1343,
appsec (1.732 ms) : 1708, 1755
. : milestone, 1732,
appsec_no_iast (1.712 ms) : 1688, 1736
. : milestone, 1712,
iast (1.488 ms) : 1466, 1511
. : milestone, 1488,
profiling (1.52 ms) : 1494, 1546
. : milestone, 1520,
tracing (1.487 ms) : 1462, 1511
. : milestone, 1487,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section baseline
no_agent (369.926 µs) : 350, 390
. : milestone, 370,
iast (476.752 µs) : 455, 498
. : milestone, 477,
iast_FULL (533.909 µs) : 513, 555
. : milestone, 534,
iast_GLOBAL (496.252 µs) : 475, 518
. : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (473.505 µs) : 452, 495
. : milestone, 474,
iast_INACTIVE (448.989 µs) : 428, 470
. : milestone, 449,
iast_TELEMETRY_OFF (474.136 µs) : 452, 496
. : milestone, 474,
tracing (436.269 µs) : 416, 456
. : milestone, 436,
section candidate
no_agent (365.871 µs) : 346, 386
. : milestone, 366,
iast (482.418 µs) : 461, 504
. : milestone, 482,
iast_FULL (538.093 µs) : 517, 559
. : milestone, 538,
iast_GLOBAL (495.176 µs) : 474, 517
. : milestone, 495,
iast_HARDCODED_SECRET_DISABLED (472.236 µs) : 451, 494
. : milestone, 472,
iast_INACTIVE (441.568 µs) : 421, 462
. : milestone, 442,
iast_TELEMETRY_OFF (468.783 µs) : 448, 489
. : milestone, 469,
tracing (437.694 µs) : 417, 458
. : milestone, 438,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1469
. : milestone, 1458,
appsec (2.189 ms) : 2156, 2223
. : milestone, 2189,
iast (1.869 ms) : 1835, 1904
. : milestone, 1869,
iast_GLOBAL (1.906 ms) : 1871, 1941
. : milestone, 1906,
profiling (1.833 ms) : 1800, 1867
. : milestone, 1833,
tracing (1.824 ms) : 1792, 1856
. : milestone, 1824,
section candidate
no_agent (1.457 ms) : 1445, 1468
. : milestone, 1457,
appsec (2.201 ms) : 2167, 2234
. : milestone, 2201,
iast (1.874 ms) : 1838, 1909
. : milestone, 1874,
iast_GLOBAL (1.909 ms) : 1874, 1943
. : milestone, 1909,
profiling (1.834 ms) : 1800, 1867
. : milestone, 1834,
tracing (1.823 ms) : 1791, 1855
. : milestone, 1823,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
dateFormat X
axisFormat %s
section baseline
no_agent (14.86 s) : 14860000, 14860000
. : milestone, 14860000,
appsec (15.071 s) : 15071000, 15071000
. : milestone, 15071000,
iast (19.023 s) : 19023000, 19023000
. : milestone, 19023000,
iast_GLOBAL (17.718 s) : 17718000, 17718000
. : milestone, 17718000,
profiling (14.962 s) : 14962000, 14962000
. : milestone, 14962000,
tracing (14.85 s) : 14850000, 14850000
. : milestone, 14850000,
section candidate
no_agent (15.004 s) : 15004000, 15004000
. : milestone, 15004000,
appsec (15.126 s) : 15126000, 15126000
. : milestone, 15126000,
iast (18.678 s) : 18678000, 18678000
. : milestone, 18678000,
iast_GLOBAL (17.895 s) : 17895000, 17895000
. : milestone, 17895000,
profiling (15.276 s) : 15276000, 15276000
. : milestone, 15276000,
tracing (15.078 s) : 15078000, 15078000
. : milestone, 15078000,
|
Perform redaction test at instrumentation time instead at runtime
e73535d
to
ccc2058
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to review CapturedContextInstrumentor
carefully, the logic there is complicated - in the meantime, can we test this somehow?
I have tested it locally on petclinic and profiled it |
I was thinking about automated testing for correctness of the redaction, not performance. I expected some test updates given we've changed the instrumentation logic. Do we have any unit tests for the logic in Or can we add a test case in the smoke tests (If we already have integration tests, that's good - they should cover this change - I haven't read the tests to see). |
We have already tests for in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Thanks!
// no name, no redaction | ||
addCapturedValueOf(insnList, limits); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could redact based on function name, but I'm not convinced we should - anyway, out of scope for the current PR.
What Does This Do
Perform redaction test at instrumentation time instead at runtime
Motivation
In some cases add significant overhead
Additional Notes
Jira ticket: DEBUG-2335