Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize Redaction detection when capturing values #6947

Merged
merged 1 commit into from
Apr 30, 2024
Merged

Optimize Redaction detection when capturing values #6947

merged 1 commit into from
Apr 30, 2024

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Apr 25, 2024 •
edited by jira bot

What Does This Do

Perform redaction test at instrumentation time instead at runtime

Motivation

In some cases add significant overhead

Additional Notes

Jira ticket: DEBUG-2335

@jpbempel jpbempel requested a review from a team as a code owner April 25, 2024 14:36
@jpbempel jpbempel requested review from cimi and removed request for a team April 25, 2024 14:36
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Apr 25, 2024
@pr-commenter
Copy link

pr-commenter bot commented Apr 25, 2024 •
edited

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/optimize-redaction-detection
git_commit_date 1714049499 1714061615
git_commit_sha ae1c4c9 ccc2058
release_version 1.34.0-SNAPSHOT~ae1c4c9475 1.34.0-SNAPSHOT~ccc20583e0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714064319 1714064319
ci_job_id 497342061 497342061
ci_pipeline_id 32938805 32938805
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081327
Total [baseline] (10.34 s) : 0, 10340164
Agent [candidate] (1.082 s) : 0, 1082403
Total [candidate] (10.495 s) : 0, 10495290
section appsec
Agent [baseline] (1.198 s) : 0, 1198191
Total [baseline] (10.479 s) : 0, 10479265
Agent [candidate] (1.188 s) : 0, 1188447
Total [candidate] (10.467 s) : 0, 10466756
section iast
Agent [baseline] (1.206 s) : 0, 1205996
Total [baseline] (10.719 s) : 0, 10718758
Agent [candidate] (1.198 s) : 0, 1197656
Total [candidate] (10.78 s) : 0, 10780241
section profiling
Agent [baseline] (1.267 s) : 0, 1266693
Total [baseline] (10.683 s) : 0, 10683190
Agent [candidate] (1.275 s) : 0, 1274673
Total [candidate] (10.597 s) : 0, 10597286
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.198 s 116.864 ms (10.8%)
Agent iast 1.206 s 124.669 ms (11.5%)
Agent profiling 1.267 s 185.366 ms (17.1%)
Total tracing 10.34 s -
Total appsec 10.479 s 139.101 ms (1.3%)
Total iast 10.719 s 378.594 ms (3.7%)
Total profiling 10.683 s 343.026 ms (3.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent appsec 1.188 s 106.044 ms (9.8%)
Agent iast 1.198 s 115.253 ms (10.6%)
Agent profiling 1.275 s 192.27 ms (17.8%)
Total tracing 10.495 s -
Total appsec 10.467 s -28.534 ms (-0.3%)
Total iast 10.78 s 284.95 ms (2.7%)
Total profiling 10.597 s 101.995 ms (1.0%) 
gantt
    title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.404 ms) : 0, 677404
BytebuddyAgent [candidate] (677.653 ms) : 0, 677653
GlobalTracer [baseline] (311.3 ms) : 0, 311300
GlobalTracer [candidate] (312.04 ms) : 0, 312040
AppSec [baseline] (49.651 ms) : 0, 49651
AppSec [candidate] (49.763 ms) : 0, 49763
Remote Config [baseline] (662.545 µs) : 0, 663
Remote Config [candidate] (667.708 µs) : 0, 668
Telemetry [baseline] (7.668 ms) : 0, 7668
Telemetry [candidate] (7.631 ms) : 0, 7631
section appsec
BytebuddyAgent [baseline] (701.49 ms) : 0, 701490
BytebuddyAgent [candidate] (694.818 ms) : 0, 694818
GlobalTracer [baseline] (293.546 ms) : 0, 293546
GlobalTracer [candidate] (291.238 ms) : 0, 291238
AppSec [baseline] (150.134 ms) : 0, 150134
AppSec [candidate] (149.586 ms) : 0, 149586
IAST [baseline] (19.536 ms) : 0, 19536
IAST [candidate] (19.306 ms) : 0, 19306
Remote Config [baseline] (635.925 µs) : 0, 636
Remote Config [candidate] (623.219 µs) : 0, 623
Telemetry [baseline] (7.918 ms) : 0, 7918
Telemetry [candidate] (8.155 ms) : 0, 8155
section iast
BytebuddyAgent [baseline] (799.261 ms) : 0, 799261
BytebuddyAgent [candidate] (793.043 ms) : 0, 793043
GlobalTracer [baseline] (289.839 ms) : 0, 289839
GlobalTracer [candidate] (287.747 ms) : 0, 287747
AppSec [baseline] (50.221 ms) : 0, 50221
AppSec [candidate] (48.258 ms) : 0, 48258
IAST [baseline] (24.186 ms) : 0, 24186
IAST [candidate] (25.454 ms) : 0, 25454
Remote Config [baseline] (569.155 µs) : 0, 569
Remote Config [candidate] (575.815 µs) : 0, 576
Telemetry [baseline] (7.379 ms) : 0, 7379
Telemetry [candidate] (8.126 ms) : 0, 8126
section profiling
BytebuddyAgent [baseline] (676.388 ms) : 0, 676388
BytebuddyAgent [candidate] (680.635 ms) : 0, 680635
GlobalTracer [baseline] (380.14 ms) : 0, 380140
GlobalTracer [candidate] (381.729 ms) : 0, 381729
AppSec [baseline] (50.321 ms) : 0, 50321
AppSec [candidate] (50.499 ms) : 0, 50499
Remote Config [baseline] (704.402 µs) : 0, 704
Remote Config [candidate] (706.717 µs) : 0, 707
Telemetry [baseline] (7.517 ms) : 0, 7517
Telemetry [candidate] (7.532 ms) : 0, 7532
ProfilingAgent [baseline] (95.288 ms) : 0, 95288
ProfilingAgent [candidate] (96.819 ms) : 0, 96819
Profiling [baseline] (95.312 ms) : 0, 95312
Profiling [candidate] (96.843 ms) : 0, 96843
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.077 s) : 0, 1076720
Total [baseline] (8.546 s) : 0, 8545932
Agent [candidate] (1.083 s) : 0, 1082635
Total [candidate] (8.594 s) : 0, 8593926
section iast
Agent [baseline] (1.194 s) : 0, 1194307
Total [baseline] (9.003 s) : 0, 9002857
Agent [candidate] (1.198 s) : 0, 1198149
Total [candidate] (9.03 s) : 0, 9030079
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.208 s) : 0, 1207683
Total [baseline] (8.966 s) : 0, 8965782
Agent [candidate] (1.199 s) : 0, 1199471
Total [candidate] (8.98 s) : 0, 8980458
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1197456
Total [baseline] (8.991 s) : 0, 8991375
Agent [candidate] (1.198 s) : 0, 1197682
Total [candidate] (9.078 s) : 0, 9078130
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.194 s 117.587 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 130.963 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.197 s 120.736 ms (11.2%)
Total tracing 8.546 s -
Total iast 9.003 s 456.926 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.966 s 419.85 ms (4.9%)
Total iast_TELEMETRY_OFF 8.991 s 445.444 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent iast 1.198 s 115.514 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.199 s 116.837 ms (10.8%)
Agent iast_TELEMETRY_OFF 1.198 s 115.047 ms (10.6%)
Total tracing 8.594 s -
Total iast 9.03 s 436.153 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 8.98 s 386.532 ms (4.5%)
Total iast_TELEMETRY_OFF 9.078 s 484.204 ms (5.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.915 ms) : 0, 674915
BytebuddyAgent [candidate] (678.278 ms) : 0, 678278
GlobalTracer [baseline] (309.529 ms) : 0, 309529
GlobalTracer [candidate] (311.788 ms) : 0, 311788
AppSec [baseline] (49.569 ms) : 0, 49569
AppSec [candidate] (49.663 ms) : 0, 49663
Remote Config [baseline] (676.978 µs) : 0, 677
Remote Config [candidate] (665.483 µs) : 0, 665
Telemetry [baseline] (7.613 ms) : 0, 7613
Telemetry [candidate] (7.58 ms) : 0, 7580
section iast
BytebuddyAgent [baseline] (791.388 ms) : 0, 791388
BytebuddyAgent [candidate] (794.26 ms) : 0, 794260
GlobalTracer [baseline] (286.994 ms) : 0, 286994
GlobalTracer [candidate] (287.489 ms) : 0, 287489
AppSec [baseline] (49.486 ms) : 0, 49486
AppSec [candidate] (50.006 ms) : 0, 50006
Remote Config [baseline] (1.325 ms) : 0, 1325
Remote Config [candidate] (575.347 µs) : 0, 575
Telemetry [baseline] (7.397 ms) : 0, 7397
Telemetry [candidate] (6.632 ms) : 0, 6632
IAST [baseline] (23.423 ms) : 0, 23423
IAST [candidate] (24.924 ms) : 0, 24924
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.746 ms) : 0, 800746
BytebuddyAgent [candidate] (794.145 ms) : 0, 794145
GlobalTracer [baseline] (289.801 ms) : 0, 289801
GlobalTracer [candidate] (288.364 ms) : 0, 288364
AppSec [baseline] (49.618 ms) : 0, 49618
AppSec [candidate] (50.319 ms) : 0, 50319
Remote Config [baseline] (568.006 µs) : 0, 568
Remote Config [candidate] (576.911 µs) : 0, 577
Telemetry [baseline] (8.136 ms) : 0, 8136
Telemetry [candidate] (8.088 ms) : 0, 8088
IAST [baseline] (24.227 ms) : 0, 24227
IAST [candidate] (23.505 ms) : 0, 23505
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (791.791 ms) : 0, 791791
BytebuddyAgent [candidate] (792.411 ms) : 0, 792411
GlobalTracer [baseline] (288.909 ms) : 0, 288909
GlobalTracer [candidate] (288.41 ms) : 0, 288410
AppSec [baseline] (48.248 ms) : 0, 48248
AppSec [candidate] (48.631 ms) : 0, 48631
Remote Config [baseline] (592.781 µs) : 0, 593
Remote Config [candidate] (595.646 µs) : 0, 596
Telemetry [baseline] (6.576 ms) : 0, 6576
Telemetry [candidate] (6.684 ms) : 0, 6684
IAST [baseline] (26.974 ms) : 0, 26974
IAST [candidate] (26.556 ms) : 0, 26556

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-04-25T16:28:49 2024-04-25T16:35:35
git_branch master jpbempel/optimize-redaction-detection
git_commit_date 1714049499 1714061615
git_commit_sha ae1c4c9 ccc2058
release_version 1.34.0-SNAPSHOT~ae1c4c9475 1.34.0-SNAPSHOT~ccc20583e0
start_time 2024-04-25T16:28:36 2024-04-25T16:35:22
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1714063279 1714063279
ci_job_id 497342062 497342062
ci_pipeline_id 32938805 32938805
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.337 ms) : 1318, 1356
.   : milestone, 1337,
appsec (1.695 ms) : 1670, 1720
.   : milestone, 1695,
appsec_no_iast (1.719 ms) : 1696, 1743
.   : milestone, 1719,
iast (1.485 ms) : 1462, 1507
.   : milestone, 1485,
profiling (1.473 ms) : 1448, 1498
.   : milestone, 1473,
tracing (1.469 ms) : 1445, 1493
.   : milestone, 1469,
section candidate
no_agent (1.343 ms) : 1323, 1362
.   : milestone, 1343,
appsec (1.732 ms) : 1708, 1755
.   : milestone, 1732,
appsec_no_iast (1.712 ms) : 1688, 1736
.   : milestone, 1712,
iast (1.488 ms) : 1466, 1511
.   : milestone, 1488,
profiling (1.52 ms) : 1494, 1546
.   : milestone, 1520,
tracing (1.487 ms) : 1462, 1511
.   : milestone, 1487,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.337 ms [1.318 ms, 1.356 ms] -
appsec 1.695 ms [1.67 ms, 1.72 ms] 358.623 µs (26.8%)
appsec_no_iast 1.719 ms [1.696 ms, 1.743 ms] 382.659 µs (28.6%)
iast 1.485 ms [1.462 ms, 1.507 ms] 147.977 µs (11.1%)
profiling 1.473 ms [1.448 ms, 1.498 ms] 136.631 µs (10.2%)
tracing 1.469 ms [1.445 ms, 1.493 ms] 132.361 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.323 ms, 1.362 ms] -
appsec 1.732 ms [1.708 ms, 1.755 ms] 389.241 µs (29.0%)
appsec_no_iast 1.712 ms [1.688 ms, 1.736 ms] 369.3 µs (27.5%)
iast 1.488 ms [1.466 ms, 1.511 ms] 145.907 µs (10.9%)
profiling 1.52 ms [1.494 ms, 1.546 ms] 177.645 µs (13.2%)
tracing 1.487 ms [1.462 ms, 1.511 ms] 144.345 µs (10.8%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.926 µs) : 350, 390
.   : milestone, 370,
iast (476.752 µs) : 455, 498
.   : milestone, 477,
iast_FULL (533.909 µs) : 513, 555
.   : milestone, 534,
iast_GLOBAL (496.252 µs) : 475, 518
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (473.505 µs) : 452, 495
.   : milestone, 474,
iast_INACTIVE (448.989 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (474.136 µs) : 452, 496
.   : milestone, 474,
tracing (436.269 µs) : 416, 456
.   : milestone, 436,
section candidate
no_agent (365.871 µs) : 346, 386
.   : milestone, 366,
iast (482.418 µs) : 461, 504
.   : milestone, 482,
iast_FULL (538.093 µs) : 517, 559
.   : milestone, 538,
iast_GLOBAL (495.176 µs) : 474, 517
.   : milestone, 495,
iast_HARDCODED_SECRET_DISABLED (472.236 µs) : 451, 494
.   : milestone, 472,
iast_INACTIVE (441.568 µs) : 421, 462
.   : milestone, 442,
iast_TELEMETRY_OFF (468.783 µs) : 448, 489
.   : milestone, 469,
tracing (437.694 µs) : 417, 458
.   : milestone, 438,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.926 µs [350.166 µs, 389.686 µs] -
iast 476.752 µs [455.085 µs, 498.42 µs] 106.826 µs (28.9%)
iast_FULL 533.909 µs [512.907 µs, 554.911 µs] 163.983 µs (44.3%)
iast_GLOBAL 496.252 µs [474.79 µs, 517.714 µs] 126.326 µs (34.1%)
iast_HARDCODED_SECRET_DISABLED 473.505 µs [451.7 µs, 495.309 µs] 103.579 µs (28.0%)
iast_INACTIVE 448.989 µs [427.91 µs, 470.068 µs] 79.063 µs (21.4%)
iast_TELEMETRY_OFF 474.136 µs [452.29 µs, 495.983 µs] 104.21 µs (28.2%)
tracing 436.269 µs [416.432 µs, 456.107 µs] 66.343 µs (17.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.871 µs [345.962 µs, 385.78 µs] -
iast 482.418 µs [461.207 µs, 503.628 µs] 116.547 µs (31.9%)
iast_FULL 538.093 µs [516.917 µs, 559.269 µs] 172.222 µs (47.1%)
iast_GLOBAL 495.176 µs [473.75 µs, 516.602 µs] 129.305 µs (35.3%)
iast_HARDCODED_SECRET_DISABLED 472.236 µs [450.547 µs, 493.926 µs] 106.365 µs (29.1%)
iast_INACTIVE 441.568 µs [420.949 µs, 462.187 µs] 75.697 µs (20.7%)
iast_TELEMETRY_OFF 468.783 µs [448.102 µs, 489.463 µs] 102.912 µs (28.1%)
tracing 437.694 µs [417.283 µs, 458.105 µs] 71.823 µs (19.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/optimize-redaction-detection
git_commit_date 1714049499 1714061615
git_commit_sha ae1c4c9 ccc2058
release_version 1.34.0-SNAPSHOT~ae1c4c9475 1.34.0-SNAPSHOT~ccc20583e0
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1714063831 1714063831
ci_job_id 497342063 497342063
ci_pipeline_id 32938805 32938805
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.458 ms) : 1447, 1469
.   : milestone, 1458,
appsec (2.189 ms) : 2156, 2223
.   : milestone, 2189,
iast (1.869 ms) : 1835, 1904
.   : milestone, 1869,
iast_GLOBAL (1.906 ms) : 1871, 1941
.   : milestone, 1906,
profiling (1.833 ms) : 1800, 1867
.   : milestone, 1833,
tracing (1.824 ms) : 1792, 1856
.   : milestone, 1824,
section candidate
no_agent (1.457 ms) : 1445, 1468
.   : milestone, 1457,
appsec (2.201 ms) : 2167, 2234
.   : milestone, 2201,
iast (1.874 ms) : 1838, 1909
.   : milestone, 1874,
iast_GLOBAL (1.909 ms) : 1874, 1943
.   : milestone, 1909,
profiling (1.834 ms) : 1800, 1867
.   : milestone, 1834,
tracing (1.823 ms) : 1791, 1855
.   : milestone, 1823,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.458 ms [1.447 ms, 1.469 ms] -
appsec 2.189 ms [2.156 ms, 2.223 ms] 731.119 µs (50.1%)
iast 1.869 ms [1.835 ms, 1.904 ms] 411.187 µs (28.2%)
iast_GLOBAL 1.906 ms [1.871 ms, 1.941 ms] 447.785 µs (30.7%)
profiling 1.833 ms [1.8 ms, 1.867 ms] 375.386 µs (25.7%)
tracing 1.824 ms [1.792 ms, 1.856 ms] 365.925 µs (25.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.457 ms [1.445 ms, 1.468 ms] -
appsec 2.201 ms [2.167 ms, 2.234 ms] 743.712 µs (51.0%)
iast 1.874 ms [1.838 ms, 1.909 ms] 416.762 µs (28.6%)
iast_GLOBAL 1.909 ms [1.874 ms, 1.943 ms] 451.824 µs (31.0%)
profiling 1.834 ms [1.8 ms, 1.867 ms] 376.903 µs (25.9%)
tracing 1.823 ms [1.791 ms, 1.855 ms] 365.793 µs (25.1%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~ccc20583e0, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
appsec (15.071 s) : 15071000, 15071000
.   : milestone, 15071000,
iast (19.023 s) : 19023000, 19023000
.   : milestone, 19023000,
iast_GLOBAL (17.718 s) : 17718000, 17718000
.   : milestone, 17718000,
profiling (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
tracing (14.85 s) : 14850000, 14850000
.   : milestone, 14850000,
section candidate
no_agent (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
appsec (15.126 s) : 15126000, 15126000
.   : milestone, 15126000,
iast (18.678 s) : 18678000, 18678000
.   : milestone, 18678000,
iast_GLOBAL (17.895 s) : 17895000, 17895000
.   : milestone, 17895000,
profiling (15.276 s) : 15276000, 15276000
.   : milestone, 15276000,
tracing (15.078 s) : 15078000, 15078000
.   : milestone, 15078000,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.86 s [14.86 s, 14.86 s] -
appsec 15.071 s [15.071 s, 15.071 s] 211.0 ms (1.4%)
iast 19.023 s [19.023 s, 19.023 s] 4.163 s (28.0%)
iast_GLOBAL 17.718 s [17.718 s, 17.718 s] 2.858 s (19.2%)
profiling 14.962 s [14.962 s, 14.962 s] 102.0 ms (0.7%)
tracing 14.85 s [14.85 s, 14.85 s] -10.0 ms (-0.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.004 s [15.004 s, 15.004 s] -
appsec 15.126 s [15.126 s, 15.126 s] 122.0 ms (0.8%)
iast 18.678 s [18.678 s, 18.678 s] 3.674 s (24.5%)
iast_GLOBAL 17.895 s [17.895 s, 17.895 s] 2.891 s (19.3%)
profiling 15.276 s [15.276 s, 15.276 s] 272.0 ms (1.8%)
tracing 15.078 s [15.078 s, 15.078 s] 74.0 ms (0.5%)

@jpbempel
Optimize Redaction detection when capturing values
ccc2058 
Perform redaction test at instrumentation time instead at runtime
@jpbempel jpbempel force-pushed the jpbempel/optimize-redaction-detection branch from e73535d to ccc2058 Compare April 25, 2024 16:13
cimi
cimi reviewed Apr 26, 2024
View reviewed changes
Copy link
Contributor

@cimi cimi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to review CapturedContextInstrumentor carefully, the logic there is complicated - in the meantime, can we test this somehow?

@jpbempel
Copy link
Member Author

I need to review CapturedContextInstrumentor carefully, the logic there is complicated - in the meantime, can we test this somehow?

I have tested it locally on petclinic and profiled it

@cimi
Copy link
Contributor

cimi commented Apr 26, 2024

I was thinking about automated testing for correctness of the redaction, not performance. I expected some test updates given we've changed the instrumentation logic.

Do we have any unit tests for the logic in CapturedContextInstrumentor.java?

Or can we add a test case in the smoke tests

Or in the captured snapshots?

(If we already have integration tests, that's good - they should cover this change - I haven't read the tests to see).

@jpbempel
Copy link
Member Author

jpbempel commented Apr 29, 2024 •
edited

I was thinking about automated testing for correctness of the redaction, not performance. I expected some test updates given we've changed the instrumentation logic.

Do we have any unit tests for the logic in CapturedContextInstrumentor.java?

Or can we add a test case in the smoke tests

Or in the captured snapshots?

(If we already have integration tests, that's good - they should cover this change - I haven't read the tests to see).

We have already tests for in

cimi
cimi approved these changes Apr 30, 2024
View reviewed changes
Copy link
Contributor

@cimi cimi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks!

...debugger/src/main/java/com/datadog/debugger/instrumentation/CapturedContextInstrumentor.java
Comment on lines +758 to 759
// no name, no redaction
addCapturedValueOf(insnList, limits);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could redact based on function name, but I'm not convinced we should - anyway, out of scope for the current PR.

@jpbempel jpbempel merged commit fcfaf54 into master Apr 30, 2024
80 checks passed
@jpbempel jpbempel deleted the jpbempel/optimize-redaction-detection branch April 30, 2024 13:06
@github-actions github-actions bot added this to the 1.34.0 milestone Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cimi cimi cimi approved these changes

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation
Projects
None yet
Milestone
1.34.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jpbempel @cimi