Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only enable byte[] or char[] call sites in full detection mode #6960

Merged
merged 1 commit into from
May 7, 2024
Merged

Only enable byte[] or char[] call sites in full detection mode #6960

merged 1 commit into from
May 7, 2024

Conversation

manuel-alvarez-alvarez
Copy link
Member

@manuel-alvarez-alvarez manuel-alvarez-alvarez commented Apr 26, 2024 •
edited by smola
Loading

What Does This Do

Moves the String call sites dealing with byte[] and char[] to a separate class that is only enabled on full mode.

Motivation

Those call sites can be specially harmful for performance so we will left them out of the default mode and only include them full detection mode is activated (DD_IAST_DETECTION_MODE=FULL).

Additional Notes

Jira ticket: [PROJ-IDENT]

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: performance Performance related changes comp: asm iast Application Security Management (IAST) labels Apr 26, 2024
@manuel-alvarez-alvarez manuel-alvarez-alvarez changed the base branch from master to malvarez/iast-refactor-propagation-methods April 26, 2024 15:48
@manuel-alvarez-alvarez manuel-alvarez-alvarez marked this pull request as ready for review April 26, 2024 15:48
@pr-commenter
Copy link

pr-commenter bot commented Apr 26, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/iast-separate-call-sites
git_commit_date 1715058813 1715065251
git_commit_sha 1465164 c5003a6
See matching parameters
Baseline Candidate
ci_job_date 1715066801 1715066801
ci_job_id 505779927 505779927
ci_pipeline_id 33738199 33738199
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 2 metrics, 0 unstable metrics.

scenario Δ mean throughput
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce worse
[-10024.217op/s; -4323.884op/s] or [-8.149%; -3.515%]
See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce unsure
[-62494.175op/s; -5442.437op/s] or [-3.409%; -0.297%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Apr 26, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/iast-separate-call-sites
git_commit_date 1715058813 1715065251
git_commit_sha 1465164 c5003a6
release_version 1.34.0-SNAPSHOT~1465164739 1.34.0-SNAPSHOT~c5003a661f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1715068144 1715068144
ci_job_id 505779923 505779923
ci_pipeline_id 33738199 33738199
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 50 metrics, 11 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Remote Config worse
[+11.708µs; +49.244µs] or [+2.057%; +8.654%]		 599.515µs 569.039µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config worse
[+16.896µs; +54.971µs] or [+2.988%; +9.721%]		 601.401µs 565.467µs
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083222
Total [baseline] (8.561 s) : 0, 8561153
Agent [candidate] (1.079 s) : 0, 1078705
Total [candidate] (8.554 s) : 0, 8553672
section iast
Agent [baseline] (1.2 s) : 0, 1199864
Total [baseline] (8.992 s) : 0, 8991509
Agent [candidate] (1.202 s) : 0, 1201936
Total [candidate] (9.02 s) : 0, 9019741
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.201 s) : 0, 1200532
Total [baseline] (8.952 s) : 0, 8952267
Agent [candidate] (1.211 s) : 0, 1210836
Total [candidate] (9.039 s) : 0, 9038905
section iast_TELEMETRY_OFF
Agent [baseline] (1.206 s) : 0, 1205606
Total [baseline] (9.029 s) : 0, 9029065
Agent [candidate] (1.204 s) : 0, 1204314
Total [candidate] (8.987 s) : 0, 8986555
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent iast 1.2 s 116.642 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.201 s 117.31 ms (10.8%)
Agent iast_TELEMETRY_OFF 1.206 s 122.385 ms (11.3%)
Total tracing 8.561 s -
Total iast 8.992 s 430.356 ms (5.0%)
Total iast_HARDCODED_SECRET_DISABLED 8.952 s 391.114 ms (4.6%)
Total iast_TELEMETRY_OFF 9.029 s 467.911 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.079 s -
Agent iast 1.202 s 123.231 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.211 s 132.132 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.204 s 125.61 ms (11.6%)
Total tracing 8.554 s -
Total iast 9.02 s 466.069 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.039 s 485.234 ms (5.7%)
Total iast_TELEMETRY_OFF 8.987 s 432.883 ms (5.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.292 ms) : 0, 678292
BytebuddyAgent [candidate] (674.982 ms) : 0, 674982
GlobalTracer [baseline] (312.018 ms) : 0, 312018
GlobalTracer [candidate] (311.21 ms) : 0, 311210
AppSec [baseline] (49.925 ms) : 0, 49925
AppSec [candidate] (49.762 ms) : 0, 49762
Remote Config [baseline] (669.785 µs) : 0, 670
Remote Config [candidate] (673.725 µs) : 0, 674
Telemetry [baseline] (7.69 ms) : 0, 7690
Telemetry [candidate] (7.706 ms) : 0, 7706
section iast
BytebuddyAgent [baseline] (793.762 ms) : 0, 793762
BytebuddyAgent [candidate] (794.935 ms) : 0, 794935
GlobalTracer [baseline] (289.742 ms) : 0, 289742
GlobalTracer [candidate] (291.304 ms) : 0, 291304
AppSec [baseline] (50.964 ms) : 0, 50964
AppSec [candidate] (49.97 ms) : 0, 49970
IAST [baseline] (23.993 ms) : 0, 23993
IAST [candidate] (24.167 ms) : 0, 24167
Remote Config [baseline] (569.039 µs) : 0, 569
Remote Config [candidate] (599.515 µs) : 0, 600
Telemetry [baseline] (6.536 ms) : 0, 6536
Telemetry [candidate] (6.607 ms) : 0, 6607
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (793.917 ms) : 0, 793917
BytebuddyAgent [candidate] (799.826 ms) : 0, 799826
GlobalTracer [baseline] (289.76 ms) : 0, 289760
GlobalTracer [candidate] (293.323 ms) : 0, 293323
AppSec [baseline] (50.6 ms) : 0, 50600
AppSec [candidate] (50.449 ms) : 0, 50449
IAST [baseline] (24.882 ms) : 0, 24882
IAST [candidate] (25.356 ms) : 0, 25356
Remote Config [baseline] (565.467 µs) : 0, 565
Remote Config [candidate] (601.401 µs) : 0, 601
Telemetry [baseline] (6.499 ms) : 0, 6499
Telemetry [candidate] (6.64 ms) : 0, 6640
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.139 ms) : 0, 798139
BytebuddyAgent [candidate] (795.725 ms) : 0, 795725
GlobalTracer [baseline] (291.39 ms) : 0, 291390
GlobalTracer [candidate] (292.554 ms) : 0, 292554
AppSec [baseline] (51.089 ms) : 0, 51089
AppSec [candidate] (49.741 ms) : 0, 49741
IAST [baseline] (23.204 ms) : 0, 23204
IAST [candidate] (24.627 ms) : 0, 24627
Remote Config [baseline] (589.118 µs) : 0, 589
Remote Config [candidate] (587.758 µs) : 0, 588
Telemetry [baseline] (6.64 ms) : 0, 6640
Telemetry [candidate] (6.568 ms) : 0, 6568
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1076240
Total [baseline] (10.431 s) : 0, 10430573
Agent [candidate] (1.083 s) : 0, 1083487
Total [candidate] (10.457 s) : 0, 10456976
section appsec
Agent [baseline] (1.192 s) : 0, 1191764
Total [baseline] (10.483 s) : 0, 10482902
Agent [candidate] (1.193 s) : 0, 1192602
Total [candidate] (10.499 s) : 0, 10498806
section iast
Agent [baseline] (1.203 s) : 0, 1203264
Total [baseline] (10.696 s) : 0, 10696171
Agent [candidate] (1.201 s) : 0, 1201041
Total [candidate] (10.721 s) : 0, 10721409
section profiling
Agent [baseline] (1.269 s) : 0, 1268738
Total [baseline] (10.557 s) : 0, 10556670
Agent [candidate] (1.27 s) : 0, 1270168
Total [candidate] (10.667 s) : 0, 10667218
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.076 s -
Agent appsec 1.192 s 115.523 ms (10.7%)
Agent iast 1.203 s 127.023 ms (11.8%)
Agent profiling 1.269 s 192.498 ms (17.9%)
Total tracing 10.431 s -
Total appsec 10.483 s 52.329 ms (0.5%)
Total iast 10.696 s 265.598 ms (2.5%)
Total profiling 10.557 s 126.097 ms (1.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.193 s 109.115 ms (10.1%)
Agent iast 1.201 s 117.555 ms (10.8%)
Agent profiling 1.27 s 186.681 ms (17.2%)
Total tracing 10.457 s -
Total appsec 10.499 s 41.83 ms (0.4%)
Total iast 10.721 s 264.433 ms (2.5%)
Total profiling 10.667 s 210.242 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.105 ms) : 0, 674105
BytebuddyAgent [candidate] (678.479 ms) : 0, 678479
GlobalTracer [baseline] (310.103 ms) : 0, 310103
GlobalTracer [candidate] (312.679 ms) : 0, 312679
AppSec [baseline] (49.478 ms) : 0, 49478
AppSec [candidate] (49.619 ms) : 0, 49619
Remote Config [baseline] (663.884 µs) : 0, 664
Remote Config [candidate] (669.044 µs) : 0, 669
Telemetry [baseline] (7.548 ms) : 0, 7548
Telemetry [candidate] (7.56 ms) : 0, 7560
section appsec
BytebuddyAgent [baseline] (696.617 ms) : 0, 696617
BytebuddyAgent [candidate] (696.341 ms) : 0, 696341
GlobalTracer [baseline] (292.986 ms) : 0, 292986
GlobalTracer [candidate] (293.821 ms) : 0, 293821
AppSec [baseline] (149.073 ms) : 0, 149073
AppSec [candidate] (149.157 ms) : 0, 149157
IAST [baseline] (19.258 ms) : 0, 19258
IAST [candidate] (19.386 ms) : 0, 19386
Remote Config [baseline] (617.837 µs) : 0, 618
Remote Config [candidate] (616.525 µs) : 0, 617
Telemetry [baseline] (8.464 ms) : 0, 8464
Telemetry [candidate] (8.538 ms) : 0, 8538
section iast
BytebuddyAgent [baseline] (797.805 ms) : 0, 797805
BytebuddyAgent [candidate] (793.394 ms) : 0, 793394
GlobalTracer [baseline] (289.446 ms) : 0, 289446
GlobalTracer [candidate] (291.254 ms) : 0, 291254
AppSec [baseline] (50.945 ms) : 0, 50945
AppSec [candidate] (50.206 ms) : 0, 50206
IAST [baseline] (23.33 ms) : 0, 23330
IAST [candidate] (23.296 ms) : 0, 23296
Remote Config [baseline] (563.857 µs) : 0, 564
Remote Config [candidate] (1.284 ms) : 0, 1284
Telemetry [baseline] (6.558 ms) : 0, 6558
Telemetry [candidate] (7.275 ms) : 0, 7275
section profiling
BytebuddyAgent [baseline] (677.348 ms) : 0, 677348
BytebuddyAgent [candidate] (677.69 ms) : 0, 677690
GlobalTracer [baseline] (380.836 ms) : 0, 380836
GlobalTracer [candidate] (381.737 ms) : 0, 381737
AppSec [baseline] (50.256 ms) : 0, 50256
AppSec [candidate] (50.312 ms) : 0, 50312
Remote Config [baseline] (721.52 µs) : 0, 722
Remote Config [candidate] (708.623 µs) : 0, 709
Telemetry [baseline] (7.475 ms) : 0, 7475
Telemetry [candidate] (7.434 ms) : 0, 7434
ProfilingAgent [baseline] (95.723 ms) : 0, 95723
ProfilingAgent [candidate] (95.801 ms) : 0, 95801
Profiling [baseline] (95.747 ms) : 0, 95747
Profiling [candidate] (95.827 ms) : 0, 95827
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-05-07T07:20:03 2024-05-07T07:26:54
git_branch master malvarez/iast-separate-call-sites
git_commit_date 1715058813 1715065251
git_commit_sha 1465164 c5003a6
release_version 1.34.0-SNAPSHOT~1465164739 1.34.0-SNAPSHOT~c5003a661f
start_time 2024-05-07T07:19:50 2024-05-07T07:26:41
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1715067158 1715067158
ci_job_id 505779924 505779924
ci_pipeline_id 33738199 33738199
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-118.789µs; -64.728µs] or [-7.484%; -4.078%]		 unstable
[-459.074op/s; +686.995op/s] or [-15.494%; +23.186%]		 1.495ms 3076.923op/s 1.587ms 2962.963op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.857 µs) : 348, 387
.   : milestone, 368,
iast (478.531 µs) : 458, 499
.   : milestone, 479,
iast_FULL (545.713 µs) : 525, 566
.   : milestone, 546,
iast_GLOBAL (496.517 µs) : 476, 517
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (470.257 µs) : 450, 491
.   : milestone, 470,
iast_INACTIVE (453.635 µs) : 432, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (473.771 µs) : 452, 495
.   : milestone, 474,
tracing (441.183 µs) : 421, 461
.   : milestone, 441,
section candidate
no_agent (381.853 µs) : 362, 402
.   : milestone, 382,
iast (478.294 µs) : 457, 500
.   : milestone, 478,
iast_FULL (542.511 µs) : 522, 563
.   : milestone, 543,
iast_GLOBAL (500.114 µs) : 478, 522
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (475.064 µs) : 455, 495
.   : milestone, 475,
iast_INACTIVE (448.383 µs) : 428, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (470.653 µs) : 449, 492
.   : milestone, 471,
tracing (454.423 µs) : 433, 475
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.857 µs [348.355 µs, 387.358 µs] -
iast 478.531 µs [458.008 µs, 499.053 µs] 110.674 µs (30.1%)
iast_FULL 545.713 µs [525.052 µs, 566.375 µs] 177.856 µs (48.3%)
iast_GLOBAL 496.517 µs [475.765 µs, 517.268 µs] 128.66 µs (35.0%)
iast_HARDCODED_SECRET_DISABLED 470.257 µs [449.677 µs, 490.837 µs] 102.401 µs (27.8%)
iast_INACTIVE 453.635 µs [432.228 µs, 475.042 µs] 85.779 µs (23.3%)
iast_TELEMETRY_OFF 473.771 µs [452.285 µs, 495.257 µs] 105.914 µs (28.8%)
tracing 441.183 µs [421.166 µs, 461.199 µs] 73.326 µs (19.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.853 µs [362.201 µs, 401.505 µs] -
iast 478.294 µs [456.915 µs, 499.673 µs] 96.441 µs (25.3%)
iast_FULL 542.511 µs [521.658 µs, 563.364 µs] 160.658 µs (42.1%)
iast_GLOBAL 500.114 µs [478.407 µs, 521.821 µs] 118.261 µs (31.0%)
iast_HARDCODED_SECRET_DISABLED 475.064 µs [454.718 µs, 495.411 µs] 93.211 µs (24.4%)
iast_INACTIVE 448.383 µs [427.818 µs, 468.947 µs] 66.53 µs (17.4%)
iast_TELEMETRY_OFF 470.653 µs [449.433 µs, 491.872 µs] 88.8 µs (23.3%)
tracing 454.423 µs [433.433 µs, 475.412 µs] 72.569 µs (19.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.343 ms) : 1324, 1362
.   : milestone, 1343,
appsec (1.729 ms) : 1705, 1754
.   : milestone, 1729,
appsec_no_iast (1.724 ms) : 1699, 1748
.   : milestone, 1724,
iast (1.488 ms) : 1465, 1511
.   : milestone, 1488,
profiling (1.587 ms) : 1562, 1612
.   : milestone, 1587,
tracing (1.465 ms) : 1440, 1490
.   : milestone, 1465,
section candidate
no_agent (1.358 ms) : 1338, 1377
.   : milestone, 1358,
appsec (1.701 ms) : 1676, 1725
.   : milestone, 1701,
appsec_no_iast (1.744 ms) : 1719, 1769
.   : milestone, 1744,
iast (1.491 ms) : 1468, 1513
.   : milestone, 1491,
profiling (1.495 ms) : 1470, 1521
.   : milestone, 1495,
tracing (1.486 ms) : 1462, 1511
.   : milestone, 1486,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.362 ms] -
appsec 1.729 ms [1.705 ms, 1.754 ms] 385.753 µs (28.7%)
appsec_no_iast 1.724 ms [1.699 ms, 1.748 ms] 380.288 µs (28.3%)
iast 1.488 ms [1.465 ms, 1.511 ms] 144.353 µs (10.7%)
profiling 1.587 ms [1.562 ms, 1.612 ms] 243.835 µs (18.2%)
tracing 1.465 ms [1.44 ms, 1.49 ms] 121.168 µs (9.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.358 ms [1.338 ms, 1.377 ms] -
appsec 1.701 ms [1.676 ms, 1.725 ms] 342.891 µs (25.3%)
appsec_no_iast 1.744 ms [1.719 ms, 1.769 ms] 386.245 µs (28.4%)
iast 1.491 ms [1.468 ms, 1.513 ms] 132.747 µs (9.8%)
profiling 1.495 ms [1.47 ms, 1.521 ms] 137.673 µs (10.1%)
tracing 1.486 ms [1.462 ms, 1.511 ms] 128.597 µs (9.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/iast-separate-call-sites
git_commit_date 1715058813 1715065251
git_commit_sha 1465164 c5003a6
release_version 1.34.0-SNAPSHOT~1465164739 1.34.0-SNAPSHOT~c5003a661f
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1715067654 1715067654
ci_job_id 505779925 505779925
ci_pipeline_id 33738199 33738199
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1454, 1477
.   : milestone, 1466,
appsec (2.208 ms) : 2173, 2242
.   : milestone, 2208,
iast (1.886 ms) : 1851, 1921
.   : milestone, 1886,
iast_GLOBAL (1.917 ms) : 1882, 1951
.   : milestone, 1917,
profiling (1.852 ms) : 1819, 1885
.   : milestone, 1852,
tracing (1.832 ms) : 1800, 1864
.   : milestone, 1832,
section candidate
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.2 ms) : 2166, 2234
.   : milestone, 2200,
iast (1.889 ms) : 1854, 1925
.   : milestone, 1889,
iast_GLOBAL (1.926 ms) : 1890, 1961
.   : milestone, 1926,
profiling (1.858 ms) : 1824, 1892
.   : milestone, 1858,
tracing (1.828 ms) : 1796, 1860
.   : milestone, 1828,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.454 ms, 1.477 ms] -
appsec 2.208 ms [2.173 ms, 2.242 ms] 741.734 µs (50.6%)
iast 1.886 ms [1.851 ms, 1.921 ms] 420.388 µs (28.7%)
iast_GLOBAL 1.917 ms [1.882 ms, 1.951 ms] 450.738 µs (30.8%)
profiling 1.852 ms [1.819 ms, 1.885 ms] 385.816 µs (26.3%)
tracing 1.832 ms [1.8 ms, 1.864 ms] 366.044 µs (25.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.457 ms, 1.48 ms] -
appsec 2.2 ms [2.166 ms, 2.234 ms] 730.903 µs (49.8%)
iast 1.889 ms [1.854 ms, 1.925 ms] 420.472 µs (28.6%)
iast_GLOBAL 1.926 ms [1.89 ms, 1.961 ms] 457.066 µs (31.1%)
profiling 1.858 ms [1.824 ms, 1.892 ms] 389.248 µs (26.5%)
tracing 1.828 ms [1.796 ms, 1.86 ms] 359.508 µs (24.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~c5003a661f, baseline=1.34.0-SNAPSHOT~1465164739
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.994 s) : 14994000, 14994000
.   : milestone, 14994000,
appsec (15.134 s) : 15134000, 15134000
.   : milestone, 15134000,
iast (18.835 s) : 18835000, 18835000
.   : milestone, 18835000,
iast_GLOBAL (17.914 s) : 17914000, 17914000
.   : milestone, 17914000,
profiling (15.904 s) : 15904000, 15904000
.   : milestone, 15904000,
tracing (15.253 s) : 15253000, 15253000
.   : milestone, 15253000,
section candidate
no_agent (15.488 s) : 15488000, 15488000
.   : milestone, 15488000,
appsec (14.907 s) : 14907000, 14907000
.   : milestone, 14907000,
iast (19.052 s) : 19052000, 19052000
.   : milestone, 19052000,
iast_GLOBAL (17.839 s) : 17839000, 17839000
.   : milestone, 17839000,
profiling (15.359 s) : 15359000, 15359000
.   : milestone, 15359000,
tracing (14.983 s) : 14983000, 14983000
.   : milestone, 14983000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.994 s [14.994 s, 14.994 s] -
appsec 15.134 s [15.134 s, 15.134 s] 140.0 ms (0.9%)
iast 18.835 s [18.835 s, 18.835 s] 3.841 s (25.6%)
iast_GLOBAL 17.914 s [17.914 s, 17.914 s] 2.92 s (19.5%)
profiling 15.904 s [15.904 s, 15.904 s] 910.0 ms (6.1%)
tracing 15.253 s [15.253 s, 15.253 s] 259.0 ms (1.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.488 s [15.488 s, 15.488 s] -
appsec 14.907 s [14.907 s, 14.907 s] -581.0 ms (-3.8%)
iast 19.052 s [19.052 s, 19.052 s] 3.564 s (23.0%)
iast_GLOBAL 17.839 s [17.839 s, 17.839 s] 2.351 s (15.2%)
profiling 15.359 s [15.359 s, 15.359 s] -129.0 ms (-0.8%)
tracing 14.983 s [14.983 s, 14.983 s] -505.0 ms (-3.3%)

@pr-commenter
Copy link

pr-commenter bot commented Apr 26, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/iast-separate-call-sites
git_commit_date 1715058813 1715065251
git_commit_sha 1465164 c5003a6
See matching parameters
Baseline Candidate
ci_job_date 1715066842 1715066842
ci_job_id 505779928 505779928
ci_pipeline_id 33738199 33738199
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

PerfectSlayer
PerfectSlayer approved these changes Apr 29, 2024
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the internal-api part. I let IAST features to the IAST team 😉

@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-refactor-propagation-methods branch 4 times, most recently from d30675f to 7ee59e4 Compare April 30, 2024 07:26
Base automatically changed from malvarez/iast-refactor-propagation-methods to master April 30, 2024 08:44
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-separate-call-sites branch 2 times, most recently from e0323b4 to d2b3dbc Compare April 30, 2024 11:23
ygree
ygree approved these changes May 6, 2024
View reviewed changes
Copy link
Contributor

@ygree ygree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@manuel-alvarez-alvarez manuel-alvarez-alvarez mentioned this pull request May 7, 2024
Merged
@manuel-alvarez-alvarez manuel-alvarez-alvarez merged commit 7461cf4 into master May 7, 2024
82 checks passed
@manuel-alvarez-alvarez manuel-alvarez-alvarez deleted the malvarez/iast-separate-call-sites branch May 7, 2024 08:58
@github-actions github-actions bot added this to the 1.34.0 milestone May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ValentinZakharov ValentinZakharov ValentinZakharov approved these changes

@ygree ygree ygree approved these changes

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@smola smola smola approved these changes

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST) tag: performance Performance related changes
Projects
None yet
Milestone
1.34.0
Development

Successfully merging this pull request may close these issues.
5 participants
@manuel-alvarez-alvarez @smola @PerfectSlayer @ygree @ValentinZakharov