Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-use helpers across all instrumenters in a module #6822

Merged
merged 1 commit into from Mar 17, 2024
Merged

Re-use helpers across all instrumenters in a module #6822

merged 1 commit into from Mar 17, 2024

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Mar 15, 2024 •
edited by jira bot

Motivation

Avoids recreating the same helpers when a module contains several instrumenters.

Jira ticket: AIT-9441

@mcculls mcculls added comp: core Tracer core tag: performance Performance related changes type: refactoring labels Mar 15, 2024
@pr-commenter
Copy link

pr-commenter bot commented Mar 15, 2024 •
edited

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/reuse-instrumenter-module-helpers
git_commit_date 1710526664 1710537466
git_commit_sha 36e924e ad6828d
release_version 1.32.0-SNAPSHOT~36e924eaab 1.32.0-SNAPSHOT~ad6828d917
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710544016 1710544016
ci_job_id 461527845 461527845
ci_pipeline_id 30235807 30235807
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 13 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081327
Total [baseline] (8.566 s) : 0, 8566273
Agent [candidate] (1.091 s) : 0, 1090983
Total [candidate] (8.638 s) : 0, 8637802
section iast
Agent [baseline] (1.204 s) : 0, 1204024
Total [baseline] (9.058 s) : 0, 9057995
Agent [candidate] (1.215 s) : 0, 1215352
Total [candidate] (9.1 s) : 0, 9099776
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.22 s) : 0, 1220434
Total [baseline] (9.049 s) : 0, 9048858
Agent [candidate] (1.208 s) : 0, 1207647
Total [candidate] (9.071 s) : 0, 9070511
section iast_TELEMETRY_OFF
Agent [baseline] (1.198 s) : 0, 1198493
Total [baseline] (9.046 s) : 0, 9046244
Agent [candidate] (1.201 s) : 0, 1200920
Total [candidate] (9.048 s) : 0, 9047667
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent iast 1.204 s 122.697 ms (11.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.22 s 139.107 ms (12.9%)
Agent iast_TELEMETRY_OFF 1.198 s 117.166 ms (10.8%)
Total tracing 8.566 s -
Total iast 9.058 s 491.722 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.049 s 482.585 ms (5.6%)
Total iast_TELEMETRY_OFF 9.046 s 479.972 ms (5.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.215 s 124.37 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 116.665 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.201 s 109.937 ms (10.1%)
Total tracing 8.638 s -
Total iast 9.1 s 461.974 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.071 s 432.709 ms (5.0%)
Total iast_TELEMETRY_OFF 9.048 s 409.865 ms (4.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.096 ms) : 0, 698096
BytebuddyAgent [candidate] (703.474 ms) : 0, 703474
GlobalTracer [baseline] (291.331 ms) : 0, 291331
GlobalTracer [candidate] (294.728 ms) : 0, 294728
AppSec [baseline] (49.015 ms) : 0, 49015
AppSec [candidate] (49.575 ms) : 0, 49575
Remote Config [baseline] (1.126 ms) : 0, 1126
Remote Config [candidate] (1.097 ms) : 0, 1097
Telemetry [baseline] (7.417 ms) : 0, 7417
Telemetry [candidate] (7.494 ms) : 0, 7494
section iast
BytebuddyAgent [baseline] (800.699 ms) : 0, 800699
BytebuddyAgent [candidate] (807.704 ms) : 0, 807704
GlobalTracer [baseline] (288.789 ms) : 0, 288789
GlobalTracer [candidate] (291.136 ms) : 0, 291136
AppSec [baseline] (51.546 ms) : 0, 51546
AppSec [candidate] (52.324 ms) : 0, 52324
IAST [baseline] (20.537 ms) : 0, 20537
IAST [candidate] (22.308 ms) : 0, 22308
Remote Config [baseline] (564.552 µs) : 0, 565
Remote Config [candidate] (567.723 µs) : 0, 568
Telemetry [baseline] (7.489 ms) : 0, 7489
Telemetry [candidate] (6.729 ms) : 0, 6729
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (811.675 ms) : 0, 811675
BytebuddyAgent [candidate] (802.02 ms) : 0, 802020
GlobalTracer [baseline] (291.818 ms) : 0, 291818
GlobalTracer [candidate] (289.929 ms) : 0, 289929
AppSec [baseline] (51.443 ms) : 0, 51443
AppSec [candidate] (48.932 ms) : 0, 48932
IAST [baseline] (23.354 ms) : 0, 23354
IAST [candidate] (24.471 ms) : 0, 24471
Remote Config [baseline] (584.405 µs) : 0, 584
Remote Config [candidate] (573.121 µs) : 0, 573
Telemetry [baseline] (6.745 ms) : 0, 6745
Telemetry [candidate] (7.362 ms) : 0, 7362
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (793.336 ms) : 0, 793336
BytebuddyAgent [candidate] (794.733 ms) : 0, 794733
GlobalTracer [baseline] (289.195 ms) : 0, 289195
GlobalTracer [candidate] (290.255 ms) : 0, 290255
AppSec [baseline] (48.817 ms) : 0, 48817
AppSec [candidate] (52.069 ms) : 0, 52069
IAST [baseline] (25.051 ms) : 0, 25051
IAST [candidate] (21.547 ms) : 0, 21547
Remote Config [baseline] (576.222 µs) : 0, 576
Remote Config [candidate] (578.212 µs) : 0, 578
Telemetry [baseline] (7.267 ms) : 0, 7267
Telemetry [candidate] (7.334 ms) : 0, 7334
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.087 s) : 0, 1087338
Total [baseline] (9.184 s) : 0, 9184487
Agent [candidate] (1.083 s) : 0, 1083413
Total [candidate] (9.273 s) : 0, 9272712
section appsec
Agent [baseline] (1.203 s) : 0, 1202544
Total [baseline] (9.288 s) : 0, 9287910
Agent [candidate] (1.213 s) : 0, 1213225
Total [candidate] (9.32 s) : 0, 9319515
section iast
Agent [baseline] (1.215 s) : 0, 1214807
Total [baseline] (9.387 s) : 0, 9387055
Agent [candidate] (1.218 s) : 0, 1217742
Total [candidate] (9.372 s) : 0, 9372064
section profiling
Agent [baseline] (1.281 s) : 0, 1280992
Total [baseline] (9.428 s) : 0, 9427896
Agent [candidate] (1.273 s) : 0, 1272927
Total [candidate] (9.371 s) : 0, 9371053
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.087 s -
Agent appsec 1.203 s 115.206 ms (10.6%)
Agent iast 1.215 s 127.469 ms (11.7%)
Agent profiling 1.281 s 193.654 ms (17.8%)
Total tracing 9.184 s -
Total appsec 9.288 s 103.423 ms (1.1%)
Total iast 9.387 s 202.568 ms (2.2%)
Total profiling 9.428 s 243.409 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.213 s 129.812 ms (12.0%)
Agent iast 1.218 s 134.329 ms (12.4%)
Agent profiling 1.273 s 189.513 ms (17.5%)
Total tracing 9.273 s -
Total appsec 9.32 s 46.803 ms (0.5%)
Total iast 9.372 s 99.352 ms (1.1%)
Total profiling 9.371 s 98.341 ms (1.1%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (701.397 ms) : 0, 701397
BytebuddyAgent [candidate] (698.68 ms) : 0, 698680
GlobalTracer [baseline] (293.521 ms) : 0, 293521
GlobalTracer [candidate] (292.615 ms) : 0, 292615
AppSec [baseline] (49.277 ms) : 0, 49277
AppSec [candidate] (49.278 ms) : 0, 49278
Remote Config [baseline] (1.081 ms) : 0, 1081
Remote Config [candidate] (1.119 ms) : 0, 1119
Telemetry [baseline] (7.513 ms) : 0, 7513
Telemetry [candidate] (7.4 ms) : 0, 7400
section appsec
BytebuddyAgent [baseline] (697.412 ms) : 0, 697412
BytebuddyAgent [candidate] (703.408 ms) : 0, 703408
GlobalTracer [baseline] (292.01 ms) : 0, 292010
GlobalTracer [candidate] (295.118 ms) : 0, 295118
AppSec [baseline] (153.467 ms) : 0, 153467
AppSec [candidate] (154.47 ms) : 0, 154470
IAST [baseline] (17.868 ms) : 0, 17868
IAST [candidate] (18.036 ms) : 0, 18036
Remote Config [baseline] (611.712 µs) : 0, 612
Remote Config [candidate] (615.82 µs) : 0, 616
Telemetry [baseline] (6.848 ms) : 0, 6848
Telemetry [candidate] (6.939 ms) : 0, 6939
section iast
BytebuddyAgent [baseline] (807.969 ms) : 0, 807969
BytebuddyAgent [candidate] (809.319 ms) : 0, 809319
GlobalTracer [baseline] (290.398 ms) : 0, 290398
GlobalTracer [candidate] (291.888 ms) : 0, 291888
AppSec [baseline] (49.847 ms) : 0, 49847
AppSec [candidate] (51.511 ms) : 0, 51511
IAST [baseline] (24.76 ms) : 0, 24760
IAST [candidate] (23.143 ms) : 0, 23143
Remote Config [baseline] (572.759 µs) : 0, 573
Remote Config [candidate] (568.637 µs) : 0, 569
Telemetry [baseline] (6.589 ms) : 0, 6589
Telemetry [candidate] (6.671 ms) : 0, 6671
section profiling
BytebuddyAgent [baseline] (693.111 ms) : 0, 693111
BytebuddyAgent [candidate] (689.182 ms) : 0, 689182
GlobalTracer [baseline] (377.542 ms) : 0, 377542
GlobalTracer [candidate] (376.039 ms) : 0, 376039
AppSec [baseline] (50.426 ms) : 0, 50426
AppSec [candidate] (49.81 ms) : 0, 49810
Remote Config [baseline] (914.769 µs) : 0, 915
Remote Config [candidate] (822.665 µs) : 0, 823
Telemetry [baseline] (7.387 ms) : 0, 7387
Telemetry [candidate] (7.315 ms) : 0, 7315
ProfilingAgent [baseline] (95.098 ms) : 0, 95098
ProfilingAgent [candidate] (93.762 ms) : 0, 93762
Profiling [baseline] (95.122 ms) : 0, 95122
Profiling [candidate] (93.785 ms) : 0, 93785

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.563 µs) : 349, 390
.   : milestone, 370,
iast (476.437 µs) : 456, 497
.   : milestone, 476,
iast_FULL (548.862 µs) : 528, 570
.   : milestone, 549,
iast_GLOBAL (519.248 µs) : 498, 541
.   : milestone, 519,
iast_HARDCODED_SECRET_DISABLED (477.201 µs) : 457, 498
.   : milestone, 477,
iast_INACTIVE (444.32 µs) : 424, 465
.   : milestone, 444,
iast_TELEMETRY_OFF (471.383 µs) : 451, 492
.   : milestone, 471,
tracing (446.966 µs) : 427, 467
.   : milestone, 447,
section candidate
no_agent (365.551 µs) : 345, 386
.   : milestone, 366,
iast (470.03 µs) : 449, 491
.   : milestone, 470,
iast_FULL (542.225 µs) : 522, 563
.   : milestone, 542,
iast_GLOBAL (489.098 µs) : 469, 509
.   : milestone, 489,
iast_HARDCODED_SECRET_DISABLED (474.231 µs) : 453, 495
.   : milestone, 474,
iast_INACTIVE (454.351 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (470.698 µs) : 450, 492
.   : milestone, 471,
tracing (444.62 µs) : 424, 465
.   : milestone, 445,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.563 µs [348.974 µs, 390.152 µs] -
iast 476.437 µs [455.53 µs, 497.344 µs] 106.874 µs (28.9%)
iast_FULL 548.862 µs [527.729 µs, 569.994 µs] 179.299 µs (48.5%)
iast_GLOBAL 519.248 µs [497.59 µs, 540.906 µs] 149.685 µs (40.5%)
iast_HARDCODED_SECRET_DISABLED 477.201 µs [456.805 µs, 497.596 µs] 107.638 µs (29.1%)
iast_INACTIVE 444.32 µs [424.087 µs, 464.554 µs] 74.758 µs (20.2%)
iast_TELEMETRY_OFF 471.383 µs [450.512 µs, 492.254 µs] 101.82 µs (27.6%)
tracing 446.966 µs [426.629 µs, 467.303 µs] 77.403 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.551 µs [344.982 µs, 386.12 µs] -
iast 470.03 µs [449.438 µs, 490.623 µs] 104.479 µs (28.6%)
iast_FULL 542.225 µs [521.72 µs, 562.73 µs] 176.674 µs (48.3%)
iast_GLOBAL 489.098 µs [468.867 µs, 509.328 µs] 123.547 µs (33.8%)
iast_HARDCODED_SECRET_DISABLED 474.231 µs [453.399 µs, 495.064 µs] 108.68 µs (29.7%)
iast_INACTIVE 454.351 µs [433.268 µs, 475.434 µs] 88.8 µs (24.3%)
iast_TELEMETRY_OFF 470.698 µs [449.836 µs, 491.56 µs] 105.147 µs (28.8%)
tracing 444.62 µs [423.863 µs, 465.377 µs] 79.069 µs (21.6%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~ad6828d917, baseline=1.32.0-SNAPSHOT~36e924eaab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.367 ms) : 1348, 1386
.   : milestone, 1367,
appsec (1.761 ms) : 1737, 1785
.   : milestone, 1761,
iast (1.53 ms) : 1506, 1553
.   : milestone, 1530,
profiling (1.512 ms) : 1488, 1535
.   : milestone, 1512,
tracing (1.529 ms) : 1505, 1552
.   : milestone, 1529,
section candidate
no_agent (1.358 ms) : 1338, 1377
.   : milestone, 1358,
appsec (1.763 ms) : 1740, 1787
.   : milestone, 1763,
iast (1.506 ms) : 1482, 1529
.   : milestone, 1506,
profiling (1.521 ms) : 1497, 1545
.   : milestone, 1521,
tracing (1.516 ms) : 1493, 1539
.   : milestone, 1516,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.367 ms [1.348 ms, 1.386 ms] -
appsec 1.761 ms [1.737 ms, 1.785 ms] 393.741 µs (28.8%)
iast 1.53 ms [1.506 ms, 1.553 ms] 162.864 µs (11.9%)
profiling 1.512 ms [1.488 ms, 1.535 ms] 144.717 µs (10.6%)
tracing 1.529 ms [1.505 ms, 1.552 ms] 161.702 µs (11.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.358 ms [1.338 ms, 1.377 ms] -
appsec 1.763 ms [1.74 ms, 1.787 ms] 405.769 µs (29.9%)
iast 1.506 ms [1.482 ms, 1.529 ms] 148.128 µs (10.9%)
profiling 1.521 ms [1.497 ms, 1.545 ms] 163.573 µs (12.0%)
tracing 1.516 ms [1.493 ms, 1.539 ms] 158.24 µs (11.7%)

@mcculls mcculls force-pushed the mcculls/reuse-instrumenter-module-helpers branch from 2ad6ed7 to ad6828d Compare March 15, 2024 21:18
@mcculls mcculls marked this pull request as ready for review March 15, 2024 22:10
@mcculls mcculls requested a review from a team as a code owner March 15, 2024 22:10
@mcculls mcculls merged commit f674b8b into master Mar 17, 2024
80 checks passed
@mcculls mcculls deleted the mcculls/reuse-instrumenter-module-helpers branch March 17, 2024 21:18
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@am312 am312 am312 approved these changes

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: core Tracer core tag: performance Performance related changes type: refactoring
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@mcculls @am312