Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add more tests about OpenTelemetry attributes conventions #7163

Merged
merged 1 commit into from
Jun 17, 2024
Merged

Add more tests about OpenTelemetry attributes conventions #7163

merged 1 commit into from
Jun 17, 2024

Conversation

PerfectSlayer
Copy link
Contributor

What Does This Do

This PR adds more tests about the OpenTelemetry attributes conventions.

Motivation

This is a follow up from the discussion on #7138

Additional Notes

Jira ticket: [PROJ-IDENT]

@PerfectSlayer PerfectSlayer added the inst: opentelemetry OpenTelemetry instrumentation label Jun 11, 2024
@PerfectSlayer PerfectSlayer requested a review from a team as a code owner June 11, 2024 15:25
@PerfectSlayer PerfectSlayer requested review from mcculls and removed request for a team June 11, 2024 15:25
@pr-commenter
Copy link

pr-commenter bot commented Jun 11, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bbujon/otel-attributes-test
git_commit_date 1718118542 1718119380
git_commit_sha 54461ee 129b372
release_version 1.36.0-SNAPSHOT~54461eefea 1.36.0-SNAPSHOT~129b37289d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1718122325 1718122325
ci_job_id 539460621 539460621
ci_pipeline_id 36412871 36412871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 18 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063097
Total [baseline] (10.457 s) : 0, 10457282
Agent [candidate] (1.062 s) : 0, 1062035
Total [candidate] (10.429 s) : 0, 10428992
section appsec
Agent [baseline] (1.179 s) : 0, 1179403
Total [baseline] (10.507 s) : 0, 10507087
Agent [candidate] (1.184 s) : 0, 1183805
Total [candidate] (10.474 s) : 0, 10474379
section iast
Agent [baseline] (1.167 s) : 0, 1167353
Total [baseline] (10.832 s) : 0, 10831907
Agent [candidate] (1.186 s) : 0, 1185673
Total [candidate] (10.805 s) : 0, 10804577
section profiling
Agent [baseline] (1.266 s) : 0, 1266261
Total [baseline] (10.6 s) : 0, 10599796
Agent [candidate] (1.269 s) : 0, 1269105
Total [candidate] (10.665 s) : 0, 10664808
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.179 s 116.306 ms (10.9%)
Agent iast 1.167 s 104.256 ms (9.8%)
Agent profiling 1.266 s 203.164 ms (19.1%)
Total tracing 10.457 s -
Total appsec 10.507 s 49.805 ms (0.5%)
Total iast 10.832 s 374.625 ms (3.6%)
Total profiling 10.6 s 142.514 ms (1.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.184 s 121.77 ms (11.5%)
Agent iast 1.186 s 123.638 ms (11.6%)
Agent profiling 1.269 s 207.07 ms (19.5%)
Total tracing 10.429 s -
Total appsec 10.474 s 45.387 ms (0.4%)
Total iast 10.805 s 375.585 ms (3.6%)
Total profiling 10.665 s 235.816 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.745 ms) : 0, 665745
BytebuddyAgent [candidate] (665.006 ms) : 0, 665006
GlobalTracer [baseline] (303.977 ms) : 0, 303977
GlobalTracer [candidate] (304.051 ms) : 0, 304051
AppSec [baseline] (50.206 ms) : 0, 50206
AppSec [candidate] (49.935 ms) : 0, 49935
Logs Intake [baseline] (386.973 µs) : 0, 387
Logs Intake [candidate] (399.4 µs) : 0, 399
Remote Config [baseline] (694.446 µs) : 0, 694
Remote Config [candidate] (661.277 µs) : 0, 661
Telemetry [baseline] (7.645 ms) : 0, 7645
Telemetry [candidate] (7.556 ms) : 0, 7556
section appsec
BytebuddyAgent [baseline] (675.438 ms) : 0, 675438
BytebuddyAgent [candidate] (677.868 ms) : 0, 677868
GlobalTracer [baseline] (297.377 ms) : 0, 297377
GlobalTracer [candidate] (298.328 ms) : 0, 298328
AppSec [baseline] (152.834 ms) : 0, 152834
AppSec [candidate] (153.232 ms) : 0, 153232
IAST [baseline] (19.76 ms) : 0, 19760
IAST [candidate] (21.363 ms) : 0, 21363
Logs Intake [baseline] (325.892 µs) : 0, 326
Logs Intake [candidate] (322.857 µs) : 0, 323
Remote Config [baseline] (631.304 µs) : 0, 631
Remote Config [candidate] (645.238 µs) : 0, 645
Telemetry [baseline] (8.556 ms) : 0, 8556
Telemetry [candidate] (8.182 ms) : 0, 8182
section iast
BytebuddyAgent [baseline] (778.728 ms) : 0, 778728
BytebuddyAgent [candidate] (791.783 ms) : 0, 791783
GlobalTracer [baseline] (293.724 ms) : 0, 293724
GlobalTracer [candidate] (297.542 ms) : 0, 297542
AppSec [baseline] (46.648 ms) : 0, 46648
AppSec [candidate] (47.269 ms) : 0, 47269
IAST [baseline] (26.344 ms) : 0, 26344
IAST [candidate] (25.187 ms) : 0, 25187
Logs Intake [baseline] (303.712 µs) : 0, 304
Logs Intake [candidate] (311.747 µs) : 0, 312
Remote Config [baseline] (579.488 µs) : 0, 579
Remote Config [candidate] (595.239 µs) : 0, 595
Telemetry [baseline] (7.76 ms) : 0, 7760
Telemetry [candidate] (9.481 ms) : 0, 9481
section profiling
BytebuddyAgent [baseline] (667.618 ms) : 0, 667618
BytebuddyAgent [candidate] (667.707 ms) : 0, 667707
GlobalTracer [baseline] (386.643 ms) : 0, 386643
GlobalTracer [candidate] (389.16 ms) : 0, 389160
AppSec [baseline] (50.41 ms) : 0, 50410
AppSec [candidate] (51.185 ms) : 0, 51185
Logs Intake [baseline] (330.183 µs) : 0, 330
Logs Intake [candidate] (338.93 µs) : 0, 339
Remote Config [baseline] (754.708 µs) : 0, 755
Remote Config [candidate] (745.467 µs) : 0, 745
Telemetry [baseline] (7.381 ms) : 0, 7381
Telemetry [candidate] (7.406 ms) : 0, 7406
ProfilingAgent [baseline] (95.985 ms) : 0, 95985
ProfilingAgent [candidate] (95.458 ms) : 0, 95458
Profiling [baseline] (96.01 ms) : 0, 96010
Profiling [candidate] (95.483 ms) : 0, 95483
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063386
Total [baseline] (8.547 s) : 0, 8546897
Agent [candidate] (1.064 s) : 0, 1063931
Total [candidate] (8.62 s) : 0, 8619984
section iast
Agent [baseline] (1.17 s) : 0, 1169988
Total [baseline] (9.001 s) : 0, 9000521
Agent [candidate] (1.168 s) : 0, 1167549
Total [candidate] (8.983 s) : 0, 8982612
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.169 s) : 0, 1168666
Total [baseline] (8.968 s) : 0, 8967523
Agent [candidate] (1.171 s) : 0, 1170929
Total [candidate] (8.979 s) : 0, 8979187
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1169027
Total [baseline] (8.975 s) : 0, 8974788
Agent [candidate] (1.172 s) : 0, 1172442
Total [candidate] (9.056 s) : 0, 9055928
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.17 s 106.602 ms (10.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.169 s 105.28 ms (9.9%)
Agent iast_TELEMETRY_OFF 1.169 s 105.641 ms (9.9%)
Total tracing 8.547 s -
Total iast 9.001 s 453.623 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.968 s 420.625 ms (4.9%)
Total iast_TELEMETRY_OFF 8.975 s 427.89 ms (5.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.168 s 103.618 ms (9.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 106.999 ms (10.1%)
Agent iast_TELEMETRY_OFF 1.172 s 108.511 ms (10.2%)
Total tracing 8.62 s -
Total iast 8.983 s 362.628 ms (4.2%)
Total iast_HARDCODED_SECRET_DISABLED 8.979 s 359.203 ms (4.2%)
Total iast_TELEMETRY_OFF 9.056 s 435.944 ms (5.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.738 ms) : 0, 665738
BytebuddyAgent [candidate] (666.253 ms) : 0, 666253
GlobalTracer [baseline] (304.384 ms) : 0, 304384
GlobalTracer [candidate] (304.361 ms) : 0, 304361
AppSec [baseline] (50.119 ms) : 0, 50119
AppSec [candidate] (50.034 ms) : 0, 50034
Logs Intake [baseline] (411.634 µs) : 0, 412
Logs Intake [candidate] (425.831 µs) : 0, 426
Remote Config [baseline] (667.046 µs) : 0, 667
Remote Config [candidate] (678.428 µs) : 0, 678
Telemetry [baseline] (7.514 ms) : 0, 7514
Telemetry [candidate] (7.575 ms) : 0, 7575
section iast
BytebuddyAgent [baseline] (780.72 ms) : 0, 780720
BytebuddyAgent [candidate] (778.123 ms) : 0, 778123
GlobalTracer [baseline] (293.941 ms) : 0, 293941
GlobalTracer [candidate] (293.604 ms) : 0, 293604
AppSec [baseline] (46.516 ms) : 0, 46516
AppSec [candidate] (46.745 ms) : 0, 46745
IAST [baseline] (25.426 ms) : 0, 25426
IAST [candidate] (26.398 ms) : 0, 26398
Logs Intake [baseline] (302.502 µs) : 0, 303
Logs Intake [candidate] (300.147 µs) : 0, 300
Remote Config [baseline] (573.902 µs) : 0, 574
Remote Config [candidate] (569.108 µs) : 0, 569
Telemetry [baseline] (9.152 ms) : 0, 9152
Telemetry [candidate] (8.452 ms) : 0, 8452
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (778.502 ms) : 0, 778502
BytebuddyAgent [candidate] (780.322 ms) : 0, 780322
GlobalTracer [baseline] (293.863 ms) : 0, 293863
GlobalTracer [candidate] (294.511 ms) : 0, 294511
AppSec [baseline] (46.625 ms) : 0, 46625
AppSec [candidate] (46.733 ms) : 0, 46733
IAST [baseline] (27.113 ms) : 0, 27113
IAST [candidate] (26.565 ms) : 0, 26565
Logs Intake [baseline] (297.399 µs) : 0, 297
Logs Intake [candidate] (303.914 µs) : 0, 304
Remote Config [baseline] (568.916 µs) : 0, 569
Remote Config [candidate] (588.604 µs) : 0, 589
Telemetry [baseline] (8.328 ms) : 0, 8328
Telemetry [candidate] (8.509 ms) : 0, 8509
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.136 ms) : 0, 779136
BytebuddyAgent [candidate] (781.113 ms) : 0, 781113
GlobalTracer [baseline] (294.785 ms) : 0, 294785
GlobalTracer [candidate] (295.276 ms) : 0, 295276
AppSec [baseline] (50.082 ms) : 0, 50082
AppSec [candidate] (48.541 ms) : 0, 48541
IAST [baseline] (24.07 ms) : 0, 24070
IAST [candidate] (25.587 ms) : 0, 25587
Logs Intake [baseline] (295.289 µs) : 0, 295
Logs Intake [candidate] (316.795 µs) : 0, 317
Remote Config [baseline] (558.102 µs) : 0, 558
Remote Config [candidate] (576.431 µs) : 0, 576
Telemetry [baseline] (6.716 ms) : 0, 6716
Telemetry [candidate] (7.543 ms) : 0, 7543
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-11T15:44:34 2024-06-11T15:51:22
git_branch master bbujon/otel-attributes-test
git_commit_date 1718118542 1718119380
git_commit_sha 54461ee 129b372
release_version 1.36.0-SNAPSHOT~54461eefea 1.36.0-SNAPSHOT~129b37289d
start_time 2024-06-11T15:44:20 2024-06-11T15:51:09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1718121428 1718121428
ci_job_id 539460622 539460622
ci_pipeline_id 36412871 36412871
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling worse
[+32.411µs; +87.561µs] or [+2.170%; +5.861%]		 unstable
[-577.854op/s; +577.854op/s] or [-18.780%; +18.780%]		 1.554ms 3076.923op/s 1.494ms 3076.923op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1333, 1371
.   : milestone, 1352,
appsec (1.693 ms) : 1668, 1718
.   : milestone, 1693,
appsec_no_iast (1.717 ms) : 1692, 1742
.   : milestone, 1717,
iast (1.477 ms) : 1455, 1500
.   : milestone, 1477,
profiling (1.494 ms) : 1468, 1519
.   : milestone, 1494,
tracing (1.469 ms) : 1445, 1493
.   : milestone, 1469,
section candidate
no_agent (1.33 ms) : 1310, 1350
.   : milestone, 1330,
appsec (1.712 ms) : 1688, 1736
.   : milestone, 1712,
appsec_no_iast (1.721 ms) : 1697, 1746
.   : milestone, 1721,
iast (1.494 ms) : 1471, 1517
.   : milestone, 1494,
profiling (1.554 ms) : 1528, 1580
.   : milestone, 1554,
tracing (1.444 ms) : 1419, 1468
.   : milestone, 1444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.371 ms] -
appsec 1.693 ms [1.668 ms, 1.718 ms] 341.234 µs (25.2%)
appsec_no_iast 1.717 ms [1.692 ms, 1.742 ms] 365.431 µs (27.0%)
iast 1.477 ms [1.455 ms, 1.5 ms] 125.83 µs (9.3%)
profiling 1.494 ms [1.468 ms, 1.519 ms] 142.284 µs (10.5%)
tracing 1.469 ms [1.445 ms, 1.493 ms] 117.401 µs (8.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.33 ms [1.31 ms, 1.35 ms] -
appsec 1.712 ms [1.688 ms, 1.736 ms] 381.581 µs (28.7%)
appsec_no_iast 1.721 ms [1.697 ms, 1.746 ms] 391.212 µs (29.4%)
iast 1.494 ms [1.471 ms, 1.517 ms] 163.779 µs (12.3%)
profiling 1.554 ms [1.528 ms, 1.58 ms] 223.556 µs (16.8%)
tracing 1.444 ms [1.419 ms, 1.468 ms] 113.628 µs (8.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~129b37289d, baseline=1.36.0-SNAPSHOT~54461eefea
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.083 µs) : 349, 389
.   : milestone, 369,
iast (485.726 µs) : 464, 507
.   : milestone, 486,
iast_FULL (553.45 µs) : 532, 575
.   : milestone, 553,
iast_GLOBAL (505.342 µs) : 484, 527
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (483.944 µs) : 463, 505
.   : milestone, 484,
iast_INACTIVE (453.888 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (472.602 µs) : 451, 494
.   : milestone, 473,
tracing (446.151 µs) : 426, 467
.   : milestone, 446,
section candidate
no_agent (364.782 µs) : 345, 385
.   : milestone, 365,
iast (486.331 µs) : 465, 508
.   : milestone, 486,
iast_FULL (552.548 µs) : 531, 574
.   : milestone, 553,
iast_GLOBAL (510.126 µs) : 488, 532
.   : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (484.814 µs) : 463, 506
.   : milestone, 485,
iast_INACTIVE (454.993 µs) : 433, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (478.788 µs) : 457, 500
.   : milestone, 479,
tracing (439.183 µs) : 419, 460
.   : milestone, 439,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.083 µs [349.361 µs, 388.804 µs] -
iast 485.726 µs [464.445 µs, 507.008 µs] 116.644 µs (31.6%)
iast_FULL 553.45 µs [532.161 µs, 574.738 µs] 184.367 µs (50.0%)
iast_GLOBAL 505.342 µs [483.933 µs, 526.752 µs] 136.26 µs (36.9%)
iast_HARDCODED_SECRET_DISABLED 483.944 µs [462.676 µs, 505.211 µs] 114.861 µs (31.1%)
iast_INACTIVE 453.888 µs [432.774 µs, 475.003 µs] 84.806 µs (23.0%)
iast_TELEMETRY_OFF 472.602 µs [451.296 µs, 493.908 µs] 103.519 µs (28.0%)
tracing 446.151 µs [425.801 µs, 466.502 µs] 77.069 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.782 µs [344.811 µs, 384.753 µs] -
iast 486.331 µs [464.904 µs, 507.758 µs] 121.549 µs (33.3%)
iast_FULL 552.548 µs [531.48 µs, 573.616 µs] 187.766 µs (51.5%)
iast_GLOBAL 510.126 µs [488.018 µs, 532.234 µs] 145.344 µs (39.8%)
iast_HARDCODED_SECRET_DISABLED 484.814 µs [463.159 µs, 506.47 µs] 120.032 µs (32.9%)
iast_INACTIVE 454.993 µs [433.473 µs, 476.513 µs] 90.211 µs (24.7%)
iast_TELEMETRY_OFF 478.788 µs [457.268 µs, 500.308 µs] 114.006 µs (31.3%)
tracing 439.183 µs [418.582 µs, 459.785 µs] 74.401 µs (20.4%)

Dacapo

@PerfectSlayer PerfectSlayer merged commit a19c6d5 into master Jun 17, 2024
83 checks passed
@PerfectSlayer PerfectSlayer deleted the bbujon/otel-attributes-test branch June 17, 2024 11:58
@github-actions github-actions bot added this to the 1.35.1 milestone Jun 17, 2024
@mcculls mcculls modified the milestones: 1.35.1, 1.36.0 Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mcculls mcculls mcculls approved these changes

Assignees
No one assigned
Labels
inst: opentelemetry OpenTelemetry instrumentation
Projects
None yet
Milestone
1.36.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@PerfectSlayer @mcculls