Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable sampling for span decoration probe #6006

Merged
merged 1 commit into from
Oct 12, 2023

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Oct 9, 2023

What Does This Do

As span decoration probes are sharing the same instrumented code than Log probes, we also evaluate the sampling during the execution which does not bring any value in the case of Span decoration probes. We set the rate limiter for span decoration probe to a constant one.

Motivation

Additional Notes

Jira ticket: DEBUG-1767

@jpbempel jpbempel requested a review from a team as a code owner October 9, 2023 14:40
@jpbempel jpbempel requested review from cimi and removed request for a team October 9, 2023 14:40
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation type: bug labels Oct 9, 2023
@pr-commenter
Copy link

pr-commenter bot commented Oct 9, 2023

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.22.0-SNAPSHOT~78c8a44a72 1.22.0-SNAPSHOT~4eff6fe5af
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.014 s) : 0, 1014380
Total [baseline] (8.716 s) : 0, 8715826
Agent [candidate] (1.013 s) : 0, 1013143
Total [candidate] (8.702 s) : 0, 8702166
section iast
Agent [baseline] (1.139 s) : 0, 1139097
Total [baseline] (9.223 s) : 0, 9222906
Agent [candidate] (1.145 s) : 0, 1144856
Total [candidate] (9.261 s) : 0, 9260706
section iast_TELEMETRY_OFF
Agent [baseline] (1.133 s) : 0, 1132504
Total [baseline] (9.24 s) : 0, 9239978
Agent [candidate] (1.13 s) : 0, 1129778
Total [candidate] (9.201 s) : 0, 9201012
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.014 s -
Agent iast 1.139 s 124.717 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.133 s 118.124 ms (11.6%)
Total tracing 8.716 s -
Total iast 9.223 s 507.08 ms (5.8%)
Total iast_TELEMETRY_OFF 9.24 s 524.152 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.013 s -
Agent iast 1.145 s 131.712 ms (13.0%)
Agent iast_TELEMETRY_OFF 1.13 s 116.634 ms (11.5%)
Total tracing 8.702 s -
Total iast 9.261 s 558.54 ms (6.4%)
Total iast_TELEMETRY_OFF 9.201 s 498.846 ms (5.7%)
gantt
    title insecure-bank - break down per module: candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (631.778 ms) : 0, 631778
BytebuddyAgent [candidate] (630.286 ms) : 0, 630286
GlobalTracer [baseline] (292.83 ms) : 0, 292830
GlobalTracer [candidate] (292.563 ms) : 0, 292563
AppSec [baseline] (48.798 ms) : 0, 48798
AppSec [candidate] (49.335 ms) : 0, 49335
Remote Config [baseline] (661.876 µs) : 0, 662
Remote Config [candidate] (668.609 µs) : 0, 669
Telemetry [baseline] (6.042 ms) : 0, 6042
Telemetry [candidate] (6.035 ms) : 0, 6035
section iast
BytebuddyAgent [baseline] (759.873 ms) : 0, 759873
BytebuddyAgent [candidate] (765.674 ms) : 0, 765674
GlobalTracer [baseline] (272.706 ms) : 0, 272706
GlobalTracer [candidate] (274.248 ms) : 0, 274248
AppSec [baseline] (46.161 ms) : 0, 46161
AppSec [candidate] (46.725 ms) : 0, 46725
IAST [baseline] (18.549 ms) : 0, 18549
IAST [candidate] (16.823 ms) : 0, 16823
Remote Config [baseline] (573.259 µs) : 0, 573
Remote Config [candidate] (587.021 µs) : 0, 587
Telemetry [baseline] (6.874 ms) : 0, 6874
Telemetry [candidate] (6.318 ms) : 0, 6318
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (754.14 ms) : 0, 754140
BytebuddyAgent [candidate] (753.238 ms) : 0, 753238
GlobalTracer [baseline] (273.09 ms) : 0, 273090
GlobalTracer [candidate] (272.776 ms) : 0, 272776
AppSec [baseline] (46.519 ms) : 0, 46519
AppSec [candidate] (46.14 ms) : 0, 46140
IAST [baseline] (17.643 ms) : 0, 17643
IAST [candidate] (15.952 ms) : 0, 15952
Remote Config [baseline] (561.747 µs) : 0, 562
Remote Config [candidate] (536.493 µs) : 0, 536
Telemetry [baseline] (6.195 ms) : 0, 6195
Telemetry [candidate] (6.842 ms) : 0, 6842
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1036486
Total [baseline] (9.4 s) : 0, 9400351
Agent [candidate] (1.015 s) : 0, 1015182
Total [candidate] (9.244 s) : 0, 9243767
section appsec
Agent [baseline] (1.101 s) : 0, 1101497
Total [baseline] (9.321 s) : 0, 9320554
Agent [candidate] (1.103 s) : 0, 1102647
Total [candidate] (9.317 s) : 0, 9316898
section iast
Agent [baseline] (1.141 s) : 0, 1141377
Total [baseline] (9.489 s) : 0, 9489318
Agent [candidate] (1.141 s) : 0, 1141498
Total [candidate] (9.451 s) : 0, 9450814
section profiling
Agent [baseline] (1.194 s) : 0, 1194170
Total [baseline] (9.506 s) : 0, 9506330
Agent [candidate] (1.185 s) : 0, 1185258
Total [candidate] (9.421 s) : 0, 9420908
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.101 s 65.011 ms (6.3%)
Agent iast 1.141 s 104.89 ms (10.1%)
Agent profiling 1.194 s 157.684 ms (15.2%)
Total tracing 9.4 s -
Total appsec 9.321 s -79.798 ms (-0.8%)
Total iast 9.489 s 88.967 ms (0.9%)
Total profiling 9.506 s 105.979 ms (1.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.015 s -
Agent appsec 1.103 s 87.465 ms (8.6%)
Agent iast 1.141 s 126.315 ms (12.4%)
Agent profiling 1.185 s 170.076 ms (16.8%)
Total tracing 9.244 s -
Total appsec 9.317 s 73.131 ms (0.8%)
Total iast 9.451 s 207.048 ms (2.2%)
Total profiling 9.421 s 177.141 ms (1.9%)
gantt
    title petclinic - break down per module: candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.557 ms) : 0, 645557
BytebuddyAgent [candidate] (631.192 ms) : 0, 631192
GlobalTracer [baseline] (298.995 ms) : 0, 298995
GlobalTracer [candidate] (293.66 ms) : 0, 293660
AppSec [baseline] (49.954 ms) : 0, 49954
AppSec [candidate] (49.286 ms) : 0, 49286
Remote Config [baseline] (682.448 µs) : 0, 682
Remote Config [candidate] (654.479 µs) : 0, 654
Telemetry [baseline] (6.197 ms) : 0, 6197
Telemetry [candidate] (6.051 ms) : 0, 6051
section appsec
BytebuddyAgent [baseline] (631.96 ms) : 0, 631960
BytebuddyAgent [candidate] (631.624 ms) : 0, 631624
GlobalTracer [baseline] (291.444 ms) : 0, 291444
GlobalTracer [candidate] (292.653 ms) : 0, 292653
AppSec [baseline] (137.439 ms) : 0, 137439
AppSec [candidate] (137.781 ms) : 0, 137781
Remote Config [baseline] (659.78 µs) : 0, 660
Remote Config [candidate] (658.244 µs) : 0, 658
Telemetry [baseline] (5.674 ms) : 0, 5674
Telemetry [candidate] (5.694 ms) : 0, 5694
section iast
BytebuddyAgent [baseline] (761.307 ms) : 0, 761307
BytebuddyAgent [candidate] (760.932 ms) : 0, 760932
GlobalTracer [baseline] (272.901 ms) : 0, 272901
GlobalTracer [candidate] (273.413 ms) : 0, 273413
AppSec [baseline] (46.165 ms) : 0, 46165
AppSec [candidate] (46.146 ms) : 0, 46146
IAST [baseline] (18.579 ms) : 0, 18579
IAST [candidate] (17.283 ms) : 0, 17283
Remote Config [baseline] (564.52 µs) : 0, 565
Remote Config [candidate] (572.166 µs) : 0, 572
Telemetry [baseline] (7.571 ms) : 0, 7571
Telemetry [candidate] (8.973 ms) : 0, 8973
section profiling
BytebuddyAgent [baseline] (647.943 ms) : 0, 647943
BytebuddyAgent [candidate] (642.447 ms) : 0, 642447
GlobalTracer [baseline] (356.116 ms) : 0, 356116
GlobalTracer [candidate] (353.761 ms) : 0, 353761
AppSec [baseline] (49.036 ms) : 0, 49036
AppSec [candidate] (48.879 ms) : 0, 48879
Remote Config [baseline] (644.85 µs) : 0, 645
Remote Config [candidate] (640.029 µs) : 0, 640
Telemetry [baseline] (6.111 ms) : 0, 6111
Telemetry [candidate] (6.077 ms) : 0, 6077
ProfilingAgent [baseline] (80.772 ms) : 0, 80772
ProfilingAgent [candidate] (80.373 ms) : 0, 80373
Profiling [baseline] (80.798 ms) : 0, 80798
Profiling [candidate] (80.398 ms) : 0, 80398
Loading

Load

Parameters

Baseline Candidate
commit 1.22.0-SNAPSHOT~78c8a44a72 1.22.0-SNAPSHOT~4eff6fe5af
config baseline candidate
end_time 2023-10-10T08:24:28 2023-10-10T08:40:44
start_time 2023-10-10T08:24:15 2023-10-10T08:40:31
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.608 µs) : 350, 391
.   : milestone, 371,
iast (465.904 µs) : 445, 487
.   : milestone, 466,
iast_FULL (529.796 µs) : 509, 550
.   : milestone, 530,
iast_INACTIVE (444.074 µs) : 423, 465
.   : milestone, 444,
iast_TELEMETRY_OFF (464.517 µs) : 444, 485
.   : milestone, 465,
tracing (441.369 µs) : 420, 463
.   : milestone, 441,
section candidate
no_agent (359.324 µs) : 339, 380
.   : milestone, 359,
iast (469.76 µs) : 449, 491
.   : milestone, 470,
iast_FULL (529.25 µs) : 509, 550
.   : milestone, 529,
iast_INACTIVE (435.117 µs) : 414, 456
.   : milestone, 435,
iast_TELEMETRY_OFF (457.943 µs) : 437, 479
.   : milestone, 458,
tracing (436.902 µs) : 416, 458
.   : milestone, 437,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.608 µs [349.943 µs, 391.274 µs] -
iast 465.904 µs [445.036 µs, 486.772 µs] 95.296 µs (25.7%)
iast_FULL 529.796 µs [509.153 µs, 550.439 µs] 159.188 µs (43.0%)
iast_INACTIVE 444.074 µs [422.777 µs, 465.371 µs] 73.466 µs (19.8%)
iast_TELEMETRY_OFF 464.517 µs [443.637 µs, 485.398 µs] 93.909 µs (25.3%)
tracing 441.369 µs [419.751 µs, 462.986 µs] 70.76 µs (19.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.324 µs [338.666 µs, 379.982 µs] -
iast 469.76 µs [448.85 µs, 490.67 µs] 110.436 µs (30.7%)
iast_FULL 529.25 µs [508.632 µs, 549.868 µs] 169.926 µs (47.3%)
iast_INACTIVE 435.117 µs [413.809 µs, 456.426 µs] 75.793 µs (21.1%)
iast_TELEMETRY_OFF 457.943 µs [437.094 µs, 478.793 µs] 98.619 µs (27.4%)
tracing 436.902 µs [415.966 µs, 457.839 µs] 77.578 µs (21.6%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~4eff6fe5af, baseline=1.22.0-SNAPSHOT~78c8a44a72
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.343 ms) : 1324, 1362
.   : milestone, 1343,
appsec (1.706 ms) : 1682, 1730
.   : milestone, 1706,
iast (1.471 ms) : 1447, 1494
.   : milestone, 1471,
profiling (1.463 ms) : 1438, 1488
.   : milestone, 1463,
tracing (1.452 ms) : 1427, 1476
.   : milestone, 1452,
section candidate
no_agent (1.34 ms) : 1321, 1359
.   : milestone, 1340,
appsec (1.694 ms) : 1669, 1718
.   : milestone, 1694,
iast (1.479 ms) : 1455, 1503
.   : milestone, 1479,
profiling (1.488 ms) : 1462, 1513
.   : milestone, 1488,
tracing (1.481 ms) : 1456, 1505
.   : milestone, 1481,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.362 ms] -
appsec 1.706 ms [1.682 ms, 1.73 ms] 363.265 µs (27.1%)
iast 1.471 ms [1.447 ms, 1.494 ms] 127.758 µs (9.5%)
profiling 1.463 ms [1.438 ms, 1.488 ms] 119.761 µs (8.9%)
tracing 1.452 ms [1.427 ms, 1.476 ms] 108.775 µs (8.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.321 ms, 1.359 ms] -
appsec 1.694 ms [1.669 ms, 1.718 ms] 353.927 µs (26.4%)
iast 1.479 ms [1.455 ms, 1.503 ms] 139.69 µs (10.4%)
profiling 1.488 ms [1.462 ms, 1.513 ms] 148.089 µs (11.1%)
tracing 1.481 ms [1.456 ms, 1.505 ms] 140.958 µs (10.5%)

As span decoration probes are sharing the same instrumented code than
Log probes, we also evaluate the sampling during the execution which
does not bring any value in the case of Span decoration probes.
We set the rate limiter for span decoration probe to a constant one.
@jpbempel jpbempel force-pushed the jpbempel/disable-sampling-span-deco branch from 167ecc1 to 4eff6fe Compare October 10, 2023 08:01
Copy link
Contributor

@shatzi shatzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!

private static Sampler createSampler(double rate) {
if (rate < 0) {
return new ConstantSampler(true);
}
if (rate < 1) {
int intRate = (int) Math.round(rate * 10);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a bug here.?if rate is less then 0.05 this then intRate rate would be rounded to zero. what happen if a rate == 0 ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if rate > epsilon, we can do AdaptiveSampler(ONE_SECOND_WINDOW / rate, 1, ...) - aka, change the size of the window instead of the rate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug I don't know but yes don't support less than 0.1.
I am not even sure why I support less than 1 per second.
also, not sure about the effect to reduce the sampling window for a high rate for example, but for rate < 1 that could be more viable option.

@jpbempel jpbempel merged commit 2cf3333 into master Oct 12, 2023
67 of 69 checks passed
@jpbempel jpbempel deleted the jpbempel/disable-sampling-span-deco branch October 12, 2023 09:58
@github-actions github-actions bot added this to the 1.22.0 milestone Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: debugger Dynamic Instrumentation type: bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants