Configurable DSM bucket duration #6171

randomanderson · 2023-11-08T14:41:52Z

What Does This Do

Allows the bucket duration for datastreams monitoring to be configurable

Motivation

To support future testing. The default of 10 seconds is too slow to test things like dynamic config.

pr-commenter · 2023-11-08T15:15:30Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	landeron/dsm-bucket-duration-config
git_commit_date	1701730732	1701738601
git_commit_sha	`d1eda60`	`d47fb22`
release_version	1.25.0-SNAPSHOT~d1eda60b66	1.25.0-SNAPSHOT~d47fb2241a

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1701741264	1701741264
ci_job_id	381274888	381274888
ci_pipeline_id	24286911	24286911
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 8 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1040855
Total [baseline] (8.699 s) : 0, 8698971
Agent [candidate] (1.041 s) : 0, 1040704
Total [candidate] (8.684 s) : 0, 8683614
section iast
Agent [baseline] (1.156 s) : 0, 1155662
Total [baseline] (9.236 s) : 0, 9235610
Agent [candidate] (1.165 s) : 0, 1164517
Total [candidate] (9.249 s) : 0, 9248773
section iast_TELEMETRY_OFF
Agent [baseline] (1.149 s) : 0, 1148572
Total [baseline] (9.243 s) : 0, 9243008
Agent [candidate] (1.152 s) : 0, 1152186
Total [candidate] (9.218 s) : 0, 9218110

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	iast	1.156 s	114.807 ms (11.0%)
Agent	iast_TELEMETRY_OFF	1.149 s	107.718 ms (10.3%)
Total	tracing	8.699 s	-
Total	iast	9.236 s	536.638 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.243 s	544.036 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	iast	1.165 s	123.813 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.152 s	111.482 ms (10.7%)
Total	tracing	8.684 s	-
Total	iast	9.249 s	565.159 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.218 s	534.495 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (644.51 ms) : 0, 644510
BytebuddyAgent [candidate] (644.685 ms) : 0, 644685
GlobalTracer [baseline] (305.64 ms) : 0, 305640
GlobalTracer [candidate] (305.587 ms) : 0, 305587
AppSec [baseline] (48.797 ms) : 0, 48797
AppSec [candidate] (48.507 ms) : 0, 48507
Remote Config [baseline] (663.472 µs) : 0, 663
Remote Config [candidate] (672.954 µs) : 0, 673
Telemetry [baseline] (7.034 ms) : 0, 7034
Telemetry [candidate] (7.006 ms) : 0, 7006
section iast
BytebuddyAgent [baseline] (764.021 ms) : 0, 764021
BytebuddyAgent [candidate] (769.026 ms) : 0, 769026
GlobalTracer [baseline] (283.901 ms) : 0, 283901
GlobalTracer [candidate] (285.368 ms) : 0, 285368
AppSec [baseline] (46.125 ms) : 0, 46125
AppSec [candidate] (46.589 ms) : 0, 46589
IAST [baseline] (18.708 ms) : 0, 18708
IAST [candidate] (21.728 ms) : 0, 21728
Remote Config [baseline] (600.495 µs) : 0, 600
Remote Config [candidate] (596.766 µs) : 0, 597
Telemetry [baseline] (8.079 ms) : 0, 8079
Telemetry [candidate] (6.621 ms) : 0, 6621
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (756.545 ms) : 0, 756545
BytebuddyAgent [candidate] (758.184 ms) : 0, 758184
GlobalTracer [baseline] (284.259 ms) : 0, 284259
GlobalTracer [candidate] (285.255 ms) : 0, 285255
AppSec [baseline] (47.138 ms) : 0, 47138
AppSec [candidate] (48.64 ms) : 0, 48640
IAST [baseline] (18.843 ms) : 0, 18843
IAST [candidate] (17.441 ms) : 0, 17441
Remote Config [baseline] (581.851 µs) : 0, 582
Remote Config [candidate] (585.915 µs) : 0, 586
Telemetry [baseline] (7.041 ms) : 0, 7041
Telemetry [candidate] (7.837 ms) : 0, 7837

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039471
Total [baseline] (9.266 s) : 0, 9266032
Agent [candidate] (1.044 s) : 0, 1044325
Total [candidate] (9.265 s) : 0, 9264685
section appsec
Agent [baseline] (1.132 s) : 0, 1131544
Total [baseline] (9.422 s) : 0, 9422027
Agent [candidate] (1.13 s) : 0, 1130447
Total [candidate] (9.443 s) : 0, 9443277
section iast
Agent [baseline] (1.159 s) : 0, 1158878
Total [baseline] (9.551 s) : 0, 9550598
Agent [candidate] (1.158 s) : 0, 1158371
Total [candidate] (9.594 s) : 0, 9594009
section profiling
Agent [baseline] (1.231 s) : 0, 1231420
Total [baseline] (9.619 s) : 0, 9619181
Agent [candidate] (1.23 s) : 0, 1229805
Total [candidate] (9.591 s) : 0, 9590515

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.132 s	92.073 ms (8.9%)
Agent	iast	1.159 s	119.407 ms (11.5%)
Agent	profiling	1.231 s	191.949 ms (18.5%)
Total	tracing	9.266 s	-
Total	appsec	9.422 s	155.995 ms (1.7%)
Total	iast	9.551 s	284.566 ms (3.1%)
Total	profiling	9.619 s	353.149 ms (3.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.13 s	86.122 ms (8.2%)
Agent	iast	1.158 s	114.046 ms (10.9%)
Agent	profiling	1.23 s	185.48 ms (17.8%)
Total	tracing	9.265 s	-
Total	appsec	9.443 s	178.593 ms (1.9%)
Total	iast	9.594 s	329.324 ms (3.6%)
Total	profiling	9.591 s	325.83 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (643.496 ms) : 0, 643496
BytebuddyAgent [candidate] (645.834 ms) : 0, 645834
GlobalTracer [baseline] (305.365 ms) : 0, 305365
GlobalTracer [candidate] (307.645 ms) : 0, 307645
AppSec [baseline] (48.56 ms) : 0, 48560
AppSec [candidate] (48.716 ms) : 0, 48716
Remote Config [baseline] (674.954 µs) : 0, 675
Remote Config [candidate] (667.222 µs) : 0, 667
Telemetry [baseline] (7.117 ms) : 0, 7117
Telemetry [candidate] (7.111 ms) : 0, 7111
section appsec
BytebuddyAgent [baseline] (645.005 ms) : 0, 645005
BytebuddyAgent [candidate] (645.128 ms) : 0, 645128
GlobalTracer [baseline] (306.212 ms) : 0, 306212
GlobalTracer [candidate] (305.761 ms) : 0, 305761
AppSec [baseline] (136.806 ms) : 0, 136806
AppSec [candidate] (137.235 ms) : 0, 137235
Remote Config [baseline] (639.779 µs) : 0, 640
Remote Config [candidate] (649.204 µs) : 0, 649
Telemetry [baseline] (8.64 ms) : 0, 8640
Telemetry [candidate] (7.501 ms) : 0, 7501
section iast
BytebuddyAgent [baseline] (766.199 ms) : 0, 766199
BytebuddyAgent [candidate] (765.948 ms) : 0, 765948
GlobalTracer [baseline] (284.681 ms) : 0, 284681
GlobalTracer [candidate] (284.545 ms) : 0, 284545
AppSec [baseline] (46.384 ms) : 0, 46384
AppSec [candidate] (46.229 ms) : 0, 46229
Remote Config [baseline] (643.579 µs) : 0, 644
Remote Config [candidate] (663.662 µs) : 0, 664
Telemetry [baseline] (6.599 ms) : 0, 6599
Telemetry [candidate] (6.593 ms) : 0, 6593
IAST [baseline] (20.102 ms) : 0, 20102
IAST [candidate] (20.21 ms) : 0, 20210
section profiling
ProfilingAgent [baseline] (88.782 ms) : 0, 88782
ProfilingAgent [candidate] (88.65 ms) : 0, 88650
BytebuddyAgent [baseline] (655.479 ms) : 0, 655479
BytebuddyAgent [candidate] (654.686 ms) : 0, 654686
GlobalTracer [baseline] (376.441 ms) : 0, 376441
GlobalTracer [candidate] (376.046 ms) : 0, 376046
AppSec [baseline] (48.574 ms) : 0, 48574
AppSec [candidate] (48.374 ms) : 0, 48374
Remote Config [baseline] (686.039 µs) : 0, 686
Remote Config [candidate] (684.272 µs) : 0, 684
Telemetry [baseline] (7.318 ms) : 0, 7318
Telemetry [candidate] (7.351 ms) : 0, 7351
Profiling [baseline] (88.807 ms) : 0, 88807
Profiling [candidate] (88.676 ms) : 0, 88676

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2023-12-05T01:33:41	2023-12-05T01:50:11
git_branch	master	landeron/dsm-bucket-duration-config
git_commit_date	1701730732	1701738601
git_commit_sha	`d1eda60`	`d47fb22`
release_version	1.25.0-SNAPSHOT~d1eda60b66	1.25.0-SNAPSHOT~d47fb2241a
start_time	2023-12-05T01:33:28	2023-12-05T01:49:58

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1701741264	1701741264
ci_job_id	381274888	381274888
ci_pipeline_id	24286911	24286911
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.384 µs) : 344, 383
.   : milestone, 363,
iast (482.822 µs) : 462, 504
.   : milestone, 483,
iast_FULL (529.972 µs) : 510, 550
.   : milestone, 530,
iast_INACTIVE (440.464 µs) : 420, 461
.   : milestone, 440,
iast_TELEMETRY_OFF (468.968 µs) : 448, 490
.   : milestone, 469,
tracing (434.624 µs) : 414, 455
.   : milestone, 435,
section candidate
no_agent (364.733 µs) : 344, 385
.   : milestone, 365,
iast (468.668 µs) : 448, 489
.   : milestone, 469,
iast_FULL (533.17 µs) : 513, 554
.   : milestone, 533,
iast_INACTIVE (445.49 µs) : 425, 466
.   : milestone, 445,
iast_TELEMETRY_OFF (469.675 µs) : 448, 491
.   : milestone, 470,
tracing (442.058 µs) : 421, 463
.   : milestone, 442,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.384 µs [343.691 µs, 383.077 µs]	-
iast	482.822 µs [462.004 µs, 503.639 µs]	119.438 µs (32.9%)
iast_FULL	529.972 µs [509.728 µs, 550.216 µs]	166.588 µs (45.8%)
iast_INACTIVE	440.464 µs [419.946 µs, 460.981 µs]	77.08 µs (21.2%)
iast_TELEMETRY_OFF	468.968 µs [447.653 µs, 490.283 µs]	105.584 µs (29.1%)
tracing	434.624 µs [414.008 µs, 455.24 µs]	71.241 µs (19.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.733 µs [344.118 µs, 385.347 µs]	-
iast	468.668 µs [448.172 µs, 489.165 µs]	103.936 µs (28.5%)
iast_FULL	533.17 µs [512.834 µs, 553.505 µs]	168.437 µs (46.2%)
iast_INACTIVE	445.49 µs [424.612 µs, 466.369 µs]	80.757 µs (22.1%)
iast_TELEMETRY_OFF	469.675 µs [448.184 µs, 491.167 µs]	104.943 µs (28.8%)
tracing	442.058 µs [421.12 µs, 462.996 µs]	77.325 µs (21.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.25.0-SNAPSHOT~d47fb2241a, baseline=1.25.0-SNAPSHOT~d1eda60b66
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.741 ms) : 1716, 1767
.   : milestone, 1741,
iast (1.524 ms) : 1499, 1548
.   : milestone, 1524,
profiling (1.538 ms) : 1512, 1565
.   : milestone, 1538,
tracing (1.484 ms) : 1460, 1509
.   : milestone, 1484,
section candidate
no_agent (1.355 ms) : 1336, 1375
.   : milestone, 1355,
appsec (1.756 ms) : 1731, 1782
.   : milestone, 1756,
iast (1.521 ms) : 1496, 1545
.   : milestone, 1521,
profiling (1.513 ms) : 1488, 1539
.   : milestone, 1513,
tracing (1.493 ms) : 1469, 1518
.   : milestone, 1493,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.346 ms [1.327 ms, 1.365 ms]	-
appsec	1.741 ms [1.716 ms, 1.767 ms]	395.212 µs (29.4%)
iast	1.524 ms [1.499 ms, 1.548 ms]	177.483 µs (13.2%)
profiling	1.538 ms [1.512 ms, 1.565 ms]	192.189 µs (14.3%)
tracing	1.484 ms [1.46 ms, 1.509 ms]	138.069 µs (10.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.355 ms [1.336 ms, 1.375 ms]	-
appsec	1.756 ms [1.731 ms, 1.782 ms]	400.995 µs (29.6%)
iast	1.521 ms [1.496 ms, 1.545 ms]	165.037 µs (12.2%)
profiling	1.513 ms [1.488 ms, 1.539 ms]	157.713 µs (11.6%)
tracing	1.493 ms [1.469 ms, 1.518 ms]	137.912 µs (10.2%)

PerfectSlayer

Hi 👋
Do you plan to add this settings to the public documentation too?

internal-api/src/main/java/datadog/trace/api/Config.java

dd-trace-api/src/main/java/datadog/trace/api/ConfigDefaults.java

dd-trace-core/src/test/groovy/datadog/trace/core/datastreams/DataStreamsWritingTest.groovy

...-core/src/test/groovy/datadog/trace/core/datastreams/DefaultDataStreamsMonitoringTest.groovy

dd-trace-core/src/test/groovy/datadog/trace/core/datastreams/DefaultPathwayContextTest.groovy

…efaultPathwayContextTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

…ataStreamsWritingTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

…efaultDataStreamsMonitoringTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

…-duration-config

waiting on open issue resolution

randomanderson · 2023-11-29T15:39:55Z

Hi 👋 Do you plan to add this settings to the public documentation too?

No, it's really just for testing. Very small bucket sizes would created additional processing load on the backend

randomanderson added 2 commits November 8, 2023 09:31

Configurable DSM bucket duration

132018c

Expose a nanosecond getter to avoid conversions everywhere

36a0933

randomanderson requested a review from a team as a code owner November 8, 2023 14:41

randomanderson requested review from PerfectSlayer and am312 November 8, 2023 14:41

codenarc

70efa18

bm1549 added the comp: data streams Data Streams Monitoring label Nov 13, 2023

PerfectSlayer reviewed Nov 20, 2023

View reviewed changes

randomanderson and others added 5 commits November 21, 2023 11:15

Update dd-trace-core/src/test/groovy/datadog/trace/core/datastreams/D…

a62e19b

…efaultPathwayContextTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

Update dd-trace-core/src/test/groovy/datadog/trace/core/datastreams/D…

f8f77af

…ataStreamsWritingTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

Update dd-trace-core/src/test/groovy/datadog/trace/core/datastreams/D…

90a2f9f

…efaultDataStreamsMonitoringTest.groovy Co-authored-by: Bruce Bujon <PerfectSlayer@users.noreply.github.com>

PR comments

a2a9202

Merge remote-tracking branch 'origin/master' into landeron/dsm-bucket…

fc7d0d0

…-duration-config

am312 previously approved these changes Nov 28, 2023

View reviewed changes

am312 approved these changes Dec 4, 2023

View reviewed changes

Merge branch 'master' into landeron/dsm-bucket-duration-config

d47fb22

randomanderson merged commit f4980d2 into master Dec 5, 2023
72 checks passed

randomanderson deleted the landeron/dsm-bucket-duration-config branch December 5, 2023 18:45

github-actions bot added this to the 1.26.0 milestone Dec 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configurable DSM bucket duration #6171

Configurable DSM bucket duration #6171

randomanderson commented Nov 8, 2023

pr-commenter bot commented Nov 8, 2023 •

edited

Loading

PerfectSlayer left a comment

randomanderson commented Nov 29, 2023

Configurable DSM bucket duration #6171

Configurable DSM bucket duration #6171

Conversation

randomanderson commented Nov 8, 2023

What Does This Do

Motivation

pr-commenter bot commented Nov 8, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

PerfectSlayer left a comment

Choose a reason for hiding this comment

randomanderson commented Nov 29, 2023

pr-commenter bot commented Nov 8, 2023 •

edited

Loading