Extract Spark Plan "simpleString" into traces #9645

charlesmyu · 2025-10-01T13:31:43Z

What Does This Do

Extracts the simpleString from SparkPlanInfo and places it into the generated JSON SQL Plan that is appended to traces as nodeDetailString
Updates tests to reflect this (expands the test into formatted JSON so it's easier to update them moving forwards as well)
Small maintenance on test to explicitly account for the _dd.spark.physical_plan tag we added to the spans in a previous PR

Motivation

We would like to be able to provide more details into the execution of the Spark query. The simpleString typically contains a wealth of info that can help users pinpoint where exactly in their code the stage or job corresponds to. This is typically the arguments to the low-level API call (e.g. BroadcastHashJoin, Sort, MergeJoin, etc) that Spark already surfaces in the Spark UI.

In a future, stacked PR we will improve this to attempt and parse the simpleString in known cases into a structured object.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: DJM-969

charlesmyu · 2025-10-01T13:32:07Z

Extract Spark Plan "simpleString" into traces #9645 👈 (View in Graphite)
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

rkaneriya · 2025-10-01T13:47:38Z

...t/instrumentation/spark/src/main/java/datadog/trace/instrumentation/spark/SparkSQLUtils.java

 import scala.collection.JavaConverters;

 public class SparkSQLUtils {
+  private static final Logger log = LoggerFactory.getLogger(SparkSQLUtils.class);


This doesn't seem to be used anywhere?

Ah, snuck in from up the stack - fixed! bbc21f9 (#9645)

FYI, just rebased a bit to clean the commits up a little and squish things together so that commit will no longer exist but this has been fixed!

datadog-datadog-prod-us1 · 2025-10-01T13:50:42Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 57.94% (-0.07%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: afc6103 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-10-01T14:45:49Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-969/simple-strings
git_commit_date	1759341231	1759343669
git_commit_sha	`c096994`	`afc6103`
release_version	1.54.0-SNAPSHOT~c096994488	1.54.0-SNAPSHOT~afc6103109

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1759345472	1759345472
ci_job_id	1158284371	1158284371
ci_pipeline_id	78118813	78118813
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-c4ifkh2j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-c4ifkh2j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 6 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 7 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:AppSec	better [-1.728ms; -0.861ms] or [-4.904%; -2.442%]	33.939ms	35.233ms
scenario:startup:insecure-bank:tracing:AppSec	better [-2.385ms; -0.782ms] or [-7.129%; -2.339%]	31.867ms	33.450ms
scenario:startup:petclinic:iast:AppSec	better [-1.805ms; -1.116ms] or [-5.127%; -3.171%]	33.740ms	35.201ms
scenario:startup:petclinic:profiling:AppSec	better [-2.472ms; -1.186ms] or [-7.499%; -3.597%]	31.135ms	32.964ms
scenario:startup:petclinic:profiling:Profiling	better [-8.405ms; -3.268ms] or [-7.763%; -3.018%]	102.436ms	108.272ms
scenario:startup:petclinic:tracing:AppSec	better [-1.817ms; -1.217ms] or [-5.501%; -3.684%]	31.507ms	33.023ms

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.012 s) : 0, 1012211
Total [baseline] (10.699 s) : 0, 10698718
Agent [candidate] (1.006 s) : 0, 1006469
Total [candidate] (10.677 s) : 0, 10677389
section appsec
Agent [baseline] (1.189 s) : 0, 1189144
Total [baseline] (11.094 s) : 0, 11094490
Agent [candidate] (1.184 s) : 0, 1184284
Total [candidate] (11.058 s) : 0, 11058295
section iast
Agent [baseline] (1.142 s) : 0, 1142195
Total [baseline] (10.887 s) : 0, 10887177
Agent [candidate] (1.14 s) : 0, 1139984
Total [candidate] (10.85 s) : 0, 10849780
section profiling
Agent [baseline] (1.164 s) : 0, 1164278
Total [baseline] (11.061 s) : 0, 11061425
Agent [candidate] (1.148 s) : 0, 1148268
Total [candidate] (11.035 s) : 0, 11034504

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.012 s	-
Agent	appsec	1.189 s	176.933 ms (17.5%)
Agent	iast	1.142 s	129.984 ms (12.8%)
Agent	profiling	1.164 s	152.067 ms (15.0%)
Total	tracing	10.699 s	-
Total	appsec	11.094 s	395.772 ms (3.7%)
Total	iast	10.887 s	188.459 ms (1.8%)
Total	profiling	11.061 s	362.707 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.006 s	-
Agent	appsec	1.184 s	177.816 ms (17.7%)
Agent	iast	1.14 s	133.515 ms (13.3%)
Agent	profiling	1.148 s	141.8 ms (14.1%)
Total	tracing	10.677 s	-
Total	appsec	11.058 s	380.906 ms (3.6%)
Total	iast	10.85 s	172.391 ms (1.6%)
Total	profiling	11.035 s	357.115 ms (3.3%)

gantt
    title petclinic - break down per module: candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (692.196 ms) : 0, 692196
BytebuddyAgent [candidate] (689.364 ms) : 0, 689364
GlobalTracer [baseline] (237.268 ms) : 0, 237268
GlobalTracer [candidate] (237.269 ms) : 0, 237269
AppSec [baseline] (33.023 ms) : 0, 33023
AppSec [candidate] (31.507 ms) : 0, 31507
Debugger [baseline] (6.416 ms) : 0, 6416
Debugger [candidate] (6.317 ms) : 0, 6317
Remote Config [baseline] (678.744 µs) : 0, 679
Remote Config [candidate] (675.384 µs) : 0, 675
Telemetry [baseline] (9.044 ms) : 0, 9044
Telemetry [candidate] (9.137 ms) : 0, 9137
Flare Poller [baseline] (10.85 ms) : 0, 10850
Flare Poller [candidate] (9.5 ms) : 0, 9500
section appsec
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (716.848 ms) : 0, 716848
BytebuddyAgent [candidate] (714.674 ms) : 0, 714674
GlobalTracer [baseline] (229.817 ms) : 0, 229817
GlobalTracer [candidate] (228.835 ms) : 0, 228835
IAST [baseline] (25.189 ms) : 0, 25189
IAST [candidate] (25.029 ms) : 0, 25029
AppSec [baseline] (174.819 ms) : 0, 174819
AppSec [candidate] (173.042 ms) : 0, 173042
Debugger [baseline] (6.104 ms) : 0, 6104
Debugger [candidate] (6.149 ms) : 0, 6149
Remote Config [baseline] (644.918 µs) : 0, 645
Remote Config [candidate] (624.658 µs) : 0, 625
Telemetry [baseline] (8.286 ms) : 0, 8286
Telemetry [candidate] (8.443 ms) : 0, 8443
Flare Poller [baseline] (4.771 ms) : 0, 4771
Flare Poller [candidate] (4.841 ms) : 0, 4841
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (812.035 ms) : 0, 812035
BytebuddyAgent [candidate] (810.996 ms) : 0, 810996
GlobalTracer [baseline] (226.726 ms) : 0, 226726
GlobalTracer [candidate] (226.898 ms) : 0, 226898
IAST [baseline] (26.426 ms) : 0, 26426
IAST [candidate] (26.646 ms) : 0, 26646
AppSec [baseline] (35.201 ms) : 0, 35201
AppSec [candidate] (33.74 ms) : 0, 33740
Debugger [baseline] (5.997 ms) : 0, 5997
Debugger [candidate] (6.027 ms) : 0, 6027
Remote Config [baseline] (616.165 µs) : 0, 616
Remote Config [candidate] (590.39 µs) : 0, 590
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (8.094 ms) : 0, 8094
Flare Poller [baseline] (4.314 ms) : 0, 4314
Flare Poller [candidate] (4.236 ms) : 0, 4236
section profiling
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.442 ms) : 0, 1442
BytebuddyAgent [baseline] (726.247 ms) : 0, 726247
BytebuddyAgent [candidate] (720.687 ms) : 0, 720687
GlobalTracer [baseline] (215.032 ms) : 0, 215032
GlobalTracer [candidate] (212.808 ms) : 0, 212808
AppSec [baseline] (32.964 ms) : 0, 32964
AppSec [candidate] (31.135 ms) : 0, 31135
Debugger [baseline] (6.581 ms) : 0, 6581
Debugger [candidate] (6.431 ms) : 0, 6431
Remote Config [baseline] (858.187 µs) : 0, 858
Remote Config [candidate] (709.985 µs) : 0, 710
Telemetry [baseline] (15.533 ms) : 0, 15533
Telemetry [candidate] (16.385 ms) : 0, 16385
Flare Poller [baseline] (5.005 ms) : 0, 5005
Flare Poller [candidate] (4.27 ms) : 0, 4270
ProfilingAgent [baseline] (106.178 ms) : 0, 106178
ProfilingAgent [candidate] (101.832 ms) : 0, 101832
Profiling [baseline] (108.272 ms) : 0, 108272
Profiling [candidate] (102.436 ms) : 0, 102436

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1023638
Total [baseline] (8.646 s) : 0, 8645595
Agent [candidate] (1.014 s) : 0, 1014348
Total [candidate] (8.645 s) : 0, 8645377
section iast
Agent [baseline] (1.145 s) : 0, 1144661
Total [baseline] (9.244 s) : 0, 9244303
Agent [candidate] (1.139 s) : 0, 1139164
Total [candidate] (9.236 s) : 0, 9236339

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.024 s	-
Agent	iast	1.145 s	121.022 ms (11.8%)
Total	tracing	8.646 s	-
Total	iast	9.244 s	598.708 ms (6.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.014 s	-
Agent	iast	1.139 s	124.815 ms (12.3%)
Total	tracing	8.645 s	-
Total	iast	9.236 s	590.963 ms (6.8%)

gantt
    title insecure-bank - break down per module: candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.483 ms) : 0, 1483
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (699.275 ms) : 0, 699275
BytebuddyAgent [candidate] (695.501 ms) : 0, 695501
GlobalTracer [baseline] (239.743 ms) : 0, 239743
GlobalTracer [candidate] (239.161 ms) : 0, 239161
AppSec [baseline] (33.45 ms) : 0, 33450
AppSec [candidate] (31.867 ms) : 0, 31867
Debugger [baseline] (6.496 ms) : 0, 6496
Debugger [candidate] (6.375 ms) : 0, 6375
Remote Config [baseline] (697.174 µs) : 0, 697
Remote Config [candidate] (686.687 µs) : 0, 687
Telemetry [baseline] (9.208 ms) : 0, 9208
Telemetry [candidate] (9.185 ms) : 0, 9185
Flare Poller [baseline] (11.981 ms) : 0, 11981
Flare Poller [candidate] (8.778 ms) : 0, 8778
section iast
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (814.381 ms) : 0, 814381
BytebuddyAgent [candidate] (810.489 ms) : 0, 810489
GlobalTracer [baseline] (227.022 ms) : 0, 227022
GlobalTracer [candidate] (226.43 ms) : 0, 226430
IAST [baseline] (26.313 ms) : 0, 26313
IAST [candidate] (26.55 ms) : 0, 26550
AppSec [baseline] (35.233 ms) : 0, 35233
AppSec [candidate] (33.939 ms) : 0, 33939
Debugger [baseline] (6.043 ms) : 0, 6043
Debugger [candidate] (5.991 ms) : 0, 5991
Remote Config [baseline] (585.847 µs) : 0, 586
Remote Config [candidate] (587.525 µs) : 0, 588
Telemetry [baseline] (8.108 ms) : 0, 8108
Telemetry [candidate] (8.21 ms) : 0, 8210
Flare Poller [baseline] (4.246 ms) : 0, 4246
Flare Poller [candidate] (4.276 ms) : 0, 4276

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-969/simple-strings
git_commit_date	1759341231	1759343669
git_commit_sha	`c096994`	`afc6103`
release_version	1.54.0-SNAPSHOT~c096994488	1.54.0-SNAPSHOT~afc6103109

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1759345134	1759345134
ci_job_id	1158284372	1158284372
ci_pipeline_id	78118813	78118813
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-hq0z18w2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-hq0z18w2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 3 performance regressions! Performance is the same for 6 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load	worse [+108.865µs; +226.377µs] or [+2.462%; +5.119%]	unstable [-147.401op/s; +73.338op/s] or [-14.202%; +7.066%]	4.590ms	1000.875op/s	4.422ms	1037.906op/s
scenario:load:insecure-bank:iast:high_load	worse [+376.981µs; +723.591µs] or [+4.017%; +7.710%]	unstable [-80.921op/s; +26.734op/s] or [-16.363%; +5.406%]	9.936ms	467.438op/s	9.386ms	494.531op/s
scenario:load:petclinic:no_agent:high_load	worse [+1.439ms; +2.078ms] or [+4.036%; +5.828%]	unstable [-14.726op/s; +2.426op/s] or [-11.230%; +1.850%]	37.411ms	124.975op/s	35.653ms	131.125op/s
scenario:load:petclinic:code_origins:high_load	better [-1.722ms; -0.925ms] or [-3.896%; -2.093%]	unstable [-3.790op/s; +10.265op/s] or [-3.580%; +9.698%]	42.880ms	109.088op/s	44.203ms	105.850op/s
scenario:load:petclinic:profiling:high_load	better [-2.213ms; -1.297ms] or [-4.462%; -2.615%]	unstable [-2.978op/s; +9.878op/s] or [-3.156%; +10.466%]	47.837ms	97.838op/s	49.592ms	94.388op/s
scenario:load:petclinic:iast:high_load	better [-1.981ms; -1.143ms] or [-4.229%; -2.439%]	unstable [-3.579op/s; +10.504op/s] or [-3.584%; +10.518%]	45.276ms	103.325op/s	46.838ms	99.862op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488
    dateFormat X
    axisFormat %s
section baseline
no_agent (35.653 ms) : 35366, 35940
.   : milestone, 35653,
appsec (47.131 ms) : 46730, 47531
.   : milestone, 47131,
code_origins (44.203 ms) : 43837, 44570
.   : milestone, 44203,
iast (46.838 ms) : 46439, 47236
.   : milestone, 46838,
profiling (49.592 ms) : 49167, 50016
.   : milestone, 49592,
tracing (45.787 ms) : 45394, 46180
.   : milestone, 45787,
section candidate
no_agent (37.411 ms) : 37105, 37718
.   : milestone, 37411,
appsec (46.131 ms) : 45744, 46518
.   : milestone, 46131,
code_origins (42.88 ms) : 42506, 43254
.   : milestone, 42880,
iast (45.276 ms) : 44896, 45656
.   : milestone, 45276,
profiling (47.837 ms) : 47410, 48264
.   : milestone, 47837,
tracing (44.513 ms) : 44134, 44892
.   : milestone, 44513,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	35.653 ms [35.366 ms, 35.94 ms]	-
appsec	47.131 ms [46.73 ms, 47.531 ms]	11.478 ms (32.2%)
code_origins	44.203 ms [43.837 ms, 44.57 ms]	8.55 ms (24.0%)
iast	46.838 ms [46.439 ms, 47.236 ms]	11.185 ms (31.4%)
profiling	49.592 ms [49.167 ms, 50.016 ms]	13.939 ms (39.1%)
tracing	45.787 ms [45.394 ms, 46.18 ms]	10.134 ms (28.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.411 ms [37.105 ms, 37.718 ms]	-
appsec	46.131 ms [45.744 ms, 46.518 ms]	8.719 ms (23.3%)
code_origins	42.88 ms [42.506 ms, 43.254 ms]	5.468 ms (14.6%)
iast	45.276 ms [44.896 ms, 45.656 ms]	7.865 ms (21.0%)
profiling	47.837 ms [47.41 ms, 48.264 ms]	10.425 ms (27.9%)
tracing	44.513 ms [44.134 ms, 44.892 ms]	7.101 ms (19.0%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.422 ms) : 4365, 4480
.   : milestone, 4422,
iast (9.386 ms) : 9232, 9539
.   : milestone, 9386,
iast_FULL (14.427 ms) : 14133, 14721
.   : milestone, 14427,
iast_GLOBAL (11.055 ms) : 10853, 11256
.   : milestone, 11055,
profiling (8.565 ms) : 8432, 8699
.   : milestone, 8565,
tracing (7.745 ms) : 7635, 7856
.   : milestone, 7745,
section candidate
no_agent (4.59 ms) : 4539, 4641
.   : milestone, 4590,
iast (9.936 ms) : 9767, 10104
.   : milestone, 9936,
iast_FULL (14.09 ms) : 13808, 14372
.   : milestone, 14090,
iast_GLOBAL (10.709 ms) : 10520, 10898
.   : milestone, 10709,
profiling (8.872 ms) : 8733, 9011
.   : milestone, 8872,
tracing (7.902 ms) : 7789, 8016
.   : milestone, 7902,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.422 ms [4.365 ms, 4.48 ms]	-
iast	9.386 ms [9.232 ms, 9.539 ms]	4.963 ms (112.2%)
iast_FULL	14.427 ms [14.133 ms, 14.721 ms]	10.004 ms (226.2%)
iast_GLOBAL	11.055 ms [10.853 ms, 11.256 ms]	6.632 ms (150.0%)
profiling	8.565 ms [8.432 ms, 8.699 ms]	4.143 ms (93.7%)
tracing	7.745 ms [7.635 ms, 7.856 ms]	3.323 ms (75.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.59 ms [4.539 ms, 4.641 ms]	-
iast	9.936 ms [9.767 ms, 10.104 ms]	5.346 ms (116.5%)
iast_FULL	14.09 ms [13.808 ms, 14.372 ms]	9.5 ms (207.0%)
iast_GLOBAL	10.709 ms [10.52 ms, 10.898 ms]	6.119 ms (133.3%)
profiling	8.872 ms [8.733 ms, 9.011 ms]	4.282 ms (93.3%)
tracing	7.902 ms [7.789 ms, 8.016 ms]	3.312 ms (72.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	charles.yu/djm-969/simple-strings
git_commit_date	1759341231	1759343669
git_commit_sha	`c096994`	`afc6103`
release_version	1.54.0-SNAPSHOT~c096994488	1.54.0-SNAPSHOT~afc6103109

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1759345652	1759345652
ci_job_id	1158284373	1158284373
ci_pipeline_id	78118813	78118813
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-b5d2nkxm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-b5d2nkxm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.579 s) : 15579000, 15579000
.   : milestone, 15579000,
appsec (14.952 s) : 14952000, 14952000
.   : milestone, 14952000,
iast (18.722 s) : 18722000, 18722000
.   : milestone, 18722000,
iast_GLOBAL (17.803 s) : 17803000, 17803000
.   : milestone, 17803000,
profiling (15.404 s) : 15404000, 15404000
.   : milestone, 15404000,
tracing (15.093 s) : 15093000, 15093000
.   : milestone, 15093000,
section candidate
no_agent (15.554 s) : 15554000, 15554000
.   : milestone, 15554000,
appsec (15.044 s) : 15044000, 15044000
.   : milestone, 15044000,
iast (18.682 s) : 18682000, 18682000
.   : milestone, 18682000,
iast_GLOBAL (18.308 s) : 18308000, 18308000
.   : milestone, 18308000,
profiling (15.426 s) : 15426000, 15426000
.   : milestone, 15426000,
tracing (14.935 s) : 14935000, 14935000
.   : milestone, 14935000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.579 s [15.579 s, 15.579 s]	-
appsec	14.952 s [14.952 s, 14.952 s]	-627.0 ms (-4.0%)
iast	18.722 s [18.722 s, 18.722 s]	3.143 s (20.2%)
iast_GLOBAL	17.803 s [17.803 s, 17.803 s]	2.224 s (14.3%)
profiling	15.404 s [15.404 s, 15.404 s]	-175.0 ms (-1.1%)
tracing	15.093 s [15.093 s, 15.093 s]	-486.0 ms (-3.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.554 s [15.554 s, 15.554 s]	-
appsec	15.044 s [15.044 s, 15.044 s]	-510.0 ms (-3.3%)
iast	18.682 s [18.682 s, 18.682 s]	3.128 s (20.1%)
iast_GLOBAL	18.308 s [18.308 s, 18.308 s]	2.754 s (17.7%)
profiling	15.426 s [15.426 s, 15.426 s]	-128.0 ms (-0.8%)
tracing	14.935 s [14.935 s, 14.935 s]	-619.0 ms (-4.0%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~afc6103109, baseline=1.54.0-SNAPSHOT~c096994488
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (3.763 ms) : 3543, 3984
.   : milestone, 3763,
iast (2.231 ms) : 2167, 2295
.   : milestone, 2231,
iast_GLOBAL (2.269 ms) : 2205, 2332
.   : milestone, 2269,
profiling (2.057 ms) : 2006, 2107
.   : milestone, 2057,
tracing (2.037 ms) : 1987, 2086
.   : milestone, 2037,
section candidate
no_agent (1.484 ms) : 1472, 1495
.   : milestone, 1484,
appsec (3.736 ms) : 3517, 3955
.   : milestone, 3736,
iast (2.219 ms) : 2155, 2282
.   : milestone, 2219,
iast_GLOBAL (2.266 ms) : 2202, 2330
.   : milestone, 2266,
profiling (2.067 ms) : 2016, 2118
.   : milestone, 2067,
tracing (2.03 ms) : 1981, 2080
.   : milestone, 2030,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.47 ms, 1.493 ms]	-
appsec	3.763 ms [3.543 ms, 3.984 ms]	2.282 ms (154.1%)
iast	2.231 ms [2.167 ms, 2.295 ms]	749.92 µs (50.6%)
iast_GLOBAL	2.269 ms [2.205 ms, 2.332 ms]	787.569 µs (53.2%)
profiling	2.057 ms [2.006 ms, 2.107 ms]	575.44 µs (38.9%)
tracing	2.037 ms [1.987 ms, 2.086 ms]	555.628 µs (37.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.484 ms [1.472 ms, 1.495 ms]	-
appsec	3.736 ms [3.517 ms, 3.955 ms]	2.252 ms (151.8%)
iast	2.219 ms [2.155 ms, 2.282 ms]	734.857 µs (49.5%)
iast_GLOBAL	2.266 ms [2.202 ms, 2.33 ms]	782.355 µs (52.7%)
profiling	2.067 ms [2.016 ms, 2.118 ms]	583.662 µs (39.3%)
tracing	2.03 ms [1.981 ms, 2.08 ms]	546.54 µs (36.8%)

… span

Make tests explicit for physical plan field

9c55b36

charlesmyu changed the title ~~Make tests explicit for physical plan field~~ Extract Spark Plan "simpleString" into traces Oct 1, 2025

charlesmyu added inst: apache spark Apache Spark instrumentation type: enhancement Enhancements and improvements labels Oct 1, 2025

charlesmyu marked this pull request as ready for review October 1, 2025 13:38

charlesmyu requested a review from a team as a code owner October 1, 2025 13:38

rkaneriya reviewed Oct 1, 2025

View reviewed changes

charlesmyu force-pushed the charles.yu/djm-969/simple-strings branch from bbc21f9 to bae8145 Compare October 1, 2025 18:30

Extract simpleString from Spark StagePlanInfo, add to JSON payload in…

afc6103

… span

charlesmyu force-pushed the charles.yu/djm-969/simple-strings branch from bae8145 to afc6103 Compare October 1, 2025 18:34

rkaneriya approved these changes Oct 1, 2025

View reviewed changes

charlesmyu enabled auto-merge (squash) October 1, 2025 22:29

charlesmyu merged commit 088156e into master Oct 2, 2025
395 of 396 checks passed

charlesmyu deleted the charles.yu/djm-969/simple-strings branch October 2, 2025 00:39

github-actions bot added this to the 1.54.0 milestone Oct 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Extract Spark Plan "simpleString" into traces #9645

Extract Spark Plan "simpleString" into traces #9645

charlesmyu commented Oct 1, 2025 •

edited

Loading

Uh oh!

charlesmyu commented Oct 1, 2025

Uh oh!

rkaneriya Oct 1, 2025

Uh oh!

charlesmyu Oct 1, 2025

Uh oh!

charlesmyu Oct 1, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Oct 1, 2025 •

edited by datadog-official bot

Loading

Uh oh!

pr-commenter bot commented Oct 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Extract Spark Plan "simpleString" into traces #9645

Extract Spark Plan "simpleString" into traces #9645

Conversation

charlesmyu commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

charlesmyu commented Oct 1, 2025

Uh oh!

rkaneriya Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

charlesmyu Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

charlesmyu Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1 bot commented Oct 1, 2025 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

charlesmyu commented Oct 1, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Oct 1, 2025 •

edited by datadog-official bot

Loading

pr-commenter bot commented Oct 1, 2025 •

edited

Loading