Skip to content

Mark TracerFlareSmokeTest 'tracer generates flare with profiling template override' as flaky#10712

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomasterfrom
mcculls/TracerFlareSmokeTest-flaky
Mar 2, 2026
Merged

Mark TracerFlareSmokeTest 'tracer generates flare with profiling template override' as flaky#10712
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomasterfrom
mcculls/TracerFlareSmokeTest-flaky

Conversation

@mcculls
Copy link
Copy Markdown
Contributor

@mcculls mcculls commented Mar 2, 2026

Motivation

Other test methods in this class are already marked as @Flaky and this test method has flaked over the last week.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@mcculls
Mark TracerFlareSmokeTest 'tracer generates flare with profiling temp…
5e3f573 
…late override' as flaky (other tests in this class are already marked as @flaky)
@mcculls mcculls requested a review from a team as a code owner March 2, 2026 12:15
@mcculls mcculls added the type: bug Bug report and fix label Mar 2, 2026
@mcculls mcculls removed the request for review from a team March 2, 2026 12:15
@mcculls mcculls added the comp: core Tracer core label Mar 2, 2026
@mcculls mcculls requested a review from mtoffl01 March 2, 2026 12:15
@mcculls mcculls added tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests labels Mar 2, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Mar 2, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/TracerFlareSmokeTest-flaky
git_commit_date 1772452469 1772453511
git_commit_sha 6c468be 5e3f573
release_version 1.60.0-SNAPSHOT~6c468be8e8 1.60.0-SNAPSHOT~5e3f573693
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1772455195 1772455195
ci_job_id 1466613264 1466613264
ci_pipeline_id 99725648 99725648
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-xp028dvz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-xp028dvz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 9 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067088
Total [baseline] (8.765 s) : 0, 8765002
Agent [candidate] (1.074 s) : 0, 1073532
Total [candidate] (8.757 s) : 0, 8757284
section iast
Agent [baseline] (1.228 s) : 0, 1228231
Total [baseline] (9.393 s) : 0, 9392769
Agent [candidate] (1.241 s) : 0, 1241248
Total [candidate] (9.387 s) : 0, 9386740
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent iast 1.228 s 161.143 ms (15.1%)
Total tracing 8.765 s -
Total iast 9.393 s 627.767 ms (7.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent iast 1.241 s 167.716 ms (15.6%)
Total tracing 8.757 s -
Total iast 9.387 s 629.456 ms (7.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (628.975 ms) : 0, 628975
BytebuddyAgent [candidate] (633.332 ms) : 0, 633332
AgentMeter [baseline] (28.999 ms) : 0, 28999
AgentMeter [candidate] (29.271 ms) : 0, 29271
GlobalTracer [baseline] (258.079 ms) : 0, 258079
GlobalTracer [candidate] (259.8 ms) : 0, 259800
AppSec [baseline] (33.081 ms) : 0, 33081
AppSec [candidate] (33.129 ms) : 0, 33129
Debugger [baseline] (65.482 ms) : 0, 65482
Debugger [candidate] (65.622 ms) : 0, 65622
Remote Config [baseline] (621.365 µs) : 0, 621
Remote Config [candidate] (612.903 µs) : 0, 613
Telemetry [baseline] (10.005 ms) : 0, 10005
Telemetry [candidate] (9.76 ms) : 0, 9760
Flare Poller [baseline] (4.602 ms) : 0, 4602
Flare Poller [candidate] (4.538 ms) : 0, 4538
section iast
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (793.756 ms) : 0, 793756
BytebuddyAgent [candidate] (803.241 ms) : 0, 803241
AgentMeter [baseline] (11.267 ms) : 0, 11267
AgentMeter [candidate] (11.46 ms) : 0, 11460
GlobalTracer [baseline] (246.948 ms) : 0, 246948
GlobalTracer [candidate] (248.949 ms) : 0, 248949
IAST [baseline] (27.019 ms) : 0, 27019
IAST [candidate] (27.232 ms) : 0, 27232
AppSec [baseline] (33.178 ms) : 0, 33178
AppSec [candidate] (34.296 ms) : 0, 34296
Debugger [baseline] (66.319 ms) : 0, 66319
Debugger [candidate] (66.009 ms) : 0, 66009
Remote Config [baseline] (539.314 µs) : 0, 539
Remote Config [candidate] (537.298 µs) : 0, 537
Telemetry [baseline] (8.634 ms) : 0, 8634
Telemetry [candidate] (8.614 ms) : 0, 8614
Flare Poller [baseline] (3.465 ms) : 0, 3465
Flare Poller [candidate] (3.443 ms) : 0, 3443
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1065769
Total [baseline] (10.921 s) : 0, 10920817
Agent [candidate] (1.066 s) : 0, 1066114
Total [candidate] (10.909 s) : 0, 10909075
section appsec
Agent [baseline] (1.24 s) : 0, 1240446
Total [baseline] (11.051 s) : 0, 11050860
Agent [candidate] (1.243 s) : 0, 1243169
Total [candidate] (11.085 s) : 0, 11085213
section iast
Agent [baseline] (1.234 s) : 0, 1233944
Total [baseline] (11.244 s) : 0, 11243707
Agent [candidate] (1.234 s) : 0, 1233993
Total [candidate] (11.271 s) : 0, 11271212
section profiling
Agent [baseline] (1.193 s) : 0, 1193007
Total [baseline] (10.967 s) : 0, 10966801
Agent [candidate] (1.192 s) : 0, 1192112
Total [candidate] (10.992 s) : 0, 10992053
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent appsec 1.24 s 174.677 ms (16.4%)
Agent iast 1.234 s 168.175 ms (15.8%)
Agent profiling 1.193 s 127.238 ms (11.9%)
Total tracing 10.921 s -
Total appsec 11.051 s 130.043 ms (1.2%)
Total iast 11.244 s 322.89 ms (3.0%)
Total profiling 10.967 s 45.984 ms (0.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent appsec 1.243 s 177.055 ms (16.6%)
Agent iast 1.234 s 167.879 ms (15.7%)
Agent profiling 1.192 s 125.998 ms (11.8%)
Total tracing 10.909 s -
Total appsec 11.085 s 176.138 ms (1.6%)
Total iast 11.271 s 362.137 ms (3.3%)
Total profiling 10.992 s 82.978 ms (0.8%) 
gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (627.356 ms) : 0, 627356
BytebuddyAgent [candidate] (628.788 ms) : 0, 628788
AgentMeter [baseline] (29.145 ms) : 0, 29145
AgentMeter [candidate] (29.216 ms) : 0, 29216
GlobalTracer [baseline] (258.069 ms) : 0, 258069
GlobalTracer [candidate] (257.838 ms) : 0, 257838
AppSec [baseline] (32.989 ms) : 0, 32989
AppSec [candidate] (32.994 ms) : 0, 32994
Debugger [baseline] (64.482 ms) : 0, 64482
Debugger [candidate] (65.768 ms) : 0, 65768
Remote Config [baseline] (614.114 µs) : 0, 614
Remote Config [candidate] (620.799 µs) : 0, 621
Telemetry [baseline] (9.885 ms) : 0, 9885
Telemetry [candidate] (9.052 ms) : 0, 9052
Flare Poller [baseline] (6.086 ms) : 0, 6086
Flare Poller [candidate] (4.502 ms) : 0, 4502
section appsec
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (658.382 ms) : 0, 658382
BytebuddyAgent [candidate] (660.366 ms) : 0, 660366
AgentMeter [baseline] (11.993 ms) : 0, 11993
AgentMeter [candidate] (12.019 ms) : 0, 12019
GlobalTracer [baseline] (258.491 ms) : 0, 258491
GlobalTracer [candidate] (259.486 ms) : 0, 259486
IAST [baseline] (25.262 ms) : 0, 25262
IAST [candidate] (25.478 ms) : 0, 25478
AppSec [baseline] (167.662 ms) : 0, 167662
AppSec [candidate] (168.447 ms) : 0, 168447
Debugger [baseline] (67.5 ms) : 0, 67500
Debugger [candidate] (66.424 ms) : 0, 66424
Remote Config [baseline] (685.757 µs) : 0, 686
Remote Config [candidate] (689.607 µs) : 0, 690
Telemetry [baseline] (9.401 ms) : 0, 9401
Telemetry [candidate] (9.395 ms) : 0, 9395
Flare Poller [baseline] (3.784 ms) : 0, 3784
Flare Poller [candidate] (3.59 ms) : 0, 3590
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (796.445 ms) : 0, 796445
BytebuddyAgent [candidate] (795.795 ms) : 0, 795795
AgentMeter [baseline] (11.35 ms) : 0, 11350
AgentMeter [candidate] (11.294 ms) : 0, 11294
GlobalTracer [baseline] (247.56 ms) : 0, 247560
GlobalTracer [candidate] (248.771 ms) : 0, 248771
IAST [baseline] (26.943 ms) : 0, 26943
IAST [candidate] (27.301 ms) : 0, 27301
AppSec [baseline] (36.001 ms) : 0, 36001
AppSec [candidate] (33.12 ms) : 0, 33120
Debugger [baseline] (65.661 ms) : 0, 65661
Debugger [candidate] (67.867 ms) : 0, 67867
Remote Config [baseline] (547.849 µs) : 0, 548
Remote Config [candidate] (544.306 µs) : 0, 544
Telemetry [baseline] (8.737 ms) : 0, 8737
Telemetry [candidate] (8.639 ms) : 0, 8639
Flare Poller [baseline] (3.566 ms) : 0, 3566
Flare Poller [candidate] (3.469 ms) : 0, 3469
section profiling
crashtracking [baseline] (1.158 ms) : 0, 1158
crashtracking [candidate] (1.167 ms) : 0, 1167
BytebuddyAgent [baseline] (683.434 ms) : 0, 683434
BytebuddyAgent [candidate] (682.867 ms) : 0, 682867
AgentMeter [baseline] (8.571 ms) : 0, 8571
AgentMeter [candidate] (8.586 ms) : 0, 8586
GlobalTracer [baseline] (216.517 ms) : 0, 216517
GlobalTracer [candidate] (216.219 ms) : 0, 216219
AppSec [baseline] (32.585 ms) : 0, 32585
AppSec [candidate] (32.6 ms) : 0, 32600
Debugger [baseline] (67.151 ms) : 0, 67151
Debugger [candidate] (67.15 ms) : 0, 67150
Remote Config [baseline] (616.031 µs) : 0, 616
Remote Config [candidate] (635.639 µs) : 0, 636
Telemetry [baseline] (8.823 ms) : 0, 8823
Telemetry [candidate] (8.919 ms) : 0, 8919
Flare Poller [baseline] (3.769 ms) : 0, 3769
Flare Poller [candidate] (3.745 ms) : 0, 3745
ProfilingAgent [baseline] (99.498 ms) : 0, 99498
ProfilingAgent [candidate] (99.389 ms) : 0, 99389
Profiling [baseline] (100.086 ms) : 0, 100086
Profiling [candidate] (99.97 ms) : 0, 99970
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/TracerFlareSmokeTest-flaky
git_commit_date 1772452469 1772453511
git_commit_sha 6c468be 5e3f573
release_version 1.60.0-SNAPSHOT~6c468be8e8 1.60.0-SNAPSHOT~5e3f573693
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1772455682 1772455682
ci_job_id 1466613268 1466613268
ci_pipeline_id 99725648 99725648
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-3tkbwkd3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-3tkbwkd3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load worse
[+92.034µs; +196.992µs] or [+3.359%; +7.190%]		 unsure
[+148.933µs; +533.869µs] or [+1.924%; +6.897%]		 unstable
[-182.517op/s; +79.955op/s] or [-13.925%; +6.100%]		 2.884ms 8.082ms 1259.438op/s 2.740ms 7.741ms 1310.719op/s
scenario:load:insecure-bank:iast_FULL:high_load unsure
[-217.187µs; -18.110µs] or [-4.002%; -0.334%]		 better
[-951.588µs; -293.695µs] or [-7.300%; -2.253%]		 unstable
[-52.561op/s; +95.686op/s] or [-6.938%; +12.630%]		 5.310ms 12.413ms 779.156op/s 5.427ms 13.036ms 757.594op/s
scenario:load:insecure-bank:profiling:high_load better
[-152.339µs; -35.783µs] or [-8.857%; -2.080%]		 unstable
[-976.476µs; -80.400µs] or [-18.763%; -1.545%]		 unstable
[-73.313op/s; +412.126op/s] or [-3.526%; +19.820%]		 1.626ms 4.676ms 2248.781op/s 1.720ms 5.204ms 2079.375op/s
scenario:load:petclinic:iast:high_load worse
[+451.406µs; +979.115µs] or [+2.655%; +5.759%]		 unsure
[+432.384µs; +1537.001µs] or [+1.542%; +5.480%]		 unstable
[-36.159op/s; +15.784op/s] or [-13.469%; +5.879%]		 17.716ms 29.032ms 258.281op/s 17.001ms 28.047ms 268.469op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.965 ms) : 18777, 19153
.   : milestone, 18965,
appsec (18.816 ms) : 18625, 19008
.   : milestone, 18816,
code_origins (17.548 ms) : 17374, 17723
.   : milestone, 17548,
iast (17.377 ms) : 17206, 17548
.   : milestone, 17377,
profiling (18.692 ms) : 18505, 18879
.   : milestone, 18692,
tracing (17.557 ms) : 17382, 17731
.   : milestone, 17557,
section candidate
no_agent (18.095 ms) : 17909, 18281
.   : milestone, 18095,
appsec (18.636 ms) : 18449, 18824
.   : milestone, 18636,
code_origins (17.673 ms) : 17495, 17851
.   : milestone, 17673,
iast (18.07 ms) : 17889, 18251
.   : milestone, 18070,
profiling (18.626 ms) : 18440, 18812
.   : milestone, 18626,
tracing (17.651 ms) : 17475, 17827
.   : milestone, 17651,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.965 ms [18.777 ms, 19.153 ms] -
appsec 18.816 ms [18.625 ms, 19.008 ms] -148.493 µs (-0.8%)
code_origins 17.548 ms [17.374 ms, 17.723 ms] -1.416 ms (-7.5%)
iast 17.377 ms [17.206 ms, 17.548 ms] -1.588 ms (-8.4%)
profiling 18.692 ms [18.505 ms, 18.879 ms] -272.837 µs (-1.4%)
tracing 17.557 ms [17.382 ms, 17.731 ms] -1.408 ms (-7.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.095 ms [17.909 ms, 18.281 ms] -
appsec 18.636 ms [18.449 ms, 18.824 ms] 541.199 µs (3.0%)
code_origins 17.673 ms [17.495 ms, 17.851 ms] -421.694 µs (-2.3%)
iast 18.07 ms [17.889 ms, 18.251 ms] -25.248 µs (-0.1%)
profiling 18.626 ms [18.44 ms, 18.812 ms] 531.462 µs (2.9%)
tracing 17.651 ms [17.475 ms, 17.827 ms] -443.791 µs (-2.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.18 ms) : 1168, 1192
.   : milestone, 1180,
iast (3.129 ms) : 3089, 3169
.   : milestone, 3129,
iast_FULL (6.108 ms) : 6045, 6170
.   : milestone, 6108,
iast_GLOBAL (3.497 ms) : 3436, 3558
.   : milestone, 3497,
profiling (2.176 ms) : 2156, 2197
.   : milestone, 2176,
tracing (1.817 ms) : 1801, 1833
.   : milestone, 1817,
section candidate
no_agent (1.173 ms) : 1162, 1185
.   : milestone, 1173,
iast (3.106 ms) : 3067, 3144
.   : milestone, 3106,
iast_FULL (5.937 ms) : 5877, 5997
.   : milestone, 5937,
iast_GLOBAL (3.643 ms) : 3587, 3699
.   : milestone, 3643,
profiling (2.007 ms) : 1989, 2025
.   : milestone, 2007,
tracing (1.826 ms) : 1810, 1842
.   : milestone, 1826,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.18 ms [1.168 ms, 1.192 ms] -
iast 3.129 ms [3.089 ms, 3.169 ms] 1.949 ms (165.2%)
iast_FULL 6.108 ms [6.045 ms, 6.17 ms] 4.928 ms (417.7%)
iast_GLOBAL 3.497 ms [3.436 ms, 3.558 ms] 2.317 ms (196.4%)
profiling 2.176 ms [2.156 ms, 2.197 ms] 996.25 µs (84.4%)
tracing 1.817 ms [1.801 ms, 1.833 ms] 636.962 µs (54.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.173 ms [1.162 ms, 1.185 ms] -
iast 3.106 ms [3.067 ms, 3.144 ms] 1.932 ms (164.7%)
iast_FULL 5.937 ms [5.877 ms, 5.997 ms] 4.763 ms (405.9%)
iast_GLOBAL 3.643 ms [3.587 ms, 3.699 ms] 2.47 ms (210.5%)
profiling 2.007 ms [1.989 ms, 2.025 ms] 833.926 µs (71.1%)
tracing 1.826 ms [1.81 ms, 1.842 ms] 652.803 µs (55.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/TracerFlareSmokeTest-flaky
git_commit_date 1772452469 1772453511
git_commit_sha 6c468be 5e3f573
release_version 1.60.0-SNAPSHOT~6c468be8e8 1.60.0-SNAPSHOT~5e3f573693
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1772455376 1772455376
ci_job_id 1466613270 1466613270
ci_pipeline_id 99725648 99725648
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-s4zbnmxz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-s4zbnmxz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.725 s) : 14725000, 14725000
.   : milestone, 14725000,
appsec (15.003 s) : 15003000, 15003000
.   : milestone, 15003000,
iast (17.832 s) : 17832000, 17832000
.   : milestone, 17832000,
iast_GLOBAL (18.261 s) : 18261000, 18261000
.   : milestone, 18261000,
profiling (14.819 s) : 14819000, 14819000
.   : milestone, 14819000,
tracing (15.021 s) : 15021000, 15021000
.   : milestone, 15021000,
section candidate
no_agent (15.256 s) : 15256000, 15256000
.   : milestone, 15256000,
appsec (15.061 s) : 15061000, 15061000
.   : milestone, 15061000,
iast (18.072 s) : 18072000, 18072000
.   : milestone, 18072000,
iast_GLOBAL (17.642 s) : 17642000, 17642000
.   : milestone, 17642000,
profiling (15.386 s) : 15386000, 15386000
.   : milestone, 15386000,
tracing (14.881 s) : 14881000, 14881000
.   : milestone, 14881000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.725 s [14.725 s, 14.725 s] -
appsec 15.003 s [15.003 s, 15.003 s] 278.0 ms (1.9%)
iast 17.832 s [17.832 s, 17.832 s] 3.107 s (21.1%)
iast_GLOBAL 18.261 s [18.261 s, 18.261 s] 3.536 s (24.0%)
profiling 14.819 s [14.819 s, 14.819 s] 94.0 ms (0.6%)
tracing 15.021 s [15.021 s, 15.021 s] 296.0 ms (2.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.256 s [15.256 s, 15.256 s] -
appsec 15.061 s [15.061 s, 15.061 s] -195.0 ms (-1.3%)
iast 18.072 s [18.072 s, 18.072 s] 2.816 s (18.5%)
iast_GLOBAL 17.642 s [17.642 s, 17.642 s] 2.386 s (15.6%)
profiling 15.386 s [15.386 s, 15.386 s] 130.0 ms (0.9%)
tracing 14.881 s [14.881 s, 14.881 s] -375.0 ms (-2.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~5e3f573693, baseline=1.60.0-SNAPSHOT~6c468be8e8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (3.76 ms) : 3539, 3980
.   : milestone, 3760,
iast (2.256 ms) : 2186, 2325
.   : milestone, 2256,
iast_GLOBAL (2.289 ms) : 2220, 2358
.   : milestone, 2289,
profiling (2.087 ms) : 2032, 2142
.   : milestone, 2087,
tracing (2.065 ms) : 2012, 2119
.   : milestone, 2065,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.786 ms) : 3566, 4006
.   : milestone, 3786,
iast (2.246 ms) : 2177, 2315
.   : milestone, 2246,
iast_GLOBAL (2.295 ms) : 2226, 2365
.   : milestone, 2295,
profiling (2.108 ms) : 2052, 2165
.   : milestone, 2108,
tracing (2.068 ms) : 2014, 2121
.   : milestone, 2068,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 3.76 ms [3.539 ms, 3.98 ms] 2.286 ms (155.2%)
iast 2.256 ms [2.186 ms, 2.325 ms] 782.705 µs (53.1%)
iast_GLOBAL 2.289 ms [2.22 ms, 2.358 ms] 815.631 µs (55.4%)
profiling 2.087 ms [2.032 ms, 2.142 ms] 614.111 µs (41.7%)
tracing 2.065 ms [2.012 ms, 2.119 ms] 591.935 µs (40.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 3.786 ms [3.566 ms, 4.006 ms] 2.314 ms (157.2%)
iast 2.246 ms [2.177 ms, 2.315 ms] 774.029 µs (52.6%)
iast_GLOBAL 2.295 ms [2.226 ms, 2.365 ms] 823.254 µs (55.9%)
profiling 2.108 ms [2.052 ms, 2.165 ms] 636.191 µs (43.2%)
tracing 2.068 ms [2.014 ms, 2.121 ms] 595.513 µs (40.4%)

@mcculls mcculls requested review from a team and amarziali March 2, 2026 17:31
amarziali
amarziali reviewed Mar 2, 2026
View reviewed changes
dd-smoke-tests/tracer-flare/src/test/groovy/datadog/smoketest/TracerFlareSmokeTest.groovy
validateNoUnexpectedFiles(zipContents, CORE_FILES + OPTIONAL_FILES)
}

@Flaky("No flare file created in 30 seconds under various JDKs: oracle8, ibm8, zulu8")
Copy link
Copy Markdown
Contributor

@amarziali amarziali Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it flakes generally on a jdk8?

Copy link
Copy Markdown
Contributor Author

@mcculls mcculls Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly JDK8, but also later IBM JDKs - I copied the original comment from the main test which was already marked as flaky, because this test is just another variant of that test...

amarziali
amarziali approved these changes Mar 2, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@amarziali amarziali left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@gh-worker-ownership-write-b05516 gh-worker-ownership-write-b05516 bot removed the request for review from a team March 2, 2026 20:52
@mcculls
Copy link
Copy Markdown
Contributor Author

mcculls commented Mar 2, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 bot commented Mar 2, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-03-02 22:08:18 UTC ℹ️ Start processing command /merge

2026-03-02 22:08:22 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-03-02 23:09:51 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit 0f5d84f into master Mar 2, 2026
573 of 575 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the mcculls/TracerFlareSmokeTest-flaky branch March 2, 2026 23:09
@github-actions github-actions bot added this to the 1.60.0 milestone Mar 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amarziali amarziali amarziali approved these changes

@mtoffl01 mtoffl01 Awaiting requested review from mtoffl01 mtoffl01 is a code owner automatically assigned from DataDog/apm-sdk-capabilities-java

Assignees

No one assigned

Labels

comp: core Tracer core tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.60.0

Development

Successfully merging this pull request may close these issues.

2 participants

@mcculls @amarziali