Antithesis poc #9993
Draft
Antithesis poc #9993
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 4 performance regressions! Performance is the same for 56 metrics, 5 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.104 s) : 0, 1103916
Total [baseline] (10.745 s) : 0, 10745171
Agent [candidate] (1.1 s) : 0, 1100349
Total [candidate] (10.898 s) : 0, 10898098
section appsec
Agent [baseline] (1.283 s) : 0, 1283479
Total [baseline] (11.094 s) : 0, 11094373
Agent [candidate] (1.283 s) : 0, 1282622
Total [candidate] (11.209 s) : 0, 11208991
section iast
Agent [baseline] (1.247 s) : 0, 1247089
Total [baseline] (11.354 s) : 0, 11354328
Agent [candidate] (1.245 s) : 0, 1245058
Total [candidate] (11.218 s) : 0, 11218239
section profiling
Agent [baseline] (1.231 s) : 0, 1230587
Total [baseline] (11.098 s) : 0, 11097956
Agent [candidate] (1.441 s) : 0, 1441188
Total [candidate] (11.455 s) : 0, 11455201
gantt
title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.474 ms) : 0, 1474
BytebuddyAgent [baseline] (708.706 ms) : 0, 708706
BytebuddyAgent [candidate] (705.809 ms) : 0, 705809
GlobalTracer [baseline] (249.533 ms) : 0, 249533
GlobalTracer [candidate] (249.323 ms) : 0, 249323
AppSec [baseline] (32.232 ms) : 0, 32232
AppSec [candidate] (32.214 ms) : 0, 32214
Debugger [baseline] (64.312 ms) : 0, 64312
Debugger [candidate] (63.78 ms) : 0, 63780
Remote Config [baseline] (640.131 µs) : 0, 640
Remote Config [candidate] (648.547 µs) : 0, 649
Telemetry [baseline] (8.29 ms) : 0, 8290
Telemetry [candidate] (8.312 ms) : 0, 8312
Flare Poller [baseline] (3.719 ms) : 0, 3719
Flare Poller [candidate] (3.777 ms) : 0, 3777
section appsec
crashtracking [baseline] (1.462 ms) : 0, 1462
crashtracking [candidate] (1.473 ms) : 0, 1473
BytebuddyAgent [baseline] (732.308 ms) : 0, 732308
BytebuddyAgent [candidate] (730.883 ms) : 0, 730883
GlobalTracer [baseline] (241.297 ms) : 0, 241297
GlobalTracer [candidate] (241.77 ms) : 0, 241770
AppSec [baseline] (174.472 ms) : 0, 174472
AppSec [candidate] (174.792 ms) : 0, 174792
Debugger [baseline] (61.135 ms) : 0, 61135
Debugger [candidate] (60.961 ms) : 0, 60961
Remote Config [baseline] (698.67 µs) : 0, 699
Remote Config [candidate] (711.444 µs) : 0, 711
Telemetry [baseline] (8.287 ms) : 0, 8287
Telemetry [candidate] (8.311 ms) : 0, 8311
Flare Poller [baseline] (3.884 ms) : 0, 3884
Flare Poller [candidate] (3.747 ms) : 0, 3747
IAST [baseline] (24.87 ms) : 0, 24870
IAST [candidate] (24.721 ms) : 0, 24721
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (835.834 ms) : 0, 835834
BytebuddyAgent [candidate] (836.18 ms) : 0, 836180
GlobalTracer [baseline] (239.776 ms) : 0, 239776
GlobalTracer [candidate] (239.832 ms) : 0, 239832
AppSec [baseline] (30.802 ms) : 0, 30802
AppSec [candidate] (25.902 ms) : 0, 25902
Debugger [baseline] (61.148 ms) : 0, 61148
Debugger [candidate] (60.086 ms) : 0, 60086
Remote Config [baseline] (551.114 µs) : 0, 551
Remote Config [candidate] (539.541 µs) : 0, 540
Telemetry [baseline] (7.651 ms) : 0, 7651
Telemetry [candidate] (7.611 ms) : 0, 7611
Flare Poller [baseline] (3.474 ms) : 0, 3474
Flare Poller [candidate] (3.46 ms) : 0, 3460
IAST [baseline] (31.235 ms) : 0, 31235
IAST [candidate] (34.825 ms) : 0, 34825
section profiling
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.464 ms) : 0, 1464
BytebuddyAgent [baseline] (732.097 ms) : 0, 732097
BytebuddyAgent [candidate] (737.636 ms) : 0, 737636
GlobalTracer [baseline] (222.197 ms) : 0, 222197
GlobalTracer [candidate] (224.288 ms) : 0, 224288
AppSec [baseline] (32.163 ms) : 0, 32163
AppSec [candidate] (32.657 ms) : 0, 32657
Debugger [baseline] (63.271 ms) : 0, 63271
Debugger [candidate] (63.335 ms) : 0, 63335
Remote Config [baseline] (648.011 µs) : 0, 648
Remote Config [candidate] (654.741 µs) : 0, 655
Telemetry [baseline] (7.973 ms) : 0, 7973
Telemetry [candidate] (8.087 ms) : 0, 8087
Flare Poller [baseline] (3.764 ms) : 0, 3764
Flare Poller [candidate] (3.811 ms) : 0, 3811
ProfilingAgent [baseline] (97.118 ms) : 0, 97118
ProfilingAgent [candidate] (298.774 ms) : 0, 298774
Profiling [baseline] (97.708 ms) : 0, 97708
Profiling [candidate] (299.375 ms) : 0, 299375
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.107 s) : 0, 1106751
Total [baseline] (8.9 s) : 0, 8900227
Agent [candidate] (1.1 s) : 0, 1100340
Total [candidate] (8.877 s) : 0, 8876873
section iast
Agent [baseline] (1.235 s) : 0, 1235174
Total [baseline] (9.59 s) : 0, 9589713
Agent [candidate] (1.248 s) : 0, 1248123
Total [candidate] (9.454 s) : 0, 9453825
gantt
title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.482 ms) : 0, 1482
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (711.138 ms) : 0, 711138
BytebuddyAgent [candidate] (707.221 ms) : 0, 707221
GlobalTracer [baseline] (249.845 ms) : 0, 249845
GlobalTracer [candidate] (249.596 ms) : 0, 249596
AppSec [baseline] (32.635 ms) : 0, 32635
AppSec [candidate] (32.476 ms) : 0, 32476
Debugger [baseline] (63.968 ms) : 0, 63968
Debugger [candidate] (62.05 ms) : 0, 62050
Remote Config [baseline] (634.28 µs) : 0, 634
Remote Config [candidate] (621.743 µs) : 0, 622
Telemetry [baseline] (8.26 ms) : 0, 8260
Telemetry [candidate] (8.197 ms) : 0, 8197
Flare Poller [baseline] (3.712 ms) : 0, 3712
Flare Poller [candidate] (3.656 ms) : 0, 3656
section iast
crashtracking [baseline] (1.473 ms) : 0, 1473
crashtracking [candidate] (1.491 ms) : 0, 1491
BytebuddyAgent [baseline] (827.424 ms) : 0, 827424
BytebuddyAgent [candidate] (837.756 ms) : 0, 837756
GlobalTracer [baseline] (237.924 ms) : 0, 237924
GlobalTracer [candidate] (240.127 ms) : 0, 240127
AppSec [baseline] (34.037 ms) : 0, 34037
AppSec [candidate] (26.468 ms) : 0, 26468
Debugger [baseline] (60.22 ms) : 0, 60220
Debugger [candidate] (59.942 ms) : 0, 59942
Remote Config [baseline] (547.603 µs) : 0, 548
Remote Config [candidate] (542.323 µs) : 0, 542
Telemetry [baseline] (7.686 ms) : 0, 7686
Telemetry [candidate] (7.632 ms) : 0, 7632
Flare Poller [baseline] (3.509 ms) : 0, 3509
Flare Poller [candidate] (3.502 ms) : 0, 3502
IAST [baseline] (27.41 ms) : 0, 27410
IAST [candidate] (29.511 ms) : 0, 29511
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 18 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (18.834 ms) : 18639, 19029
. : milestone, 18834,
appsec (18.59 ms) : 18399, 18782
. : milestone, 18590,
code_origins (17.632 ms) : 17457, 17806
. : milestone, 17632,
iast (17.604 ms) : 17429, 17778
. : milestone, 17604,
profiling (18.687 ms) : 18499, 18874
. : milestone, 18687,
tracing (17.457 ms) : 17286, 17628
. : milestone, 17457,
section candidate
no_agent (17.876 ms) : 17696, 18057
. : milestone, 17876,
appsec (19.57 ms) : 19370, 19770
. : milestone, 19570,
code_origins (17.726 ms) : 17551, 17900
. : milestone, 17726,
iast (17.615 ms) : 17439, 17791
. : milestone, 17615,
profiling (18.507 ms) : 18327, 18688
. : milestone, 18507,
tracing (17.45 ms) : 17275, 17625
. : milestone, 17450,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (1.217 ms) : 1205, 1230
. : milestone, 1217,
iast (3.36 ms) : 3269, 3450
. : milestone, 3360,
iast_FULL (5.928 ms) : 5845, 6012
. : milestone, 5928,
iast_GLOBAL (3.698 ms) : 3639, 3757
. : milestone, 3698,
profiling (2.07 ms) : 2051, 2090
. : milestone, 2070,
tracing (1.805 ms) : 1789, 1821
. : milestone, 1805,
section candidate
no_agent (1.205 ms) : 1193, 1217
. : milestone, 1205,
iast (3.206 ms) : 3168, 3245
. : milestone, 3206,
iast_FULL (5.889 ms) : 5830, 5948
. : milestone, 5889,
iast_GLOBAL (3.621 ms) : 3569, 3674
. : milestone, 3621,
profiling (1.975 ms) : 1958, 1992
. : milestone, 1975,
tracing (1.813 ms) : 1798, 1829
. : milestone, 1813,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (1.481 ms) : 1470, 1493
. : milestone, 1481,
appsec (2.475 ms) : 2423, 2526
. : milestone, 2475,
iast (2.212 ms) : 2148, 2275
. : milestone, 2212,
iast_GLOBAL (2.26 ms) : 2195, 2324
. : milestone, 2260,
profiling (2.069 ms) : 2017, 2120
. : milestone, 2069,
tracing (2.049 ms) : 1999, 2099
. : milestone, 2049,
section candidate
no_agent (1.482 ms) : 1470, 1493
. : milestone, 1482,
appsec (2.463 ms) : 2412, 2514
. : milestone, 2463,
iast (2.214 ms) : 2150, 2278
. : milestone, 2214,
iast_GLOBAL (2.257 ms) : 2193, 2322
. : milestone, 2257,
profiling (2.503 ms) : 2341, 2665
. : milestone, 2503,
tracing (2.049 ms) : 1999, 2099
. : milestone, 2049,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~35bd8be91a, baseline=1.56.0-SNAPSHOT~8d538789b9
dateFormat X
axisFormat %s
section baseline
no_agent (14.802 s) : 14802000, 14802000
. : milestone, 14802000,
appsec (15.121 s) : 15121000, 15121000
. : milestone, 15121000,
iast (18.553 s) : 18553000, 18553000
. : milestone, 18553000,
iast_GLOBAL (17.996 s) : 17996000, 17996000
. : milestone, 17996000,
profiling (14.872 s) : 14872000, 14872000
. : milestone, 14872000,
tracing (14.647 s) : 14647000, 14647000
. : milestone, 14647000,
section candidate
no_agent (14.931 s) : 14931000, 14931000
. : milestone, 14931000,
appsec (14.612 s) : 14612000, 14612000
. : milestone, 14612000,
iast (18.309 s) : 18309000, 18309000
. : milestone, 18309000,
iast_GLOBAL (17.953 s) : 17953000, 17953000
. : milestone, 17953000,
profiling (14.6 s) : 14600000, 14600000
. : milestone, 14600000,
tracing (14.526 s) : 14526000, 14526000
. : milestone, 14526000,
|
6 tasks
|
🎯 Code Coverage 🔗 Commit SHA: 35bd8be | Docs | Datadog PR Page | Was this helpful? Give us feedback! |
bric3
reviewed
Nov 25, 2025
Comment on lines
152
to
158
| ObjectNode httpResponseDetails = JsonNodeFactory.instance.objectNode(); | ||
| httpResponseDetails.put("trace_count", payload.traceCount()); | ||
| httpResponseDetails.put("payload_size_bytes", sizeInBytes); | ||
| httpResponseDetails.put("http_status", response.code()); | ||
| httpResponseDetails.put("http_message", response.message()); | ||
| httpResponseDetails.put("success", response.code() == 200); | ||
| httpResponseDetails.put("agent_url", tracesUrl.toString()); |
Contributor
There was a problem hiding this comment.
thought: I'm very uncomfortable with using jackson API in these parts. And the Assert methods rely on it.
Comment on lines
175
to
178
| log.debug("ANTITHESIS_ASSERT: Non-200 HTTP response (unreachable) - code: {}, message: {}, traces: {}", response.code(), response.message(), payload.traceCount()); | ||
| Assert.unreachable( | ||
| "Non-200 HTTP response from agent indicates API failure - traces may be lost", | ||
| errorDetails); |
Contributor
There was a problem hiding this comment.
thought: Another thing is the human cost to maintain these assertions.
bric3
reviewed
Nov 25, 2025
| implementation(project(":internal-api")) | ||
|
|
||
| // Antithesis SDK for assertions and property testing - bundled in tracer JAR | ||
| implementation(group = "com.antithesis", name = "sdk", version = "1.4.5") |
Contributor
There was a problem hiding this comment.
thought: Another concern, is how much more weight it adds to the jar.
Contributor
There was a problem hiding this comment.
Right now this PR adds ~20mb to the final jar... but a lot of that is because it embeds antithesis, its native FFI wrapper, and various jackson dependencies multiple times in each of the product directories in the agent jar.
If we move it to the shared section in dd-java-agent/build.gradle and add the necessary excludes to gradle/dependencies.gradle like we do for other shared dependencies then the overhead is 3mb compressed and 8mb uncompressed.
That still feels too big to have in the general deliverable for something only used for testing purposes.
One option might be to only include the direct dependency in the release (i.e. without jackson or the ffi wrapper.) - in other words just enough to allow the classes to load. We'd then have to look at how to combine the other parts for testing, whether that's via -Xbootclasspath/a: or something similar.
Debugger benchmarksParameters
See matching parameters
SummaryFound 4 performance improvements and 0 performance regressions! Performance is the same for 2 metrics, 9 unstable metrics.
See unchanged results
Request duration reports for reportsgantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (319.234 µs) : 298, 341
. : milestone, 319,
basic (301.788 µs) : 284, 319
. : milestone, 302,
loop (8.96 ms) : 8956, 8963
. : milestone, 8960,
section candidate
noprobe (334.76 µs) : 289, 381
. : milestone, 335,
basic (296.562 µs) : 288, 305
. : milestone, 297,
loop (8.452 ms) : 8391, 8514
. : milestone, 8452,
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Add Antithesis assertions in the tracer
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]