chore(antithesis): Track dropped transactions in as sometimes! assertion

[blt]

Summary

It should be the case that for particularly extreme situations ADP will
drop transactions. We are currently unable to see this in-rig except by
implication. This commit adds a sometimes! assertion to give visibility
into this.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

How did you test this PR?

References

[blt]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• chore(antithesis): Pin all rig container dependencies #2017
• chore(antithesis): dogstatsd generation with byte limit #1974
• chore(antithesis): Add Pyld25, series are not empty #1961
• chore(antithesis): Add two new Pyld properties #1960
• chore(antithesis): Track dropped transactions in as sometimes! assertion #1949 👈 (View in Graphite)
• chore(antithesis): Differential comparison between ADP and Datadog Agent context egress #1891
• chore(antithesis): Parallel drivers retry sends #1950
• chore(antithesis): Generate configs for scenario purposes #1938
• chore(antithesis): LogLevel set only to 'error' #1937
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: 92cb9bd · Comparison: 5943dc3 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.68 MiB (baseline) vs 41.68 MiB (comparison)
Size Change: -1.34 KiB (-0.00%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
anon.bc27ec12f8abfd741d2cc888c8b01dc7.6.llvm.4158422603915796461	-1.87 KiB	1
anon.2c42b77b551de5ce0eb4683a5f3348ef.394.llvm.6471492298253374226	+1.87 KiB	1
anon.d851f2884fec383d59770b6e1053557d.496.llvm.14733898691217894837	-1.77 KiB	1
anon.d851f2884fec383d59770b6e1053557d.496.llvm.2729676012635161149	+1.77 KiB	1
anon.9af47ef3bc1f734053efc1a81f309c81.30.llvm.17299067839495397339	-1.69 KiB	1
anon.d851f2884fec383d59770b6e1053557d.181.llvm.2729676012635161149	+1.69 KiB	1
anon.1ae064f8f7c88592968577f5572fe6a1.81.llvm.10427259948598879219	-1.45 KiB	1
anon.84c2a157bbf679719fc5fed5de7b9895.104.llvm.17643131471645320202	+1.45 KiB	1
anon.1ae064f8f7c88592968577f5572fe6a1.76.llvm.10427259948598879219	-1.45 KiB	1
anon.b91e3af34c069ae48e7bbe7df595d981.128.llvm.10130343606843929883	+1.44 KiB	1
anon.1ae064f8f7c88592968577f5572fe6a1.78.llvm.10427259948598879219	-1.43 KiB	1
anon.105ca0d0b09b38782024af0b4190a881.82.llvm.10103066070350205741	+1.43 KiB	1
anon.168fdbdcbd293237ddac692d5d37499c.1219.llvm.13116985673205370977	-1.33 KiB	1
anon.1ae064f8f7c88592968577f5572fe6a1.597.llvm.12508115777240049810	+1.33 KiB	1
anon.d851f2884fec383d59770b6e1053557d.1940.llvm.14733898691217894837	-1.07 KiB	1
anon.19577c0eeee3802814ec8984397e7805.38.llvm.4244394631308212861	+1.07 KiB	1
anon.d851f2884fec383d59770b6e1053557d.1582.llvm.14733898691217894837	-1000 B	1
anon.56afeeebbb405e88722c851be8501b5f.28.llvm.9557380210322302320	+999 B	1
_RNvMs5_NtNtCsebJFismJkYh_3std2io5errorNtB5_5Error4kind.llvm.13116985673205370977	-971 B	1
_RNvMs5_NtNtCsebJFismJkYh_3std2io5errorNtB5_5Error4kind.llvm.3824531816487215769	+970 B	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +4.72Ki  [NEW]     +49    core::ptr::drop_in_place<tower::util::map_future::MapFuture<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::metrics_service_server::MetricsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>,tonic::service::router::Routes::add_service<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::metrics_service_server::MetricsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>>::{{closure}}>>,tower::util::boxed_clone_sync::BoxCloneSyncService<http::request::Request<axum_core::body::Body>,http::response::Response<axum_core::body::Body>,core::convert::Infallible>::new<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics
  [NEW] +2.49Ki  [NEW]    +422    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_components::common::datadog::validation::ValidationReadiness>>::hca0a8333f13b4a04
  [NEW] +1.87Ki  [NEW]     +97    anon.2c42b77b551de5ce0eb4683a5f3348ef.394.llvm.6471492298253374226
  [NEW] +1.77Ki  [NEW] +1.68Ki    anon.d851f2884fec383d59770b6e1053557d.496.llvm.2729676012635161149
  [NEW] +1.75Ki  [NEW]     +33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<tonic::codec::encode::EncodeBody<tonic_prost::codec::ProstEncoder<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::ExportMetricsServiceResponse>,tokio_stream::once::Once<core::result::Result<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::ExportMetricsServiceResponse,tonic::status::Status>>>,tonic::status::Status::map_error<tonic::status::Status>>>::hc7bb46af7020a455
  [NEW] +1.69Ki  [NEW]     +90    anon.d851f2884fec383d59770b6e1053557d.181.llvm.2729676012635161149
  [NEW] +1.45Ki  [NEW]     +22    anon.84c2a157bbf679719fc5fed5de7b9895.104.llvm.17643131471645320202
  [NEW] +1.44Ki  [NEW]     +22    anon.b91e3af34c069ae48e7bbe7df595d981.128.llvm.10130343606843929883
  [NEW] +1.43Ki  [NEW]     +92    anon.105ca0d0b09b38782024af0b4190a881.82.llvm.10103066070350205741
  [NEW] +1.33Ki  [NEW]     +91    anon.1ae064f8f7c88592968577f5572fe6a1.597.llvm.12508115777240049810
  -0.1% -1.32Ki  [ = ]       0    [2750 Others]
  [DEL] -1.33Ki  [DEL]     -91    anon.168fdbdcbd293237ddac692d5d37499c.1219.llvm.13116985673205370977
  [DEL] -1.43Ki  [DEL]     -92    anon.1ae064f8f7c88592968577f5572fe6a1.78.llvm.10427259948598879219
  [DEL] -1.45Ki  [DEL]     -22    anon.1ae064f8f7c88592968577f5572fe6a1.76.llvm.10427259948598879219
  [DEL] -1.45Ki  [DEL]     -22    anon.1ae064f8f7c88592968577f5572fe6a1.81.llvm.10427259948598879219
  [DEL] -1.69Ki  [DEL]     -90    anon.9af47ef3bc1f734053efc1a81f309c81.30.llvm.17299067839495397339
  [DEL] -1.75Ki  [DEL]     -33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<tonic::codec::encode::EncodeBody<tonic_prost::codec::ProstEncoder<otlp_protos::otlp_include::opentelemetry::proto::collector::logs::v1::ExportLogsServiceResponse>,tokio_stream::once::Once<core::result::Result<otlp_protos::otlp_include::opentelemetry::proto::collector::logs::v1::ExportLogsServiceResponse,tonic::status::Status>>>,tonic::status::Status::map_error<tonic::status::Status>>>::h1fa0da3cd6bc290b
  [DEL] -1.77Ki  [DEL] -1.68Ki    anon.d851f2884fec383d59770b6e1053557d.496.llvm.14733898691217894837
  [DEL] -1.87Ki  [DEL]     -97    anon.bc27ec12f8abfd741d2cc888c8b01dc7.6.llvm.4158422603915796461
  [DEL] -2.50Ki  [DEL]    -422    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_components::common::datadog::transaction::Transaction<saluki_common::buf::chunked::FrozenChunkedBytesBuffer>>>::h352480850e3af1cc
  [DEL] -4.72Ki  [DEL]     -49    core::ptr::drop_in_place<tower::util::map_future::MapFuture<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::trace::v1::trace_service_server::TraceServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>,tonic::service::router::Routes::add_service<otlp_protos::otlp_include::opentelemetry::proto::collector::trace::v1::trace_service_server::TraceServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>>::{{closure}}>>,tower::util::boxed_clone_sync::BoxCloneSyncService<http::request::Request<axum_core::body::Body>,http::response::Response<axum_core::body::Body>,core::convert::Infallible>::new<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::trace::v1::trace_se
  -0.0% -1.34Ki  [ = ]       0    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 9a18a3f4-6a2b-4f79-9eb1-4af81240fe5a
Baseline: 92cb9bda · Comparison: 5943dc34 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_low	memory	⚪ +0.44	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.30	metrics profiles logs
quality_gates_rss_idle	memory	⚪ +0.14	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ +0.06	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.09	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 138 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 43 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 64.6 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 191 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 29.1 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.