chore(ci): fix fuzzing harness to properly build fuzz target#1240
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
There was a problem hiding this comment.
Pull request overview
This PR adjusts the fuzzing setup to reliably compile/run a DogStatsD fuzz target in CI by simplifying the fuzz target(s) and explicitly passing the required Rust cfg flag.
Changes:
- Replaced two metric-focused fuzz targets with a single
dogstatsd_decode_packetharness. - Updated
lib/saluki-io/fuzz/Cargo.tomlto build the new fuzz target binary. - Updated GitLab fuzz job to set
RUSTFLAGS="--cfg tokio_unstable"due tocargo-fuzzignoring.cargo/config.toml.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| lib/saluki-io/fuzz/fuzz_targets/parse_dsd_metric_with_conf.rs | Removed prior fuzz target that exercised config permutations + metric conversion. |
| lib/saluki-io/fuzz/fuzz_targets/parse_dsd_metric.rs | Removed prior fuzz target that performed decode + metric conversion under default config. |
| lib/saluki-io/fuzz/fuzz_targets/dogstatsd_decode_packet.rs | Added new minimal fuzz target that only calls decode_packet. |
| lib/saluki-io/fuzz/Cargo.toml | Switched the fuzz binary to dogstatsd_decode_packet. |
| .gitlab/fuzz.yml | Ensures CI sets tokio_unstable cfg when running fuzz infra. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Binary Size Analysis (Agent Data Plane)Target: 6fa9373 (baseline) vs 1c2ef0e (comparison) diff
|
| Module | File Size | Symbols |
|---|
Detailed Symbol Changes
FILE SIZE VM SIZE
-------------- --------------
[ = ] 0 [ = ] 0 TOTAL
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Regression Detector (Agent Data Plane)Regression Detector ResultsRun ID: 74948140-0c29-4ac7-9aaa-1cb1973ff9ed Baseline: 6fa9373 Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | otlp_ingest_logs_5mb_throughput | ingress throughput | -0.01 | [-0.13, +0.12] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_cpu | % cpu utilization | -0.67 | [-5.64, +4.30] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_memory | memory utilization | -1.52 | [-1.98, -1.06] | 1 | (metrics) (profiles) (logs) |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ➖ | dsd_uds_512kb_3k_contexts_cpu | % cpu utilization | +6.78 | [-50.13, +63.68] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_cpu | % cpu utilization | +1.39 | [-29.91, +32.69] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_cpu | % cpu utilization | +1.31 | [-4.97, +7.58] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_cpu | % cpu utilization | +1.16 | [-7.31, +9.63] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_cpu | % cpu utilization | +0.83 | [-1.60, +3.26] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_throughput | ingress throughput | +0.69 | [+0.56, +0.83] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_low | memory utilization | +0.32 | [+0.13, +0.52] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_heavy | memory utilization | +0.26 | [+0.12, +0.40] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_cpu | % cpu utilization | +0.22 | [-1.23, +1.67] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_memory | memory utilization | +0.19 | [+0.00, +0.38] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_512kb_3k_contexts_memory | memory utilization | +0.15 | [-0.03, +0.32] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_ultraheavy | memory utilization | +0.11 | [-0.02, +0.24] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_cpu | % cpu utilization | +0.07 | [-2.09, +2.22] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_memory | memory utilization | +0.04 | [-0.23, +0.30] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_memory | memory utilization | +0.02 | [-0.16, +0.19] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_memory | memory utilization | +0.01 | [-0.24, +0.26] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_throughput | ingress throughput | +0.01 | [-0.12, +0.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_512kb_3k_contexts_throughput | ingress throughput | +0.00 | [-0.05, +0.06] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_memory | memory utilization | +0.00 | [-0.17, +0.17] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_throughput | ingress throughput | -0.00 | [-0.06, +0.05] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_throughput | ingress throughput | -0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_throughput | ingress throughput | -0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_throughput | ingress throughput | -0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_throughput | ingress throughput | -0.01 | [-0.15, +0.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_throughput | ingress throughput | -0.01 | [-0.13, +0.12] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_throughput | ingress throughput | -0.02 | [-0.05, +0.01] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_medium | memory utilization | -0.04 | [-0.23, +0.15] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_memory | memory utilization | -0.08 | [-0.26, +0.10] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_memory | memory utilization | -0.09 | [-0.33, +0.15] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_idle | memory utilization | -0.23 | [-0.26, -0.19] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_cpu | % cpu utilization | -0.67 | [-5.64, +4.30] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_memory | memory utilization | -0.69 | [-1.01, -0.36] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_cpu | % cpu utilization | -1.48 | [-3.71, +0.75] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_memory | memory utilization | -1.52 | [-1.98, -1.06] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_cpu | % cpu utilization | -2.46 | [-53.39, +48.48] | 1 | (metrics) (profiles) (logs) |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | quality_gates_rss_dsd_heavy | memory_usage | 10/10 | 111.37MiB ≤ 140MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_low | memory_usage | 10/10 | 33.52MiB ≤ 50MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_medium | memory_usage | 10/10 | 52.58MiB ≤ 75MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_ultraheavy | memory_usage | 10/10 | 165.99MiB ≤ 200MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_idle | memory_usage | 10/10 | 21.22MiB ≤ 40MiB | (metrics) (profiles) (logs) |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
7288315 to
1c2ef0e
Compare

Summary
This PR fixes an issue with building fuzz targets due to a bug in
cargo-fuzznot respecting Cargo configuration.In order to build any Saluki crate that depends on
saluki-core(includingsaluki-coreitself), we depend on specifying thetokio_unstableconfiguration flag which unlocks certain unstable features intokiothat we depend on. We manage this by setting it as a "rustflag" (direct flag passed torustc) in.cargo/config.toml, which ensures it's applied to all builds going throughcargo. However, when usingcargo-fuzzto build our fuzzing targets,cargo-fuzzfails to respect the Cargo configuration, leading to a failed build of the fuzz target.This PR updates our CI job that invokes the fuzz script to set
RUSTFLAGSwith the necessary unstable flag, which is then picked up bycargo-fuzzand passed through during the build. We've also collapsed down the two fuzz targets we had into a single one:dogstatsd_decode_packet. This was done just to simplify things since we don't really care about testing the functional conversions betweenMetricPacket<'a>andMetric... and I wanted the fuzz target name to reflect the actual method we're fuzzing.Change Type
How did you test this PR?
Triggered a manual CI run of the
fuzz:infrajob and ensured that it built the fuzz target correctly.References
N/A