Skip to content

chore(ci): fix fuzzing harness to properly build fuzz target#1240

Merged
tobz merged 1 commit into
mainfrom
tobz/dsd-fuzzing-fix
Mar 12, 2026
Merged

chore(ci): fix fuzzing harness to properly build fuzz target#1240
tobz merged 1 commit into
mainfrom
tobz/dsd-fuzzing-fix

Conversation

@tobz
Copy link
Copy Markdown
Member

@tobz tobz commented Mar 11, 2026

Summary

This PR fixes an issue with building fuzz targets due to a bug in cargo-fuzz not respecting Cargo configuration.

In order to build any Saluki crate that depends on saluki-core (including saluki-core itself), we depend on specifying the tokio_unstable configuration flag which unlocks certain unstable features in tokio that we depend on. We manage this by setting it as a "rustflag" (direct flag passed to rustc) in .cargo/config.toml, which ensures it's applied to all builds going through cargo. However, when using cargo-fuzz to build our fuzzing targets, cargo-fuzz fails to respect the Cargo configuration, leading to a failed build of the fuzz target.

This PR updates our CI job that invokes the fuzz script to set RUSTFLAGS with the necessary unstable flag, which is then picked up by cargo-fuzz and passed through during the build. We've also collapsed down the two fuzz targets we had into a single one: dogstatsd_decode_packet. This was done just to simplify things since we don't really care about testing the functional conversions between MetricPacket<'a> and Metric... and I wanted the fuzz target name to reflect the actual method we're fuzzing.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

Triggered a manual CI run of the fuzz:infra job and ensured that it built the fuzz target correctly.

References

N/A

Copilot AI review requested due to automatic review settings March 11, 2026 19:34
Copy link
Copy Markdown
Member Author

tobz commented Mar 11, 2026

@dd-octo-sts dd-octo-sts Bot added area/io General I/O and networking. area/ci CI/CD, automated testing, etc. labels Mar 11, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts the fuzzing setup to reliably compile/run a DogStatsD fuzz target in CI by simplifying the fuzz target(s) and explicitly passing the required Rust cfg flag.

Changes:

  • Replaced two metric-focused fuzz targets with a single dogstatsd_decode_packet harness.
  • Updated lib/saluki-io/fuzz/Cargo.toml to build the new fuzz target binary.
  • Updated GitLab fuzz job to set RUSTFLAGS="--cfg tokio_unstable" due to cargo-fuzz ignoring .cargo/config.toml.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/saluki-io/fuzz/fuzz_targets/parse_dsd_metric_with_conf.rs Removed prior fuzz target that exercised config permutations + metric conversion.
lib/saluki-io/fuzz/fuzz_targets/parse_dsd_metric.rs Removed prior fuzz target that performed decode + metric conversion under default config.
lib/saluki-io/fuzz/fuzz_targets/dogstatsd_decode_packet.rs Added new minimal fuzz target that only calls decode_packet.
lib/saluki-io/fuzz/Cargo.toml Switched the fuzz binary to dogstatsd_decode_packet.
.gitlab/fuzz.yml Ensures CI sets tokio_unstable cfg when running fuzz infra.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/saluki-io/fuzz/fuzz_targets/dogstatsd_decode_packet.rs
Comment thread lib/saluki-io/fuzz/fuzz_targets/dogstatsd_decode_packet.rs
@tobz tobz added the type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo. label Mar 11, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Mar 11, 2026

Binary Size Analysis (Agent Data Plane)

Target: 6fa9373 (baseline) vs 1c2ef0e (comparison) diff
Analysis Type: Stripped binaries (debug symbols excluded)
Baseline Size: 26.19 MiB
Comparison Size: 26.19 MiB
Size Change: +0 B (+0.00%)
Pass/Fail Threshold: +5%
Result: PASSED ✅

Changes by Module

Module File Size Symbols

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL

@tobz tobz marked this pull request as ready for review March 11, 2026 19:48
@tobz tobz requested a review from a team as a code owner March 11, 2026 19:48
Copilot AI review requested due to automatic review settings March 11, 2026 19:48
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/saluki-io/fuzz/Cargo.toml
Comment thread .gitlab/fuzz.yml Outdated
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Mar 11, 2026

Regression Detector (Agent Data Plane)

Regression Detector Results

Run ID: 74948140-0c29-4ac7-9aaa-1cb1973ff9ed

Baseline: 6fa9373
Comparison: 1c2ef0e
Diff

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf experiment goal Δ mean % Δ mean % CI trials links
otlp_ingest_logs_5mb_throughput ingress throughput -0.01 [-0.13, +0.12] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_cpu % cpu utilization -0.67 [-5.64, +4.30] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_memory memory utilization -1.52 [-1.98, -1.06] 1 (metrics) (profiles) (logs)

Fine details of change detection per experiment

perf experiment goal Δ mean % Δ mean % CI trials links
dsd_uds_512kb_3k_contexts_cpu % cpu utilization +6.78 [-50.13, +63.68] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_cpu % cpu utilization +1.39 [-29.91, +32.69] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_cpu % cpu utilization +1.31 [-4.97, +7.58] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_cpu % cpu utilization +1.16 [-7.31, +9.63] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_filtering_5mb_cpu % cpu utilization +0.83 [-1.60, +3.26] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_throughput ingress throughput +0.69 [+0.56, +0.83] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_low memory utilization +0.32 [+0.13, +0.52] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_heavy memory utilization +0.26 [+0.12, +0.40] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_cpu % cpu utilization +0.22 [-1.23, +1.67] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_memory memory utilization +0.19 [+0.00, +0.38] 1 (metrics) (profiles) (logs)
dsd_uds_512kb_3k_contexts_memory memory utilization +0.15 [-0.03, +0.32] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_ultraheavy memory utilization +0.11 [-0.02, +0.24] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_transform_5mb_cpu % cpu utilization +0.07 [-2.09, +2.22] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_memory memory utilization +0.04 [-0.23, +0.30] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_memory memory utilization +0.02 [-0.16, +0.19] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_memory memory utilization +0.01 [-0.24, +0.26] 1 (metrics) (profiles) (logs)
otlp_ingest_metrics_5mb_throughput ingress throughput +0.01 [-0.12, +0.14] 1 (metrics) (profiles) (logs)
dsd_uds_512kb_3k_contexts_throughput ingress throughput +0.00 [-0.05, +0.06] 1 (metrics) (profiles) (logs)
dsd_uds_500mb_3k_contexts_memory memory utilization +0.00 [-0.17, +0.17] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_throughput ingress throughput -0.00 [-0.06, +0.05] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_filtering_5mb_throughput ingress throughput -0.00 [-0.02, +0.02] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_throughput ingress throughput -0.00 [-0.02, +0.02] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_transform_5mb_throughput ingress throughput -0.00 [-0.02, +0.02] 1 (metrics) (profiles) (logs)
dsd_uds_10mb_3k_contexts_throughput ingress throughput -0.01 [-0.15, +0.14] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_throughput ingress throughput -0.01 [-0.13, +0.12] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_throughput ingress throughput -0.02 [-0.05, +0.01] 1 (metrics) (profiles) (logs)
quality_gates_rss_dsd_medium memory utilization -0.04 [-0.23, +0.15] 1 (metrics) (profiles) (logs)
dsd_uds_100mb_3k_contexts_memory memory utilization -0.08 [-0.26, +0.10] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_transform_5mb_memory memory utilization -0.09 [-0.33, +0.15] 1 (metrics) (profiles) (logs)
quality_gates_rss_idle memory utilization -0.23 [-0.26, -0.19] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_cpu % cpu utilization -0.67 [-5.64, +4.30] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_ottl_filtering_5mb_memory memory utilization -0.69 [-1.01, -0.36] 1 (metrics) (profiles) (logs)
otlp_ingest_traces_5mb_cpu % cpu utilization -1.48 [-3.71, +0.75] 1 (metrics) (profiles) (logs)
otlp_ingest_logs_5mb_memory memory utilization -1.52 [-1.98, -1.06] 1 (metrics) (profiles) (logs)
dsd_uds_1mb_3k_contexts_cpu % cpu utilization -2.46 [-53.39, +48.48] 1 (metrics) (profiles) (logs)

Bounds Checks: ✅ Passed

perf experiment bounds_check_name replicates_passed observed_value links
quality_gates_rss_dsd_heavy memory_usage 10/10 111.37MiB ≤ 140MiB (metrics) (profiles) (logs)
quality_gates_rss_dsd_low memory_usage 10/10 33.52MiB ≤ 50MiB (metrics) (profiles) (logs)
quality_gates_rss_dsd_medium memory_usage 10/10 52.58MiB ≤ 75MiB (metrics) (profiles) (logs)
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 165.99MiB ≤ 200MiB (metrics) (profiles) (logs)
quality_gates_rss_idle memory_usage 10/10 21.22MiB ≤ 40MiB (metrics) (profiles) (logs)

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

  • ✅ = significantly better comparison variant performance
  • ❌ = significantly worse comparison variant performance
  • ➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

  1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.

  2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.

  3. Its configuration does not mark it "erratic".

Base automatically changed from tobz/dsd-events-allow-empty-stringy-fields to main March 11, 2026 20:29
@tobz tobz force-pushed the tobz/dsd-fuzzing-fix branch from 7288315 to 1c2ef0e Compare March 11, 2026 20:29
@tobz tobz merged commit 0a6de86 into main Mar 12, 2026
60 checks passed
@tobz tobz deleted the tobz/dsd-fuzzing-fix branch March 12, 2026 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci CI/CD, automated testing, etc. area/io General I/O and networking. type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants