Skip to content

test: fix vacuous assertions in audit analyzer tests#885

Merged
jamesadevine merged 1 commit into
mainfrom
test-reducer/fix-vacuous-otel-missing-assertions-4323309f404c66da
Jun 7, 2026
Merged

test: fix vacuous assertions in audit analyzer tests#885
jamesadevine merged 1 commit into
mainfrom
test-reducer/fix-vacuous-otel-missing-assertions-4323309f404c66da

Conversation

@github-actions

@github-actions github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Test Suite Reduction: src/audit/analyzers/otel.rs + src/audit/analyzers/missing.rs

What was wrong

  • analyze_otel_reads_otel_only (otel.rs): The assertion assert_eq!(analysis.metrics.effective_tokens, analysis.metrics.token_usage) is vacuous. In the implementation both fields are unconditionally set to the same total_tokens value, so the comparison is structurally equivalent to assert_eq!(x, x) — it can never fail and provides no regression protection. Separately, assert!(analysis.metrics.turns > 0) would pass even if the fixture only produced 1 turn instead of the expected 2.

  • mixed_ndjson_file_filters_each_signal_and_ignores_unrelated_entries (missing.rs): The assertion assert!(noops.iter().all(|report| report.tool.is_none())) is vacuous. extract_noops unconditionally sets tool: None on every NoopReport it constructs — the assertion is tautologically true regardless of the input data and can never expose a regression.

Changes

File Test Action Reason
otel.rs analyze_otel_reads_otel_only Rewritten Replaced self-referential field comparison with pinned token count from fixture (33 185) and exact turn count (2)
missing.rs mixed_ndjson_file_filters_each_signal_and_ignores_unrelated_entries Rewritten Replaced always-true tool.is_none() loop with assertions on noop context values and the descriptionreason fallback path

Verification

  • cargo test: all 1 761 unit + 135 integration tests pass ✅
  • cargo clippy --all-targets --all-features: no warnings ✅

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • spsprodeus21.vssps.visualstudio.com
  • spsprodweu4.vssps.visualstudio.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "spsprodeus21.vssps.visualstudio.com"
    - "spsprodweu4.vssps.visualstudio.com"

See Network Configuration for more information.

Generated by Test Reducer · sonnet46 2.9M ·

same total_tokens value in the implementation, making the original
  comparison self-referential and unable to catch regressions; also
  pin turns == 2 from the fixture instead of just > 0.

  with assertions on actual noop context and reason values — extract_noops
  unconditionally sets tool: None, so the original assertion could never
  fail; new assertions verify context routing and the description→reason
  fallback for the third noop entry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jamesadevine jamesadevine marked this pull request as ready for review June 7, 2026 08:01
@jamesadevine jamesadevine merged commit 47c46c7 into main Jun 7, 2026
@jamesadevine jamesadevine deleted the test-reducer/fix-vacuous-otel-missing-assertions-4323309f404c66da branch June 7, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant