Skip to content

fix(quickcheck): omit empty categories from ObservationSummary debug rendering - #3992

Merged
bobzhang merged 4 commits into
mainfrom
hongbo/qc-observation-debug-render
Aug 7, 2026
Merged

fix(quickcheck): omit empty categories from ObservationSummary debug rendering#3992
bobzhang merged 4 commits into
mainfrom
hongbo/qc-observation-debug-render

Conversation

@bobzhang

@bobzhang bobzhang commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

In the snapshot-first testing workflow, a report's Debug rendering is effectively the API surface — and every classes-only coverage snapshot (the common case when using classify) currently carries a noisy labels: {} field, e.g. in a downstream project (moonbit-community/toml-parser#122):

observations={ labels: {}, classes: { "contains-table": 570, ... } }

ObservationSummary now renders via a custom Debug impl that omits empty categories, so the above reads observations={ classes: {...} } (symmetrically, labels-only summaries drop classes: {}). The text failure_report path already skipped empty categories; this aligns the Debug path.

Test plan: new classes-only rendering test; one existing labels-only snapshot updated (drops , classes: {}); moon test quickcheck 68/68; moon check --deny-warn, moon fmt, moon info clean (no interface change — the type is private).

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 7, 2026 04:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves snapshot stability/readability for QuickCheck reports by changing ObservationSummary’s Debug rendering to omit empty observation categories (labels/classes), matching the existing failure_report output behavior and reducing noise in snapshot-first workflows.

Changes:

  • Replace derive(@debug.Debug) on ObservationSummary with a custom @debug.Debug::to_repr that conditionally includes labels / classes only when non-empty.
  • Update an existing snapshot expectation to drop the now-omitted empty classes: {} field.
  • Add a new snapshot test intended to assert classes-only rendering omits the empty labels field.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
quickcheck/observation.mbt Adds a custom Debug impl for ObservationSummary that omits empty categories for snapshot-friendly output.
quickcheck/driver_test.mbt Updates one snapshot and adds a new snapshot test for the new classes-only rendering behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread quickcheck/driver_test.mbt Outdated
test "classes-only summary omits the empty labels field" {
let report = @quickcheck.report(
(_ : Int) => true,
observe=input => [@quickcheck.classify(input >= 0, "non-negative")],

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not flaky as-is (the driver's default seed=37 makes the run deterministic), but the robustness point stands — the count would drift if Int generation or the default seed ever changed, and this test only exercises the rendering shape. Fixed in 1f9fa57: the classification is now unconditional (classify(true, "observed")), so the count always equals the test count.

@coveralls

coveralls commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 5815

Coverage increased (+0.003%) to 90.323%

Details

  • Coverage increased (+0.003%) from the base build.
  • Patch coverage: 5 of 5 lines across 1 file are fully covered (100%).
  • No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 17970
Covered Lines: 16231
Line Coverage: 90.32%
Coverage Strength: 169206.38 hits per line

💛 - Coveralls

Comment thread quickcheck/observation.mbt Outdated
if !self.classes.is_empty() {
fields["classes"] = Repr(self.classes)
}
@debug.Repr::record(fields)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@debug.Repr(Map[
         .. if !self.labels.is_empty() { [ ("labels", Repr(self.labels)]},
          .. if !self.class.is_empty() { ...}
         ])

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the record is now built with conditional spreads in the literal. One deliberate deviation from the snippet: kept Repr::record(...) instead of Repr(Map(...)), since rendering the summary as a Map would quote the keys ({ "labels": ... }) and churn every observation snapshot; record keeps the unquoted field style.

@bobzhang
bobzhang enabled auto-merge (squash) August 7, 2026 06:00
bobzhang and others added 4 commits August 7, 2026 14:17
…rendering

With snapshot tests as the primary way to consume reports, the Debug
rendering is effectively API surface. A classes-only summary previously
rendered a noisy labels: {} field (and vice versa); empty categories are
now omitted, so classify-based coverage snapshots read as
observations={ classes: {...} }.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: classify unconditionally so the class count always
equals the test count and cannot drift if Int generation or the default
seed changes; the test only exercises the rendering shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: construct the field map declaratively in the literal
instead of mutating it. Repr::record is kept (rather than Repr on a
Map) so keys keep the unquoted record rendering that snapshots rely on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bobzhang
bobzhang force-pushed the hongbo/qc-observation-debug-render branch from d7f21fb to 26a8f24 Compare August 7, 2026 06:17
@bobzhang
bobzhang merged commit 5dc2be0 into main Aug 7, 2026
15 checks passed
@bobzhang
bobzhang deleted the hongbo/qc-observation-debug-render branch August 7, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants