Refactor example_db: stored ground truth, human-friendly JSON, no static fixture#696
Refactor example_db: stored ground truth, human-friendly JSON, no static fixture#696
Conversation
Replace petgraph's internal serde format with minimal representations:
- SimpleGraph: {"num_vertices": N, "edges": [[u,v],...]}
- DirectedGraph: {"num_vertices": N, "arcs": [[u,v],...]}
Update all .inner. graph accessor patterns in the Typst paper (24 sites).
Regenerate examples.json fixture with new format.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace closure-based ModelExampleSpec with pure data: Box<dyn DynProblem> instance, optimal_config, and optimal_value (serde_json::Value). - Add evaluate_json to DynProblem trait - Simplify ModelExample: remove SampleEval, samples field - Migrate all 44 model spec files to new format - No solver runs at spec construction time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove all solver-calling helpers from specs.rs (direct_best_example, direct_satisfying_example, direct_ilp_example, etc.). Add rule_example_with_witness helper. Migrate all 35 rule spec files to use hardcoded solution pairs instead of solver calls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- build_example_db() now computes from specs directly (fast, no solver) - Delete static fixture file and regeneration binary - Add export_examples binary that writes to docs/paper/data/ (gitignored) - Update paper pipeline: make paper runs export_examples first - Update Typst paper for new ModelExample format (single optimal_config instead of Vec<SampleEval>) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove normalization helpers and fixture-vs-computed verification tests (tautological now that build == compute). Add model_specs_are_self_consistent test (evaluate_json must match stored optimal_value) and rule_specs_solution_pairs_are_consistent test. Fix MIS/SimpleGraph/One optimal_config (was invalid IS on Petersen graph) and MIS/SimpleGraph/i32 optimal_value (was 12, correct is 10). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clarify that expected outcome is stored as ground truth and cross-validated by ILP. Allow dead_code on ModelExampleSpec (id field is used by test code). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve conflicts: - example_db/fixtures/examples.json: deleted (our branch removed it) - strong_connectivity_augmentation.rs: adopt main's new instance, convert to new spec format - longestcommonsubsequence_ilp.rs: adopt main's new LCS constructor, compute correct witness Convert 8 new model files from main's old `build:` closure format to new `instance: Box::new(...)` + `optimal_config` + `optimal_value` format: hamiltonian_circuit, longest_common_subsequence, sequencing_with_release_times_and_deadlines, staff_scheduling, string_to_string_correction, sum_of_squares_partition, consecutive_sets, minimum_cardinality_key. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR standardizes graph JSON serialization, removes the embedded example-db fixture workflow in favor of in-code “ground truth” specs (optimal configs/values + rule witnesses), and adds helper APIs to solve type-erased problems via reduction to ILP (used by the CLI and tests).
Changes:
- Implement custom
serdeJSON formats forSimpleGraph/DirectedGraph({num_vertices, edges}/{num_vertices, arcs}) and update affected tests/CLI fixtures. - Replace example-db fixtures + “compute” pipeline with spec-driven construction using
DynProblem,optimal_config,optimal_value, and pre-stored rule witness pairs. - Add
DynProblem::evaluate_jsonandILPSolver::solve_via_reduction, and switch CLI ILP solving to use the new API.
Reviewed changes
Copilot reviewed 110 out of 112 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/topology/graph.rs | Custom Serialize/Deserialize for SimpleGraph using {num_vertices, edges} |
| src/topology/directed_graph.rs | Custom Serialize/Deserialize for DirectedGraph using {num_vertices, arcs} |
| src/unit_tests/topology/graph.rs | Add JSON roundtrip/format tests for new SimpleGraph schema |
| src/unit_tests/topology/directed_graph.rs | Add JSON roundtrip/format tests for new DirectedGraph schema |
| src/unit_tests/models/graph/multiple_choice_branching.rs | Update test JSON inputs to new directed graph schema |
| src/registry/dyn_problem.rs | Add evaluate_json + require Metric: Serialize; use variant_to_map helper |
| src/solvers/ilp/solver.rs | Add solve_dyn and solve_via_reduction for type-erased solving via ILP reductions |
| src/export.rs | Replace samples/optimal with optimal_config/optimal_value; remove SampleEval |
| src/example_db/specs.rs | Redefine model specs as DynProblem + optimal ground truth; add rule_example_with_witness |
| src/example_db/model_builders.rs | Build exported ModelExample from ModelExampleSpec (instance JSON + optimal data) |
| src/example_db/rule_builders.rs | Remove compute-time rule example tests; build rule examples directly from specs |
| src/example_db/mod.rs | Remove fixture loading/computation split; build DB directly from specs |
| src/unit_tests/export.rs | Refactor export tests (shared sample DB + temp dir helper; new model example fields) |
| src/unit_tests/example_db.rs | Update tests to new model fields; add self-consistency / witness validation tests |
| src/rules/travelingsalesman_qubo.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/travelingsalesman_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/spinglass_qubo.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/spinglass_maxcut.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/sat_minimumdominatingset.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/sat_maximumindependentset.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/sat_ksat.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/sat_coloring.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/sat_circuitsat.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/qubo_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/minimumvertexcover_minimumsetcovering.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/minimumvertexcover_maximumindependentset.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/minimumsetcovering_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/minimumfeedbackvertexset_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/minimumdominatingset_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximumsetpacking_qubo.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximumsetpacking_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximummatching_maximumsetpacking.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximummatching_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximumindependentset_maximumsetpacking.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/maximumindependentset_maximumclique.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximumclique_maximumindependentset.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/maximumclique_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/longestcommonsubsequence_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/ksatisfiability_subsetsum.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/ksatisfiability_qubo.rs | Switch canonical rule example specs to explicit SolutionPair witnesses |
| src/rules/knapsack_qubo.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/ilp_qubo.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/factoring_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/factoring_circuit.rs | Replace computed witness generation with explicit SolutionPair witness |
| src/rules/coloring_qubo.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/coloring_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/circuit_spinglass.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/circuit_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/rules/binpacking_ilp.rs | Switch canonical rule example spec to explicit SolutionPair witness |
| src/models/set/set_basis.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/minimum_set_covering.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/minimum_cardinality_key.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/maximum_set_packing.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/exact_cover_by_3_sets.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/consecutive_sets.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/set/comparative_containment.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/sum_of_squares_partition.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/string_to_string_correction.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/staff_scheduling.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/shortest_common_supersequence.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/sequencing_within_intervals.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/sequencing_with_release_times_and_deadlines.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/paintshop.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/multiprocessor_scheduling.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/minimum_tardiness_sequencing.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/longest_common_subsequence.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/misc/factoring.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/undirected_two_commodity_integral_flow.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/traveling_salesman.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/strong_connectivity_augmentation.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/steiner_tree.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/spin_glass.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/partition_into_triangles.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/multiple_choice_branching.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/minimum_vertex_cover.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/minimum_sum_multicenter.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/minimum_multiway_cut.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/minimum_feedback_vertex_set.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/minimum_dominating_set.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/maximum_matching.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/maximum_independent_set.rs | Convert canonical model examples to instance + optimal_config/optimal_value |
| src/models/graph/maximum_clique.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/maximal_is.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/max_cut.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/length_bounded_disjoint_paths.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/kcoloring.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/isomorphic_spanning_tree.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/hamiltonian_path.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/hamiltonian_circuit.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/directed_two_commodity_integral_flow.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/bounded_component_spanning_forest.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/biconnectivity_augmentation.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/biclique_cover.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/graph/balanced_complete_bipartite_subgraph.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/formula/sat.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/formula/ksat.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/formula/circuit.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/algebraic/qubo.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/algebraic/ilp.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/algebraic/closest_vector_problem.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| src/models/algebraic/bmf.rs | Convert canonical model example to instance + optimal_config/optimal_value |
| problemreductions-cli/tests/cli_tests.rs | Update CLI JSON fixtures to new graph schema |
| problemreductions-cli/src/dispatch.rs | Use solve_via_reduction; update graph JSON parsing assumptions in tests |
| examples/regenerate_fixtures.rs | Remove fixture regeneration example binary |
| examples/export_examples.rs | Add example-db export binary for paper data generation |
| docs/paper/reductions.typ | Update paper to load generated examples JSON and new model/rule fields |
| Makefile | Remove fixture regeneration target; paper build now generates examples data on demand |
| Cargo.toml | Swap example binary registration to export_examples |
| .gitignore | Ignore generated docs/paper/data/ artifacts; stop whitelisting fixture JSON |
| .github/ISSUE_TEMPLATE/problem.md | Clarify that examples are stored as ground truth and ILP-cross-validated |
| .claude/skills/final-review/SKILL.md | Update final-review checklist guidance |
| src/unit_tests/models/graph/multiple_choice_branching.rs | Update embedded JSON fixtures to match new directed graph schema |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for spec in specs { | ||
| let actual = spec.instance.evaluate_json(&spec.optimal_config); | ||
| assert_eq!( | ||
| actual, spec.optimal_value, | ||
| "Model spec '{}': evaluate(optimal_config) = {} but stored optimal_value = {}", | ||
| spec.id, actual, spec.optimal_value | ||
| ); |
src/unit_tests/example_db.rs
Outdated
| for pair in &example.solutions { | ||
| assert!( | ||
| !pair.source_config.is_empty(), | ||
| "Rule {label} has empty source_config" | ||
| ); | ||
| assert!( | ||
| !pair.target_config.is_empty(), | ||
| "Rule {label} has empty target_config" | ||
| ); |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #696 +/- ##
==========================================
+ Coverage 97.23% 97.25% +0.01%
==========================================
Files 316 316
Lines 41105 41058 -47
==========================================
- Hits 39970 39930 -40
+ Misses 1135 1128 -7 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…refactor # Conflicts: # src/example_db/fixtures/examples.json
Summary
Closes #680
{"num_vertices":N,"edges":[[u,v],...]}instead of petgraph internalsBox<dyn DynProblem>+ optimal config + value) — no solver calls at construction timereduce_to()only, no solverexamples.jsonremoved — generated on demand viaexport_examplesbinary for paper exportevaluate_json(optimal_config) == stored optimal_valueTest plan
cargo test --features "ilp-highs example-db" -- --include-ignored)cargo clippy --features "ilp-highs example-db" -- -D warnings)make paperbuilds PDF successfully (requires Typst)🤖 Generated with Claude Code