Skip to content

Expand scientific validation beyond the planted cpnet comparison corpus #5

Description

@simoneSantoni

Motivation

The current audit uses three small planted networks and cpnet commit 6aad458a6d434a3617d33e74f7163d514a27fecb. It is useful causal evidence, but it is not broad scientific validation and cpnet is not a correctness oracle.

Proposed validation matrix

  • Add direct mathematical or exhaustive small-graph oracles where feasible.
  • Add canonical networks from the cited papers with provenance and expected summary statistics.
  • Sweep synthetic size, density, noise, core fraction, pair count, weight heterogeneity, and directionality.
  • Repeat stochastic algorithms over documented seed ensembles and report uncertainty.
  • Compare against additional maintained implementations when mathematical estimands match.
  • Separate correctness checks from concordance and recovery benchmarks.

Acceptance criteria

  • Every algorithm has at least one independent oracle or literature-backed fixture.
  • Dataset provenance, licenses, preprocessing, seeds, and package revisions are recorded.
  • Validation reports score semantics before comparing outputs.
  • A small deterministic subset runs in CI; the full study is reproducible offline.
  • Results are summarized in versioned machine-readable data and human-readable documentation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions