You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An authored gold corpus for find_implementations channel accuracy (benchmarks/goldset/). Every implementation relation — declared subclasses, duck-typed conformers, decorator-registered handlers — and every deliberate false-positive trap (a module-homonym base class, same-name-different-domain methods like electrical/legal charge, substring decorator matches) is labeled in gold.json with a per-pair rationale, so ground truth is exact by construction.
A reproducible measurement harness (benchmarks/goldset/measure.py) writing per-channel precision/recall to benchmarks/provenance/channel_accuracy.json. First measured round: AST 0.833 precision, duck 0.6, decorator 0.6 — recall 1.0 across all channels. The shipped priors survive contact with measurement: 0.85 declared vs 0.833 measured for AST, 0.65 vs 0.6 for duck, and the 0.45 decorator prior is more conservative than measured.
The measurement re-runs in CI: the committed artifact must equal a live re-measurement and the corpus content hash — the numbers cannot drift from the reproducible run.
Registry and responses carry the measured reference. Each heuristic channel's provenance entry gains measured_ref beside its declared ranking prior, surfaced in _meta.confidence_provenance. Operating constants deliberately stay declared: recalibrating ranking to a small-n corpus would trade honest stability for false precision, and the artifact states its scope (authored-pattern discrimination, not in-the-wild base rates).