v1.13.0 — LLM eval harness: the deep-review layer is now measured

Added

LLM eval harness (eval/) — Pierre's Phase 4 deep review (28 semantic checks, incl. the 6 heuristic Tier B checks) is now measured against a human-labelled dataset, not just described. Additive and opt-in: nothing in the default scan path changes, no new network calls, verdict logic untouched.

• Dataset: 21 human-labelled cases — positive+negative pairs per covered check, deliberate false-positive traps, borderline cases, pre-fetched-URL cases for the privacy-policy check. Unconfirmed labels report as UNLABELED and never enter headline metrics.
• Runner (eval/run.sh): pinned model + generation params recorded per run, --repeat for consistency measurement, response caching with resume. Per-model baseline dirs with a guard that refuses to mix models in one cache.
• Offline scorer (eval/score.py → docs/llm-scorecard.md): per-check and per-tier precision/recall/F1, Tier-B false-positive rate, majority-vote scoring, unanimity-based consistency.
• Committed baselines: Claude Sonnet 5 and Claude Opus 4.8, both 21/21 correct (Tier-A F1 1.00, Tier-B FP rate 0.00). Consistency: 1.00 (Sonnet) vs 0.95 (Opus — one pass/not-applicable wording drift). Sonnet is the pinned default: equally accurate here, more consistent, 2.5× cheaper.
• CI: blocking offline gate (dataset validity, scorecard freshness, Tier-A F1 ≥ 0.80 floor on the committed baseline) + a non-blocking live smoke job.

Honest caveat, straight from the scorecard: these numbers measure fidelity to this project's own labels on minimal synthetic fixtures — not agreement with Apple's actual review decisions. Counts are printed next to every rate so small samples stay visible.

Full changelog: https://github.com/berkayturk/appstore-precheck/blob/main/CHANGELOG.md