LLM model registry needs evidence review

[github-actions]

Maint-77 found an overdue/unproved selection or provider catalog drift.
Catalog additions are candidates only and must not be auto-selected.

Follow docs/MODEL_SELECTION_POLICY.md: refresh facts, run the paired
adjudicated benchmark, attach evidence, and update the explicit selection
and review date. Maint-68 then propagates the registry.

Findings:

{
  "fresh": false,
  "ok": true,
  "blocking": [],
  "advisory": [
    {
      "kind": "review_overdue",
      "detail": "registry review is overdue by 3 day(s) (review_by=2026-07-24, today=2026-07-27)."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/openai review was due 2026-07-24."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/anthropic review was due 2026-07-24."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/github-models review was due 2026-07-24."
    }
  ],
  "findings": [
    {
      "kind": "review_overdue",
      "detail": "registry review is overdue by 3 day(s) (review_by=2026-07-24, today=2026-07-27)."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/openai review was due 2026-07-24."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/anthropic review was due 2026-07-24."
    },
    {
      "kind": "provisional_overdue",
      "detail": "selection verifier-balanced/github-models review was due 2026-07-24."
    }
  ]
}

Catalog discovery:

{
  "schema": "workflows-model-catalog-discovery/v1",
  "generated_at": "2026-07-27T06:31:31.505067+00:00",
  "drift": true,
  "providers": [
    {
      "provider": "openai",
      "status": "current",
      "added_candidates": [],
      "removed_from_catalog": [],
      "observed_count": 129,
      "note": "Catalog changes require benchmark review; they do not auto-promote models."
    },
    {
      "provider": "anthropic",
      "status": "drift",
      "added_candidates": [
        "claude-opus-5"
      ],
      "removed_from_catalog": [],
      "observed_count": 11,
      "note": "Catalog changes require benchmark review; they do not auto-promote models."
    },
    {
      "provider": "github-models",
      "status": "current",
      "added_candidates": [],
      "removed_from_catalog": [],
      "observed_count": 15,
      "note": "Catalog changes require benchmark review; they do not auto-promote models."
    }
  ]
}

Run: https://github.com/stranske/Workflows/actions/runs/
30243010697

[stranske]

Closer audit — every recorded finding is satisfied on main by merged PR #2852; closing

All four findings in this issue are date comparisons, and each is now satisfied on main. Checked directly against config/model_registry.json:

finding in this issue	state on main today
review_overdue: registry review_by=2026-07-24	.review_by = 2026-08-30
provisional_overdue: verifier-balanced/openai due 2026-07-24	.selections[0].review_by = 2026-08-30
provisional_overdue: verifier-balanced/anthropic due 2026-07-24	.selections[1].review_by = 2026-08-30
provisional_overdue: verifier-balanced/github-models due 2026-07-24	.selections[2].review_by = 2026-08-30

The 2026-07-24 dates survive only in .selection_history[], which is the superseded record. The change landed in merged PR #2852 (2026-07-31T16:20:00Z), which advanced the three verifier-balanced provisional selections to the current catalog generation (gpt-5.4→gpt-5.6-terra, claude-opus-4-6→claude-sonnet-5, codex-mini-latest→openai/gpt-5).

Nothing was auto-promoted. This run was advisory throughout (ok: true, blocking: []), and #2852 explicitly kept all three selections status: provisional, recording that the only evidence in the registry remains catalog-review-2026-07-10 (catalog-only, which cannot approve a selection) and that the paired adjudicated pilot is still the sole path to approved.

The remaining benchmark debt is already tracked and is owner-gated elsewhere. The catalog-drift advisory (anthropic added claude-opus-5) is candidates-only by policy. Producing the adjudicated benchmark evidence is blocked on the same model-access decision that holds #2768 (candidate availability) and #2819 (auto-promote/rollback policy), both currently await-human. A duplicate tracker here adds no signal.

maint-77-model-registry-freshness.yml runs weekly (Mon 05:20 UTC); the next run on 2026-08-03 re-files automatically if any blocking or overdue condition returns. Closing as resolved.