phase5-B: link-check gate + fix validate-repo-meta binary-URL decode bug (TDD)

[rafael5]

Summary

Phase 5 Track B per phase5-plan.md §3. Gate 2 of 4. Bundles two pieces because B0 was a documented prerequisite for B1+B2.

B0 — validate-repo-meta.py binary-URL fix (Phase-4 carry-over)

Pre-Track-B, resolve_one() blindly UTF-8-decoded every fetched body. Any exposes.* URL pointing at a binary asset (e.g. m-dev-tools-mcp's v0.1.0 wheel) crashed with UnicodeDecodeError; the documented workaround was ARGS=--no-resolve on every call.

Fix: detect non-text extensions (.whl / .tar.gz / .tgz / .zip / .gz) and skip the body-read entirely — HEAD-equivalent semantics. Text URLs unchanged.

• New _BINARY_EXTENSIONS constant + _is_binary_url() helper.
• resolve_one() branches on binary-ness; binary URLs only check "did the server answer?".
• tests/test_validate_repo_meta.py — 5 cases pinning the behaviour (text decode + JSON-validate, invalid-JSON text, binary .whl no-decode, .tar.gz no-decode, binary-URL transport failure still surfaces).

Verified: make check-repo-meta META=…/m-dev-tools-mcp/dist/repo.meta.json in full-resolve mode now returns OK against the v0.1.0 wheel URL.

B1 + B2 — check-links.py

Walks every URL surfaced by the catalog:

Source	Surfaces
profile/llms.txt	Markdown link targets (line number captured as label)
profile/tools.json	top-level $schema + every tools.<key>.<field>_url value
profile/task_index.json	top-level $schema + every doc field on every category/row

Per-URL: HEAD with 15s timeout; 405→GET fallback; 200/301/302 = OK (urllib follows redirects when --allow-redirects is on, default). Anything else = FAIL with HTTP status + reason captured.

Modes:

• --offline — inventory only. No network. Status column reads INVENTORIED. Used by per-PR CI.
• default — full HEAD walk. Used by the weekly cron firing.

Exit codes: 0 all OK / INVENTORIED; 1 any FAIL; 2 fixture error.

13 new TDD cases (tests/test_check_links.py):

Surface	Cases
Inventory	3 (tools.json *_url, task_index doc, llms.txt Markdown links)
check_url()	5 (200 OK, 404 FAIL, 405→GET fallback, DNS failure, redirect-follow OK)
Driver	2 (worst-first aggregator, all-OK case)
CLI	3 (--offline must NOT touch network — patched urlopen to raise; 404 → rc=1; smoke against committed catalog)

B3 — Make + CI wiring

• make check-links — invokes the script with --offline (PR-mode).
• .github/workflows/ci.yml: PR-mode adds a Link inventory step to the check job. The weekly cron firing on the handshake job adds a parallel step running the full live HEAD walk so a broken upstream URL surfaces within 7 days.

Verified locally

• pytest profile/build/ — 84 / 84 (66 prior + 5 validate-repo-meta + 13 check-links)
• make check-links — clean (58 URLs catalogued offline)
• make check-freshness — still clean (Track A unchanged)
• make check-repo-meta full mode (no --no-resolve) — OK against the v0.1.0 wheel URL

What's NOT in this PR

• JSON-body validation in the cron firing — check-links only HEAD-checks; deeper "is the body well-formed" lives in build-catalog + validate-catalog. Duplicating that path here would be churn.
• Retry-with-backoff for transient 5xx on the cron firing — phase5-plan.md §9 explicitly defers this until CI demonstrates flake.

Test plan

• All 18 new pytest cases pass (5 + 13)
• Full suite stays green (84 / 84 total)
• Both make targets exit 0 on main state
• Pre-existing make check-repo-meta calls (which still use --no-resolve in places) continue to work — the fix is purely additive
• CI green