Portal v0.32.0 — Real-agent MCP evals and symptom-language action search

Real agents now certify the MCP surface before it ships, search understands symptom language, and a dispatch can carry its reasoning chain.

Real-agent conformance evals

A scheduled workflow drives the real Claude Code and Codex CLIs — not mocks — through a fail-closed loopback relay against a live stack. The relay alone holds the API key; the agent process never sees it. A run hard-fails on policy-blocked calls, invalid mutation arguments, a run_action without a prior get_action for the same action, placeholder reasons, and runs not driven to a terminal status. Required outcomes score as equivalence groups, so the eval judges results, not one recipe — an API change that confuses a model now fails a build instead of a customer.

Symptom-language action search

The production packs carry 562 operator phrases across 256 actions, written under an honesty contract: symptom language an operator would type, never capability claims the action can't back. Ranking now filters stop words, matches word prefixes in both directions, and weighs rare words over common ones; a small curated synonym map expands shorthand like db, mem, and k8s. On a deterministic benchmark against the production catalog, page-one recall for symptom queries ("the db is slow") went from 46% to 100%.

The justification chain

run_action accepts optional evidence (what the agent already observed) and expected (the outcome that would confirm its hypothesis) beside a reason that can now run to 2000 characters. Both are deliberately unenforced — the eval measures whether models send them — and the approval screen and run details render the chain, so a reviewer sees the basis and the prediction, not just the request.

MCP API

• Paginated reads return a copy-ready next call instead of a bare cursor; an agent continues by echoing one object.
• list_runners inlines each runner's dispatchable pack ids — one call answers what a named host can do.
• Actions can opt into typed JSON results, dispatched against the pinned trusted descriptor.

Fleet and registry

• The registry serves the catalog compact and gzip-encoded behind a CDN — about a tenth of the previous transfer. Pack tarballs stay unencoded on purpose: they are already gzip content, and an encoding layer would break content-hash pinning.
• Runner access is explicit: a member is scoped to the runners and groups they may use.
• A missing client binary travels as separate host readiness evidence — the action stays advertised for manifest verification and is simply not offered for dispatch.

Also

ChatGPT connector OAuth consent renders under its sandboxed CSP; empty-runner onboarding points at installing a pack catalog; Enterprise plans name their dedicated Slack support channel; the Nomad Autopilot health action is fixed.