feat: move the runtime onto the agent and stamp the agent config onto the trace

[mikasenghaas]

Summary

• Runtime is an agent field: runtime moves from HarnessConfig to AgentConfig — where a run provisions its box is the seat's placement decision, not the harness program's. It addresses as --env.<role>.runtime.* ([env.agent.runtime] in TOML); the harness config keeps its own fields nested under --env.<role>.harness.*. A harness-level runtime key now fails as an unknown field.
• The trace records the agent losslessly: AgentInfo is {config, runtime, name, trainable}. config is the resolved AgentConfig the rollout ran as — the exact value that rebuilds the agent (make_agent(trace.agent.config)) — embedded as a plain typed field (the configs package sits below the record, so no forward references). runtime is the box it ran in (the resolved config plus the provisioned resource ID and borrowed), moved off the top-level trace under the agent that owns it (Trace.runtime stays as a read-only property).
• Strict vs. wire reads are the reader's explicit choice (mirroring WireTaskData): a bare Trace is the strict read — agent.config validates as AgentConfig, narrowing the harness by id, so its package must be importable — while WireTrace/WireEpisode parameterize the new AgentConfigT with WireAgentConfig (extra-allow WireHarnessConfig inside), so consumers without the run's packages (e.g. a trainer) load any record plugin-free, knobs readable via model_extra. The serve wire and replay read loose; CLI/config parsing stays strict.
• Configs separate from logic: the main abstractions' config classes move into a verifiers/v1/configs/ package — one module per domain (configs.agent, configs.env, configs.harness, configs.task, configs.taskset, configs.judge, configs.retries) — and the CLI entrypoints' run configs move to configs/cli/. Config modules import only types and other configs, so any module (the trace record included) can embed them without import cycles. Hard cut internally; the public vf.* surface is unchanged.
• The env's subprocess warning, _runs_local, the agentic-judge container check, and the dashboard read each seat's runtime; runtime overrides get their own overview row.
• The debug CLI (agent-less traces) stamps an explicit AgentInfo(name="debug", trainable=False) seat carrying the runtime policy, and lands the provisioned box on it — persisted debug records keep the box (resource id, resolved type, cache status) under the same shape as eval records.
• TRACE_VERSION bumped to 3.

Breaking

• --env.<role>.harness.runtime.* → --env.<role>.runtime.* (TOML: runtime moves from [env.agent.harness] to [env.agent.runtime]). The old placement fails validation as an unknown harness field.
• HarnessConfig.runtime is gone; code reading harness.config.runtime should use the agent's config (AgentConfig.runtime / Agent.runtime_config).
• Trace records (TRACE_VERSION 3): trace.agent is {config, runtime, name, trainable} — agent.model/agent.sampling/agent.harness become agent.config.model/.sampling/.harness, and the top-level trace.runtime field moves to agent.runtime (still readable as the Trace.runtime property). Custom harness knobs serialize with the record; read them plugin-free via WireTrace/WireEpisode (WireAgentConfig/WireHarnessConfig), or fully typed via a bare Trace read where the harness package is importable.
• Internal imports of config classes move to verifiers.v1.configs.* (e.g. from verifiers.v1.configs.harness import HarnessConfig); the old logic-module paths no longer export them. vf.* names are unchanged.

Verification

• Rollout smoke test: the record carries agent.config (model, narrowed harness, runtime policy, resolved sampling) and agent.runtime (type, resource id, borrowed); Trace.model_validate round-trips typed (NullHarnessConfig narrows back), and make_agent(trace.agent.config) rebuilds an equivalent agent.
• uv run eval @ configs/<each>.toml --dry-run passes for every eval config.
• uv run pytest tests/v1 -k "not docker and not prime and not modal" passes (excluding the docker/prime legs plus two pre-existing macOS-only failures — rlm/kimi-code installs hard-code flock — all unrelated and failing on main too).
• uv run ruff check . and uv run pre-commit run --all-files pass.

🤖 Generated with Claude Code

Note

Move runtime config from HarnessConfig to AgentConfig and stamp it onto traces

• Runtime isolation policy (subprocess, docker, prime) is now configured at AgentConfig.runtime instead of HarnessConfig.runtime. CLI flags change from --env.<agent>.harness.runtime.type to --env.<agent>.runtime.type.
• AgentInfo in traces now carries the full AgentConfig (including resolved harness and sampling) under agent.config, and runtime info is recorded at agent.runtime instead of a top-level trace.runtime field.
• Trace schema version bumped from 2 to 3; serialized traces no longer include a top-level runtime field.
• Config types (HarnessConfig, TaskConfig, TasksetConfig, RetryConfig, JudgeConfig, AgentConfig) are relocated from their runtime modules into a new verifiers/v1/configs/ package, with CLI-specific configs under verifiers/v1/configs/cli/.
• Risk: breaking change to trace schema and all import paths for config types; TOML config files referencing [env.agent.harness.runtime] must be updated to [env.agent.runtime].

Changes since #2106 opened

• Introduced WireAgentConfig and WireHarnessConfig classes that bypass plugin resolution and preserve extra harness-specific fields, and added a field_validator on AgentInfo.config that validates dict inputs as WireAgentConfig instead of AgentConfig [20d01f0]
• Made Trace and AgentInfo models generic over a new AgentConfigT type parameter [9c27ec5]
• Made Episode model generic over a third type parameter AgentConfigT [9c27ec5]
• Updated WireTrace and WireEpisode type aliases to specify WireAgentConfig as the concrete agent configuration type [9c27ec5]
• Updated RunGroupResponse and EnvClient.run_group to use WireTrace type [9c27ec5]
• Updated run_replay function to read episodes with WireAgentConfig generic parameter [9c27ec5]
• Pinned ruff version to 0.15.21 in the .github/workflows/style.yml CI workflow [9d158c8]

📊 Macroscope summarized 9b4cd26. 3 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

---

Note

High Risk
Breaking changes to eval TOML/CLI (harness.runtime → agent.runtime), trace JSON schema v3, and internal config import paths affect every rollout consumer and saved runs.

Overview
Runtime placement moves from the harness to the agent seat: RuntimeConfig lives on AgentConfig (--env.<role>.runtime.*, TOML [env.agent.runtime]), not under HarnessConfig. Harness configs keep program knobs only; pairing/validation and env warnings read each seat's runtime.

Trace records bump to TRACE_VERSION 3: AgentInfo carries the resolved agent.config (full AgentConfig, including harness and sampling) plus agent.runtime (provisioned box). The old top-level trace.runtime field is removed (a Trace.runtime property still reads agent.runtime). WireTrace / WireEpisode use WireAgentConfig / WireHarnessConfig so replay and trainers can load records without resolving harness plugins.

Config vs logic: pydantic config types move into verifiers/v1/configs/ (domain modules) and verifiers/v1/configs/cli/ (eval/serve/replay/etc.); runtime modules import from there. Public vf.* re-exports are preserved; internal imports and docs/skills reference the new paths.

Misc: sample TOMLs and tests switch to runtime on agents; eval dashboard shows per-seat runtime overrides; debug traces stamp a minimal AgentInfo; CI pins ruff 0.15.21 to match uv.lock.

^{Reviewed by Cursor Bugbot for commit 9d158c8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR restructures the configuration hierarchy by moving runtime from harness to agent level, reorganizes config modules into a new directory structure, and changes the trace schema (version bump 2→3). While largely mechanical, the scope of architectural changes to core configuration infrastructure warrants human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c5c62d0. Configure here.}