v0.3.1

Highlights

• Model interception. Primitives to inspect, rewrite, or stop canonical vf.Request and vf.Response boundaries with @vf.intercept and @vf.stop for live monitoring of agent rollouts. (#2164, #2165, #2166, #2229, #2178).
• Persistent ACP sessions. Claude Code, Codex, Hermes, Kimi Code, OpenClaw, Pi, Pool, and RLM now use one live ACP session per rollout. Native conversation state remains intact across turns (#2291).
• Configurable task hooks. Config can plug in stop conditions, metrics, and rewards from Python modules or files. Config entries can override decorated hooks and their priority, weight, or metadata (#2309, #2323).
• Training-native episode and run artifacts. Episodes now carry task, group, run, policy, reference-logprob, and loss-weight data. Stable task keys, named run directories, exact config capture, and native Platform uploads make resume and training handoffs durable (#2278, #2357, #2358, #2409, #2429).
• Runtime and serving improvements. Eval uses elastic env serving by default and propagates cancellation to workers. Prime sandboxes default to VM isolation and support unbounded lifetimes. (#2356, #2362, #2396, #2401, #2412, #2413, #2422, #2428).

Breaking

Legacy and environment packages

• The v0 implementation moves under verifiers.legacy. Existing v0 imports remain aliased. The v1 bridge is removed, including vf.LegacyEnvConfig, EvalConfig.legacy, --legacy.id, init --v0, legacy serve routes, and row-indexed v1 resume. Use the vf-* entrypoints for v0 environments (#2303).
• Example environment IDs drop the -v1 suffix. For example, use gsm8k instead of gsm8k-v1. These packages now require Python 3.11 or later (#2303).

Runtimes and serving

• Prime VM is the default runtime. Eval and GEPA no longer default to subprocess. Validate and debug no longer default to Docker. Set the runtime explicitly to keep local execution, such as --env.agent.runtime.type subprocess for eval or --runtime.type docker for validate and debug (#2356).
• Prime sandboxes no longer have a 24-hour hard lifetime. VM sandboxes and containers without an idle timeout can run indefinitely. Use idle_timeout or explicit deletion for cleanup (#2412, #2413).
• Eval uses the env-server pool by default. The server boolean and --server flag are removed. Use --no-serve for in-process execution. run_eval now takes only the config, and run_eval_server is removed (#2422).
• rich is now an optional config block. Replace --rich false with --no-rich. Run logs now live under logs/attempt_<n>/eval.log; use the logs/latest symlink (#2422).

Run configs and records

• Run output uses output_dir / run.dir. The top-level uuid fields are removed. Use run.name, run.dir, and the process-created run.id. Resume is now a boolean flag on the saved config, not --resume <dir> (#2358).
• Saved configs use separate launch and resolved paths. Launch TOML lives at configs/<cli>.toml. Resolved JSON lives at configs/resolved/<cli>.json. Resume and replay can still read the earlier configs/<cli>.json layout (#2429).
• Episode records are the durable artifact. Episode.task is required. Run data moves from Trace.run to Episode.run. Bare-trace JSONL input, Trace.record_run, and Episode.of are removed (#2409).

Plugins and APIs

• V1 no longer installs Hub plugins automatically. Install each taskset, harness, judge, or env package before use. Plugin IDs must name installed packages; org/name[@version] IDs no longer resolve (#2390).
• ACP harnesses require live sessions. The ACP export and one-shot runner are removed. Use ACPConfig, ACPHarness, and rollout-scoped sessions (#2291).
• Custom dialects must implement interception methods. Dialect.parse_request now returns vf.Request. Dialects must implement request rewriting, response rewriting, and stream event generation. Rollout.open() can return False when a pre-open stop fires (#2164, #2165, #2166, #2229).
• Retry replay requires x-stainless-retry-count. Clients without this header resample repeated requests instead of replaying a recorded response (#2368).

Changelog

• Upload native v1 Episodes to Platform by @mrmoxon in #2278
• fix: clamp per-call input token accounting so it never goes negative by @mikasenghaas in #2297
• feat: make ServeConfig.address optional with no default by @mikasenghaas in #2301
• Fix plain HTTP CONNECT for Docker framework routes by @xeophon in #2298
• fix(v1): restore the unbounded model-call timeout by @mikasenghaas in #2304
• chore(deps): bump gitpython from 3.1.57 to 3.1.58 by @dependabot[bot] in #2305
• Use native model catalogs in agent harnesses by @xeophon in #2308
• feat: config-plugged functions (stops, metrics, rewards) by @mikasenghaas in #2309
• feat!: move v0 under verifiers.legacy (aliased), remove the v1 bridge, drop -v1 suffixes by @mikasenghaas in #2303
• fix: harden the verifiers.legacy alias finder by @mikasenghaas in #2311
• fix: revert native model catalogs in agent harnesses by @mikasenghaas in #2314
• feat(compact): one tool call per turn, notes-only carryover by @mikasenghaas in #2313
• feat: metadata-only overrides for config-plugged functions by @mikasenghaas in #2323
• fix: compact harness runs without a container by @mikasenghaas in #2329
• fix: keep host paths out of shared /tmp by @mikasenghaas in #2331
• feat: record the task data hash on the trace by @mikasenghaas in #2339
• chore: bump prime-sandboxes to 0.2.36 by @mikasenghaas in #2344
• fix(v1): isolate Bash commands from harness venv by @eigenpaul in #2251
• fix: pin MiniSWEAgent LiteLLM proxy dependency by @xeophon in #2336
• Run all ACP harnesses on persistent live sessions by @xeophon in #2291
• Fix runtime script and IPC paths by @xeophon in #2346
• Block provider network escapes in restricted runtimes by @xeophon in #2299
• Add interception example environments by @xeophon in #2178
• Add interception transport and replay by @xeophon in #2164
• Add typed model interception by @xeophon in #2165
• Stop at typed model boundaries by @xeophon in #2166
• Connect harness tool interception hooks by @xeophon in #2229
• feat: default runtime to prime vm sandbox by @mikasenghaas in #2356
• feat(v1): add stable task keys by @hallerite in #2357
• feat(v1): run configs + align resume logic with prl by @mikasenghaas in #2358
• Update coding harness and adapter versions by @xeophon in #2360
• Prevent --clean from deleting output roots by @xeophon in #2363
• feat: require an explicit retry signal to replay intercepted responses by @mikasenghaas in #2368
• Pin RLM harness to nano-rlm by @hallerite in #2373
• Revert nano-rlm harness pin, default version back to main by @samsja in #2374
• Make MCP tool tests naming-agnostic by @xeophon in #2378
• fix: use upstream ACP notification ordering by @hallerite in #2389
• chore: bump prime-sandboxes to 0.2.37 by @mikasenghaas in #2388
• feat!: remove hub auto-install of plugin packages by @mikasenghaas in #2390
• chore: bump prime-pydantic-config to >=0.4.3 by @mikasenghaas in #2391
• feat(v1): add DeepSeek Harness ACP integration by @xeophon in #2377
• Revert DeepSeek Harness ACP integration by @xeophon in #2394
• fix(mcp): support prebuilt runtime Python by @rasdani in #2395
• Support Harbor runtime environment and health checks by @xeophon in #2362
• fix(mcp): use runtime Python for readiness probe by @xeophon in #2398
• Retry interaction episodes in live tests by @xeophon in #2399
• Propagate network policy to provider capabilities by @xeophon in https://github.c...

v0.3.0

Highlights

• Multi-agent environments. An Agent you run like a function and an Env that writes the control flow between agents as plain Python — one Episode per task, with bundled single-agent, best-of-n, and agentic-judge envs (#1939) and turn-level interleaving between agents (#2049).
• ACP integration. Supported harnesses (Codex, Claude Code, ...) run through ACP adapters with resumable sessions (#2257), and stateful harnesses keep one live process across turns over runtime processes (#2249).
• Client-side tasksets. The client owns the taskset and ships each task's data to a stateless env server; resume identity is the task's content, so a changed task re-runs instead of silently matching (#2039).
• Improved performance. The interception server owns and multiplexes the live clients (#2218, #2279), and the train client's tokenizers come from a process-wide elastic renderer pool (#2218).
• New harnesses and integrations. browser_use for CDP browser control (#2192), Hermes Agent (#2173), OpenClaw (#2174), NeMo Gym V1 (#2212), pluggable skills in harnesses (#2102), and execution-time network isolation with egress allow/block lists on Docker and Prime (#2024, #2115).

Breaking

Configs

• Every run axis lives under [env]. --taskset.* / --harness.* become --env.taskset.* and per-agent --env.<agent>.* (--env.agent.harness.id on a plain run); per-run caps and retries move onto the agent (#1939).
• Runtime is an agent field. [env.agent.harness.runtime] moves to [env.agent.runtime] and HarnessConfig.runtime is gone — where a rollout provisions its box is the seat's decision, not the harness's (#2106).
• env.max_concurrent removed; serving is its own block. serve.max_concurrent bounds a worker's episodes and env.max_concurrent_agents (default 1) bounds one episode's agents; --pool.* → --serve.pool.*, --address → --serve.address, --id/--args/--extra-env-kwargs → --legacy.*; EnvServerConfig is gone (#2157).
• ServingConfig renamed to ServeConfig (same fields: pool, address, max_concurrent) (#2237).
• TrainClientConfig.pool_size replaced by multiplex, with inverted meaning. pool_size was "how many renderers"; multiplex is "concurrent rollouts per renderer" (default 256) — drop the old value to take the default, or set it to concurrency / desired_renderers (#2218).
• share_runtime removed from the agentic judge. --env.id agentic-judge is isolated judging; same-box grading is the explicit --env.id shared-agentic-judge, and the judge always follows the solver's runtime policy (#2234).

Artifacts (trace, episode)

• traces.jsonl holds episodes. Each line is one Episode — {id, env, ok, errors, traces} — written whole after the env-rollout finishes; traces gain ok and agent.name/agent.trainable (#1939).
• Trace API renames. Trace.stamp → record_run, capture_error → record_error, error → last_error; the agent_name/trainable/runtime passthroughs are removed — read trace.agent.name/.trainable/.runtime (#2172).
• The agent is stamped on the trace. trace.agent is {config, runtime, name, trainable}: agent.model/agent.sampling/agent.harness become agent.config.*, and the top-level trace.runtime field moves to agent.runtime; consumers without the run's packages read records via WireTrace/WireEpisode (#2106).
• Timing.generation → Timing.agent (GenerationSpan → AgentSpan, Trace.split_generation → split_agent_time) (#2187).
• Episode.env is a typed EnvInfo (persisted as {"id": ...} — read episode.env.id), and Episode.error → Episode.last_error (#2187).
• vf.StrictBaseModel is gone. The whole record tree parses non-strict — unknown fields are ignored instead of rejected, so additive schema evolution no longer breaks older readers (#2187).

Authoring surface

• Deprecated group rewards. @group_reward, run_group, and group resume are removed; -r n means n independent episodes, and sibling comparison lives in an env's finalize() (e.g. best-of-n) (#1939).
• Tool declaration is explicit. The Task.tools/Taskset.tools classvars, server_config, and resolve_server_config are removed — override toolsets(cls, config) (a classmethod) and construct each server with exactly the config it needs (#2199).
• Agents no longer take a live client. Agent(config, client=...)/make_agent(client=...) are removed — put the endpoint on AgentConfig.client. Harness hooks see a config-valued ctx.client (ClientConfig, not a live Client); the rollout's live client is at RolloutSession.client (#2218). Agent.run(shared_tools=...) is now Agent.run(tools=...) (#2094).
• Import moves. Config classes live in verifiers.v1.configs.* (#2106), and scoring/push/artifacts/decorators/loaders/retries move under verifiers.v1.utils.* (scoring → utils.score, push → utils.platform) (#2204); RolloutRun is renamed Rollout and takes bundled RolloutTimeouts (#2204, #2205); TaskTimeout.harness → TaskTimeout.agent (#2172). Public vf.* re-exports are unchanged.

CLI

• The serve CLI is removed. Host an env server programmatically via serve_env, or through a trainer (prime-rl's uv run env) (#2237).

Changelog

• fix(v1): don't install envs from the Hub during config parse by @eexwhyzee in #2084
• chore: bump GitHub Actions to latest major versions by @mikasenghaas in #2083
• feat(v1): add capture_patch/resolve_head git utils for SWE tasksets by @rasdani in #2054
• chore: make stable release manual-dispatch only by @mikasenghaas in #2085
• feat(v1): multi-agent api by @hallerite in #1939
• fix: rename Agent.run shared_tools kwarg to tools by @mikasenghaas in #2094
• chore: remove redundant agent artifacts + lab artifacts by @mikasenghaas in #2092
• chore: rename env-rollout to episode by @mikasenghaas in #2095
• docs: hand-written v1 agent and env docs by @mikasenghaas in #2096
• fix: point mint.json at renamed v1 env page by @mikasenghaas in #2097
• fix(v1): close agent-owned clients by @xeophon in #2098
• chore: apply automatic markdownlint fixes by @snimu in #2099
• chore: lint Markdown in pre-commit by @snimu in #2103
• feat(v1): execution-time network isolation for Docker (Linux+macOS) with allow/block egress lists by @xeophon in #2024
• feat(v1): client-side tasksets + stateless env server (v1-only) by @mikasenghaas in #2039
• feat: pluggable skills in harnesses by @mikasenghaas in #2102
• feat(v1): interleaving agents by @hallerite in #2049
• fix: sync docs to public-docs on release tags only by @mikasenghaas in #2108
• fix(v1): prepare Docker runtime once by @xeophon in #2112
• fix(v1): keep standalone Docker tool test runtimes unrestricted by @xeophon in #2113
• Fix oversized evaluation sample uploads by @xeophon in #2114
• Support Prime sandbox egress policies by @xeophon in #2115
• feat: move the runtime onto the agent and stamp the agent config onto the trace by @mikasenghaas in #2106
• feat(v1): configurable agentic-judge grading policy, uploads, and reward weights by @mikasenghaas in #2109
• feat: store raw score and weight on trace rewards by @mikasenghaas in #2119
• Fix Prime runtime config example by @xeophon in #2122
• Tear down MathRubric workers in tests by @xeophon in #2126
• Let rollout timeout own null harness generations by @xeophon in #2133
• Fix provider state graph identity by @xeophon in #2130
• fix(v1): trace Responses custom tools by @hallerite in #2129
• Remove redundant v1 structure by @xeophon in #2146
• Resolve Ruff 0.16 findings by @xeophon in #2147
• Update Ruff and ty tooling by @xeophon in #2148
• feat(v1): expose MCP tools to Codex by @hallerite in #2140
• Fix repository links in v1 docs by @hallerite in #2153
• Align Docker and Prime network policy modes by @xeophon in #2124
• Split deterministic v1 tests from live E2Es by @xeophon in #2127
• Pin standalone MCP clients to v1 ...

v0.2.1

Highlights

• New built-in harnesses. Claude Code (#1691), Pi (#2004) and Pool (#2056).
• New built-in tasksets. OpenEnv environments load as first-class tasksets (openenv) behind the verifiers[openenv] extra (#1997, #2059)
• Lazy/ infinite tasksets. Taskset.load() may now yield tasks instead of returning a finite list, enabling dynamically generated, never-ending task streams for online training (#1969). Taskset loading is also lazy end-to-end (#2023).
• Richer traces. Traces record per-call ModelCall metadata (#2061), split generation time into model vs. harness time (#2060), report usage-based and self-consistent token counts (#2040), persist advertised tool definitions for tool-use SFT (#1963), carry platform-integration provenance fields (#2053), and thread kept-set sampling masks from generate responses to Branch (#1978).
• GEPA. GEPA prompt optimization for v1 environments (#1952).
• Improved reliability. Improved MCP tool serving under high concurrency (#2062). The message graph is atomic w.r.t. harness retries (#2036), env server worker failures surface as structured errors (#1976), and the interception layer is a first-class contract with warm elastic pools by default (#1989, #2002).

Breaking

• default harness renamed to bash. Configs with harness.id = "default" must use "bash"; DefaultHarness/DefaultHarnessConfig are now BashHarness/BashHarnessConfig (#2063).
• EnvConfig.multiplex replaced by EnvConfig.interception. Interception is a pluggable config: InterceptionServerConfig, StaticInterceptionPoolConfig, or ElasticInterceptionPoolConfig (the previous multiplex behavior) (#1989).
• OpenEnv folded behind an extra. The openenv-v1 package moved to verifiers.v1.tasksets.openenv; depend on verifiers[openenv] instead (#2059).
• Harbor task timeouts ignored by default. HarborConfig.ignore_timeouts defaults to True; opt back in with --no-taskset.ignore-timeouts (#2050).
• Removals. Legacy v0 environments (#2055) and the composable search taskset (#1854, migrated to research-environments) are gone.

Changelog

• feat(v1): persist advertised tool defs on the Trace for tool-use SFT by @anravich13-cloud in #1963
• feat: warn and block a second Ctrl-C during eval cleanup by @mikasenghaas in #1972
• Fix harness docs env variable by @xeophon in #1973
• Use Luna for tool response image CI test by @xeophon in #1980
• Use a valid image fixture for tool response test by @xeophon in #1981
• Make Codex multi-agent configurable by @xeophon in #1979
• feat(v1): GEPA prompt optimization for native v1 environments by @Ziems in #1952
• feat(v1): add mmmu-v1 environment (native port of mmmu) by @xeophon in #1975
• feat(v1): support image-bearing Messages prompts by @xeophon in #1974
• fix(v1): structure env server worker failures by @xeophon in #1976
• fix(v1): install the MMMU example taskset by @xeophon in #1984
• fix(v1): tolerate an unknown service_tier in the chat dialect by @Ziems in #1986
• feat: thread kept-set sampling masks from generate responses to Branch by @faresobeid in #1978
• feat(v1): time runtime boot separately from setup by @mikasenghaas in #1990
• feat(v1): lazy and infinite tasksets (load() may yield) by @mikasenghaas in #1969
• feat: cleanup interception by @mikasenghaas in #1989
• fix(v1): docker runtime on macOS (Docker Desktop) by @snimu in #2006
• Prefer pip for uv bootstrap by @xeophon in #1996
• Use direct command for Prime workdir setup by @xeophon in #1998
• Combine uv script publication and setup by @xeophon in #1999
• fix(v1): install a server's package extras in sandbox venvs by @snimu in #2007
• Warm elastic interception by default by @xeophon in #2002
• Use direct command for Prime file writes by @xeophon in #2000
• fix: disambiguate errors and invalids in validate --rich summary by @hallerite in #2013
• fix(v1): bound null harness MCP connect at 60s by @eligotts in #1977
• fix: reset entrypoint when starting docker sandboxes by @mikasenghaas in #2015
• docs(v1): rename environments guide to tasksets by @xeophon in #2010
• Reorder interception operations by @xeophon in #2025
• Clean up replay dashboard progress rendering by @xeophon in #2026
• Reuse persisted trace runtime metadata by @xeophon in #2028
• Reuse v1 image encoding and typed user messages by @xeophon in #2029
• Add Pi harness support by @xeophon in #2004
• Make taskset loading lazy by @xeophon in #2023
• fix: parse legacy harbor schema-1.0 size strings in [environment] by @mikasenghaas in #2031
• docs(v1): add GEPA page and drop perfect_score knob by @Ziems in #2012
• feat(v1): TOOL_PREFIX = None for bare tool names by @mikasenghaas in #2035
• fix: make the message graph atomic w.r.t. harness retries by @mikasenghaas in #2036
• fix: make Trace token counts usage-based and consistent by @mikasenghaas in #2040
• chore: bump renderers to 0.1.8 by @mikasenghaas in #2042
• fix(v1): reject duplicate default harness tools by @xeophon in #2044
• feat(v1): dtype-aware routed-experts payload decode by @mikasenghaas in #2043
• feat: auto-built prime sandbox images - cache signal, build warning, dashboard mark by @mikasenghaas in #2047
• Add Claude Code harness with E2E coverage by @xeophon in #1691
• feat: ignore harbor task timeouts by default by @mikasenghaas in #2050
• chore(deps): bump mcp from 1.27.0 to 1.28.1 by @dependabot[bot] in #2051
• refactor(search): remove composable search taskset (migrated to research-environments v1) by @hallerite in #1854
• chore: remove legacy environments by @xeophon in #2055
• chore: use 'uv run init' in create-environments skill by @xeophon in #2057
• Add native OpenEnv V1 taskset support by @xeophon in #1997
• fix(v1): tolerate provider usage without cache_write_tokens by @dumko2001 in #2019
• feat(v1): add platform-integration metadata fields to Trace by @mikasenghaas in #2053
• fix(v1): fold openenv into verifiers.v1.tasksets behind an openenv extra by @mikasenghaas in #2059
• feat: record per-call model call metadata on v1 traces by @mikasenghaas in #2061
• feat(v1): split generation time into model vs. harness time by @mikasenghaas in #2060
• feat: rename the default harness to bash by @mikasenghaas in #2063
• chore: update pinned harness versions by @xeophon in #2064
• fix: mcp robustness fixes under high-concurrency by @mikasenghaas in #2062
• Add pool harness (Poolside Agent CLI) by @xeophon in #2056
• Refresh dependency requirements and optional extras by @xeophon in #2065
• chore(deps): bump prime-pydantic-config to 0.4.2 by @mikasenghaas in #2081
• chore: remove PR template by @mikasenghaas in #2082

New Contributors

• @anravich13-cloud made their first contribution in #1963
• @Ziems made their first...

v0.2.0

Highlights

• A task-centric v1 API. Verifiers v1 now separates serializable per-row TaskData from the Task class that owns lifecycle hooks, tools, metrics, rewards, and judges. A Taskset loads typed tasks, a Harness defines how the model runs, and the two compose into an environment without mixing v1 and legacy APIs.
• Portable agent execution. The same taskset can run across built-in and custom harnesses on local subprocesses, Docker, or remote Prime and Modal sandboxes. Model traffic passes through the interception layer, which adapts the harness's provider dialect, records the trace as it happens, applies sampling configuration, and supports response and tool-result interception.
• Typed authoring, tools, and scoring. Strict Pydantic configs cover tasksets, tasks, harnesses, runtimes, and evaluation. Environment authors can scaffold packages with uv run init, expose taskset- or task-scoped MCP tools, attach user simulators, and score traces with rewards, metrics, group rewards, reusable scoring helpers, and configurable LLM judges.
• A complete v1 evaluation workflow. uv run eval supports TOML configs, dotted CLI overrides, dry runs, concurrency controls, rich progress and paging, resumable runs, and platform upload. Native validate, debug, and replay entrypoints cover environment checks, interactive diagnosis, and offline re-scoring of saved traces.
• First-class traces and observability. Traces capture the message graph, task identity and data, rewards, metrics, errors, token and timing information, runtime metadata, and per-message creation timestamps. Evaluation artifacts now use traces.jsonl, with improved streaming, memory use, and dashboard reporting for long or highly concurrent runs.
• Harbor and production environment support. v1 includes a Harbor taskset base, reusable built-in harnesses and tasksets, container-aware runtime configuration, and a companion catalog of production benchmarks in research-environments.

Changelog

• Require auth for interception endpoints by @xeophon in #1304
• feat(composable): render pre-compaction branches via harness hook by @snimu in #1291
• fix: mute httpcore/httpx DEBUG when env-worker json_logging sets root level by @rasdani in #1234
• Make sandbox worker caps configurable by @rasdani in #1305
• Support setup_state return-state compatibility by @willccbb in #1308
• Update v1 Taskset/Harness docs and opencode_harbor migration by @willccbb in #1309
• bump interception client size to handle trajectories with images by @eligotts in #1310
• Fix eval summary for heterogeneous metrics by @xeophon in #1295
• Refactor deep-agents envs to v1 taskset-harness and add Wikispeedia example by @willccbb in #1317
• Add SWE debug environment by @rasdani in #1306
• Move OpenCode config into reusable harness package by @willccbb in #1318
• Remove direct prime-sandboxes dependency from rlm-swe v1 by @willccbb in #1316
• Install sandbox fn program packages by @willccbb in #1319
• opencode: write config under XDG_CONFIG_HOME by @rasdani in #1320
• feat: surface swallowed response_future errors in interception proxy by @rasdani in #1196
• apt: harden sandbox bootstrap against transient archive.ubuntu.com flakes by @rasdani in #1284
• fix(renderer-client): wrap tools in OpenAI envelope to match training distribution by @hallerite in #1307
• fixes rlm compaction prompt so we still get tito hit by @eligotts in #1322
• Fix interception keepalive env override by @rasdani in #1321
• Pin OpenCode small model to intercepted provider by @rasdani in #1323
• Avoid holding sandbox workers with run_background_job() by @rasdani in #1328
• Skip OpenCode install when binary is pre-baked by @rasdani in #1176
• fix: narrow send_cancel BaseException catch to Exception by @rasdani in #1198
• Fix Harbor Hub task root handling by @xeophon in #1345
• Pin OpenCode small model in legacy configs by @rasdani in #1327
• Remove local training config scaffolding from Verifiers by @willccbb in #1348
• retry Read file timed out by @kcoopermiller in #1350
• chore: v0.1.15.dev0 dev release by @rasdani in #1352
• Tighten BugBot releasability and dependency rules by @willccbb in #1353
• Strengthen BugBot release and dependency rules by @willccbb in #1354
• Fix v1 lifecycle handler discovery by @xeophon in #1347
• Fix ZMQ env timeouts by @xeophon in #1355
• feat(renderer-client): thread multimodal sidecar through rollout + transport by @hallerite in #1346
• Map Anthropic reasoning effort by provider by @xeophon in #1338
• chore: v0.1.15.dev1 dev release by @xeophon in #1357
• Disable OpenCode title generation in harness configs by @rasdani in #1359
• chore: v0.1.15.dev2 dev release by @rasdani in #1360
• Use background jobs for v1 sandbox commands by @xeophon in #1364
• fix: treat OverlongPromptError as stop condition in interception proxy by @mikasenghaas in #1365
• chore: v0.1.15.dev3 dev release by @mikasenghaas in #1367
• [codex] Display eval cost from Prime pricing by @willccbb in #1368
• Add repo development best practices to AGENTS, docs, and skills by @willccbb in #1361
• Tighten v1 config, bindings, and handler typing by @willccbb in #1362
• Prepare v0.1.15.dev4 release notes and version bump by @willccbb in #1370
• Add Terminus2 harness by @xeophon in #1356
• chore: v0.1.15.dev5 dev release by @xeophon in #1372
• feat: consume vllm routed experts payloads by @S1ro1 in #1363
• fix: move routed experts helpers to response utils by @S1ro1 in #1373
• Tighten v1 EnvConfig typing by @willccbb in #1371
• perf(save): skip backward-byte scan on hot path by @mikasenghaas in #1376
• chore: v0.1.15.dev6 dev release by @willccbb in #1375
• Clarify managed Lab AGENTS guidance and add AGENTS.local.md overlay by @willccbb in #1383
• Add per-eval names for duplicate environment runs by @willccbb in #1384
• Document eval name labels in skill by @willccbb in #1388
• improvement: switch verifiers PyPI publish to OIDC trusted publishing by @JannikSt in #1386
• release v0.1.15.dev7 by @JannikSt in #1389
• Modernize eval viewer TUI by @xeophon in #1393
• chore: bump renderers to 0.1.8.dev2 (supersedes #1366) by @hallerite in #1395
• Retire vf-tui entrypoint by @xeophon in #1398
• fix(save_utils): reset delta baseline on non-monotonic trajectories by @eligotts in #1400
• fix(renderer-client): translate renderers.OverlongPromptError into vf.OverlongPromptError by @hallerite in #1408
• [Router Replay]: Improve performance by removing Pydantic validation by @S1ro1 in #1394
• Revert "[Router Replay]: Improve performance by removing Pydantic validation" by @willccbb in #1422
• Support routed experts response sidecar by @S1ro1 in https://github.com/PrimeIntellec...

v0.1.14

Verifiers v0.1.14 Release Notes

Date: 05/07/2026

Highlights since v0.1.13.dev8

• Composable v1 Taskset/Harness API. Adds the verifiers.v1 authoring surface around serializable Task/State data, composable Taskset and Harness objects, and the vf.Env(taskset, harness) adapter for existing eval and training workers. The release includes lifecycle decorators, typed config objects, endpoint routing, toolsets, MCP tools, sandbox/program utilities, nested harness support, v1 docs, migration notes, and several v1 example environments, including new OpenAI Agents, LangChain Deep Agents, and DSPy RLM harness examples.
• Consistent v1 environment configuration. Eval and RL/Hosted Training TOML now share the same public projection shape through [*.args], [*.taskset], and [*.harness] sections. v1 loaders accept both mapping and model-backed config objects through a common access helper, and strict child config parsing strips loader-local routing keys at the boundary.
• Model-family starter configs. Restructures bundled eval, RL, and GEPA starter configs around model families such as Qwen 3.5, Qwen 3.5 MoE, Nemotron 3, and Llama 3, with setup mirroring the new config set into Lab workspaces.
• New client and rendering paths. Adds an OpenAI Responses API client and a renderer-backed client path for exact token rendering and multi-turn bridge metrics. The renderer implementations now live in the external renderers package, exposed from Verifiers through an optional renderers extra and client integration. Renderer clients also forward preserve_all_thinking and preserve_thinking_between_tool_calls flags into the underlying renderer.
• More rollout observability and artifacts. Adds per-turn timing through eval outputs and TUI display, token-id preservation for Nemotron client responses, GEPA system-prompt artifact export plus path-based prompt loading, and Lean guard markers with tamper-aware LeanRubric scoring.
• Release and infrastructure hardening. Adds universal locks and a 7-day PyPI freshness cooldown, scopes Hub install freshness filtering to registry packages, skips secret-backed environment tests on fork PRs, points the composable RLM harness at rlm-harness, and routes opencode AGENT_WORKDIR per rollout.

Changes included in v0.1.14 (since v0.1.13.dev8)

Features and enhancements

• Taskset Harness (v1) (#1277)
• ApiEnv examples for OpenAI Agents, LangChain Deep Agents, and DSPy RLM (#1121)
• Refactor tau2 bench into a taskset-owned v1 environment (#1293)
• Restructure example configs around model families (#1297)
• add openai responses client (#1261)
• Renderer-backed client integration via the external renderers package (#1068, #1279, #1282)
• feat(renderer-client): forward preserve_*_thinking config flags (#1298)
• feat: per-turn timing (#1182)
• Add GEPA system prompt export and path-based prompt loading (#1268)
• feat(lean): lean-guard markers + tamper-aware LeanRubric (#1271)
• token id support for Nemotron client responses (#1231)

Fixes and maintenance

• Fix v1 env config projection and typed child loader boundaries (#1294)
• Skip secret-backed environment tests for fork PRs (#1292)
• Scope uv freshness filtering to PyPI for Hub installs (#1286)
• opencode harness: route AGENT_WORKDIR per-rollout instead of baked-in (#1280)
• chore: add 7-day supply chain cooldown via uv exclude-newer (#1274)
• chore: point DEFAULT_RLM_REPO_URL to rlm-harness (#1267)
• Update Lab workspace setup guidance (#1299)

Full Changelog: v0.1.13.dev8...v0.1.14

v0.1.13.dev8

Verifiers v0.1.13.dev8 Release Notes

Date: 04/28/2026

Highlights since v0.1.13.dev7

• Per-rollout wall-clock timeout for MultiTurnEnv. New timeout_seconds: float | None kwarg bounds total rollout via asyncio.wait_for; on fire, mark_timed_out(state) writes timed_out=True, is_completed=True, stop_condition="timeout_reached". New vf-eval --timeout SECONDS CLI flag injects timeout_seconds into the env constructor (and recognizes timeout = N at the top of [[eval]] TOML tables); wins over --extra-env-kwargs. CliAgentEnv derives sandbox timeout_minutes from this (rollout deadline + 60min scoring buffer, clamped to a 24h SDK ceiling) and exposes compute_sandbox_timeout_minutes on SandboxMixin for taskset overrides via SandboxSpec(timeout_minutes=None).
• Several smaller CliAgentEnv / composable / RLM fixes.

Changes included in v0.1.13.dev8 (since v0.1.13.dev7)

Features and enhancements

• feat: rollout timeout (#1258)
• Harness.keep_trajectory_step; rlm: rlm_max_depth, include_sub_rlm_trajectories (#1253)
• TITO: treat content='' / None as equal in prefix-match + warn on MITO fallback past turn 1 (#1259)
• Harness.environment_vars: per-rollout callable, rlm: rand threshold (#1248)
• TaskSet: accept DatasetBuilder for lazy dataset construction (#1251)
• cli_agent_env: bump default poll_interval from 1s to 5s (#1255)

Fixes and maintenance

• fix: avoid TOCTOU port race in CliAgentEnv interception server (#1264)
• rlm harness: remove sandbox-side git shim (replaced by rlm tool-level block, rlm#70) (#1262)
• fix: narrow math_verify BaseException catch to specific TimeoutException (#1197)
• fix: handle dict word_list in TextArenaEnv.ta_to_hf() (#1214)
• fix: correct timing accumulation in RubricGroup score_rollout and score_group (#1215)
• composable_env: skip caches/.git/.venv when tarring upload dirs (#1257)
• swe tasksets: default ds_num_proc to None for all SWE tasksets (#1256)
• git_checkout_cache: hold per-process in-use lock so concurrent resolves don't nuke active worktrees (#1252)
• fix: prepend vllm/ to slashless OPENAI_MODEL in composable opencode harness (#1250)
• rlm harness: stage git-refusal shim into $HOME/.local/bin so scoring can run git (#1244)
• rlm harness: remove dead RLM_KERNEL_PYTHON detection block (#1242)
• SWEBenchTaskSet.setup: symlink venv at /testbed/.venv matching WORKDIR (#1241)

Full Changelog: v0.1.13.dev7...v0.1.13.dev8

v0.1.13.dev7

Verifiers v0.1.13.dev7 Release Notes

Date: 04/24/2026

Highlights since v0.1.13.dev6

• rlm_harness swaps turn-based context caps for token-based auto-compaction: new summarize_at_tokens: int | None kwarg maps to RLM_SUMMARIZE_AT_TOKENS, while rlm_max_turns_in_context / RLM_MAX_TURNS_IN_CONTEXT are removed to match upstream rlm. summarize also drops out of the default rlm_tools set. Invalid shapes fail at harness-build time instead of deep inside the sandbox.
• Reverted TaskSet.filter / .take returning Self (originally #1232) — the change broke Python 3.10/3.11 compatibility. CI now exercises the 3.10 and 3.11 test matrices so the fix can be restored with confidence.

Changes included in v0.1.13.dev7 (since v0.1.13.dev6)

Features and enhancements

• rlm_harness: add summarize_at_tokens, drop rlm_max_turns_in_context (#1236)

Fixes and maintenance

• Revert "types: TaskSet.filter / .take return Self, not TaskSet (#1232)" (#1237)
• ci: add Python 3.10 and 3.11 to the test matrix (#1237)

Full Changelog: v0.1.13.dev6...v0.1.13.dev7

v0.1.13.dev6

Verifiers v0.1.13.dev6 Release Notes

Date: 04/23/2026

Highlights since v0.1.13.dev5

• rlm_harness is now the single source of truth for RLM_* sandbox env vars. New kwargs rlm_max_turns, rlm_max_turns_in_context, rlm_exec_timeout map 1:1 onto the matching env vars on Harness.environment_vars and merge into the sandbox via ComposableEnv.build_env_vars (harness-wins). Research envs can stop setting these via ComposableEnv(environment_vars=…) — pass them through as harness kwargs instead.
• TaskSet.filter / .take now return Self, not TaskSet, so subclass types survive taskset chaining for downstream typed consumers.

Changes included in v0.1.13.dev6 (since v0.1.13.dev5)

Features and enhancements

• rlm_harness: own RLM_MAX_TURNS / _IN_CONTEXT / _EXEC_TIMEOUT env vars (#1229)

Fixes and maintenance

• types: TaskSet.filter / .take return Self, not TaskSet (#1232)

Full Changelog: v0.1.13.dev5...v0.1.13.dev6

v0.1.13.dev5

Verifiers v0.1.13.dev5 Release Notes

Date: 04/22/2026

Highlights since v0.1.13.dev4

• Made the interception proxy's streaming response resilient to upstream cuts: 10s SSE keepalive comments keep idle streams warm, per-chunk asyncio.sleep(0) forces an event-loop yield so content and close can't race the transport flush under warmup-burst contention, and transport exceptions at prepare/write/write_eof are surfaced as StreamInterrupted into state["error"] so rollouts reschedule instead of looking like clean zero-turn completions.
• Added a new experimental mini_swe_agent composable harness (pip/uv install with SHA256-verified wheel download), exported alongside existing rlm and opencode harnesses.
• Extended SandboxMixin to cover VM sandboxes in addition to containers (including GPU VMs via CreateSandboxRequest), with documentation clarifying feature parity (file I/O, background jobs, cleanup) and container-only features (port exposure, SSH).

Changes included in v0.1.13.dev5 (since v0.1.13.dev4)

Latest changes from main

• Includes the latest main changes through the interception proxy streaming resilience fix (#1194), along with the mini_swe_agent harness (#1219) and SandboxMixin VM sandbox support/docs (#1222).

Features and enhancements

• Add mini-swe-agent harness (#1219)
• Update SandboxMixin (#1222)

Fixes and maintenance

• fix: make interception proxy streaming resilient to upstream cuts (#1194)

Full Changelog: v0.1.13.dev4...v0.1.13.dev5

v0.1.13.dev4

Verifiers v0.1.13.dev4 Release Notes

Date: 04/22/2026

Highlights since v0.1.13.dev3

• RLM harness: new rlm_tools kwarg sets both Harness.tool_names (for ToolMonitorRubric) and the sandbox RLM_TOOLS env var from a single source, plus new Harness.environment_vars field merged harness-wins-on-conflict by ComposableEnv.
• Refactored experimental RLM checkout caching; DEFAULT_RLM_BRANCH renamed to DEFAULT_RLM_REF and rlm_harness(..., rlm_branch=...) renamed to rlm_ref= to reflect that any git ref (branch, tag, sha) is accepted.
• Added SandboxTimeouts dataclass centralizing per-operation sandbox HTTP timeouts.
• Expanded task coverage with SWE-rebench-V2 and a multilingual SWESmith taskset, plus a filter_fn kwarg on all tasksets for ad-hoc row filtering.
• vf-eval: renamed -d/--debug to --disable-tui and --tui to --fullscreen for clearer intent.
• RLM rollout metrics (context tokens, programmatic tool calls) exposed to verifiers and auto-merged by the composable env.

Changes included in v0.1.13.dev4 (since v0.1.13.dev3)

Features and enhancements

• vf-eval: replace -d/--debug with --disable-tui, rename --tui to --fullscreen (#1183)
• Expose RLM metrics to verifiers (#1195)
• Add streaming observability + resume to TaskSet.validate() (#1169)
• Refactor experimental RLM checkout caching (#1202)
• feat: add filter_fn kwarg to all tasksets for ad-hoc row filtering (#1199)
• feat: add multilingual SWESmithTaskSet (#1186)
• feat: add SWE-rebench-V2 TaskSet (#1187)
• HarborMCPMixin (#1146)
• feat: SandboxTimeouts dataclass — centralize per-operation sandbox HTTP timeouts (#1207)
• Run SWE-Lego eval via dataset's canonical test_cmd (#1205)
• Authenticate interception server via INTERCEPTION_SECRET (#1180)
• feat: revert agent test edits at grading (swe_lego, swe_rebench_v2) (#1212)
• AgentError: rollout_id, sandbox_id, ... (#1218)
• Remove RLM_DEFAULT_TOOL_NAMES, accept rlm_tools (#1223)
• r2e_gym: add hide_tests_from_agent flag + expose instance_id/repo aliases (#1208)
• feat(rlm): upload a /usr/local/bin/git shim, gated by allow_git (#1225)

Fixes and maintenance

• Keep harness metrics merge inside experimental composable env (#1201)
• Propagate typed exceptions from SWE/Harbor validate_instance (#1204)
• fix: pass explicit 60s timeout to get_background_job in poll_job_completion (#1206)
• fix: bump opencode harness default release to v1.1.63-rl2 (#1184)
• validate(): extract resume-file parsing into a named helper (#1209)
• fix: SandboxTimeouts fields must be int (sidecar deserializes as u64) (#1210)
• fix: respect framework-injected OPENAI_API_KEY in RLM and opencode harnesses (#1213)
• fix: offload composable _upload_dir tar build to thread (#1224)

Full Changelog: v0.1.13.dev3...v0.1.13.dev4