Skip to content

chore: rollout module cleanups - #2208

Merged
mikasenghaas merged 7 commits into
mainfrom
chore/rollout-cleanup
Jul 31, 2026
Merged

chore: rollout module cleanups#2208
mikasenghaas merged 7 commits into
mainfrom
chore/rollout-cleanup

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Jul 31, 2026

Copy link
Copy Markdown
Member

Summary

  • Inline _as_messages at its single call site (Interaction._turn in agent.py) and remove it from rollout.py — it normalized wire-dict user turns to typed messages and had no other consumer.
  • Move _serve_interception out of rollout.py into the interception package as public serve_interception, alongside requires_tunnel/make_interception and symmetric with mcp's serve_tools.
  • Rename RolloutRunRollout.
  • Require timeouts and limits on Rollout (previously | None = None): its only constructor, Agent._rollout_params, always passes both, and "no caps" is already spelled RolloutTimeouts() / RolloutLimits().
  • Remove mask_prompt (on Agent.interaction) and wire_data (on Rollout) entirely: trace.task.data is now always task.data. The user-sim env instead hands the assistant a task copy with prompt=None; the scenario stays the user side's knowledge and task scoring reads non-prompt fields (answer, a judge's question_field) — which every in-tree taskset already does. Docs and the create-environments skill updated to match.
  • Docstring pass over rollout.py, AgentConfig, and TimeoutConfig: per-field docs instead of prose paragraphs, fields grouped by role.
  • Per-field docstrings on RolloutTimeouts, matching TaskTimeout's style.

These are the rollout-adjacent cleanups from #2204's branch that missed its squash merge (it merged before the last two commits were pushed), rebased onto main together with the docstring pass.

Breaking

  • Agent.interaction(mask_prompt=True) is removed: pass a task whose data.prompt is None instead, and keep scoring on non-prompt fields. A user-sim taskset whose rewards/judges read the prompt must move that content to a scoring-side field.

  • verifiers.v1.rollout.RolloutRunverifiers.v1.rollout.Rollout (not re-exported from verifiers.v1; its only in-tree driver is Agent).

Verification

uv run ruff check, uv run pre-commit run --all-files, and uv run pytest tests/v1 (66 passed, 67 skipped — skips need PRIME_API_KEY) all pass.

🤖 Generated with Claude Code

Note

Remove mask_prompt from Agent.interaction and rename RolloutRun to Rollout

  • Removes the mask_prompt parameter from Agent.interaction and _EpisodeAgent.interaction; callers must now pass a Task with data.prompt=None to hide the scenario prompt from the assistant.
  • Renames RolloutRun to Rollout throughout rollout.py; its constructor now requires explicit RolloutTimeouts and RolloutLimits and drops the wire_data masking parameter.
  • Extracts a reusable serve_interception async context manager into verifiers/v1/interception/init.py, replacing an internal helper in rollout.
  • Updates UserSimEnv.run to construct assistant_task with prompt=None instead of using mask_prompt=True.
  • Risk: any caller passing mask_prompt=True to Agent.interaction will break; prompt hiding is now the caller's responsibility.

Macroscope summarized 74bd3c9.


Note

Medium Risk
Renames and moving interception setup are mechanical, but removing mask_prompt/wire_data changes how traces record task data and breaks callers that relied on hidden prompts or RolloutRun.

Overview
Rollout lifecycle cleanup: RolloutRunRollout, _serve_interception → public serve_interception in the interception package (used from rollout.py), and Rollout now requires timeouts and limits (callers always passed them).

Breaking API change: Agent.interaction(mask_prompt=...) and Rollout’s wire_data are removedtrace.task.data is always the task you pass in. User-sim now gives the assistant a task copy with prompt=None instead of framework masking; docs/skills describe scoring off non-prompt fields (answer, etc.). Interaction.turn inlines wire-dict normalization via parse_message (removed _as_messages from rollout).

Docs/config touch-ups: agent/user-sim guidance, shorter AgentConfig / RolloutTimeouts field docstrings, e2e comments updated for the nulled-prompt contract.

Reviewed by Cursor Bugbot for commit 6d33ce4. Bugbot is set up for automated code reviews on this repo. Configure here.

mikasenghaas and others added 7 commits July 31, 2026 12:07
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…> Rollout

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k's prompt

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikasenghaas
mikasenghaas marked this pull request as ready for review July 31, 2026 21:17

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6d33ce4. Configure here.

Comment thread verifiers/v1/envs/user_sim/env.py
@mikasenghaas
mikasenghaas requested a review from hallerite July 31, 2026 21:40
@mikasenghaas
mikasenghaas merged commit f6e420b into main Jul 31, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants