Skip to content

v0.4 rollout, sandbox, and reward foundations#263

Closed
xdotli wants to merge 22 commits into
mainfrom
cursor/rollout-sandbox-architecture-1d09
Closed

v0.4 rollout, sandbox, and reward foundations#263
xdotli wants to merge 22 commits into
mainfrom
cursor/rollout-sandbox-architecture-1d09

Conversation

@xdotli

@xdotli xdotli commented May 15, 2026

Copy link
Copy Markdown
Member

Summary

This PR lays the v0.4 architecture foundation around rollout-first execution and native BenchFlow boundaries:

  • adds canonical rollout config/types (Role, Turn, Scene, RolloutConfig, RolloutResult)
  • routes public benchflow.run, jobs, CLI single-run paths, and self-gen through rollout-native entry points
  • moves lifecycle helper logic out of SDK toward benchflow.rollouts
  • adds native sandbox protocols/specs and a native DockerSandbox provider (no core HarborSandbox wrapper)
  • converts benchflow.tasks into a package with native task config/path models
  • adds native rewards primitives (RewardEvent, Rubric, RewardFunc, TestRewardFunc)
  • adds agent capability declarations
  • removes top-level Harbor reexports/fallback in favor of native Task, TaskConfig, ExecResult, and SandboxSpec

Compatibility modules (sdk.py, runtime.py, trial.py, trial_yaml.py) still exist internally for incremental migration, but they are no longer top-level public API.

Validation

  • uv run python -m pytest tests/
    • 835 passed, 1 skipped, 1 deselected
  • uv run ruff check .
  • uv run ty check src/

Notes

Harbor runtime removal is intentionally not completed in this PR. The direction is native providers first (DockerSandbox, then Daytona/Modal), with Harbor kept only as a temporary internal bridge until rollout execution no longer depends on it.

Open in Web Open in Cursor 

cursoragent and others added 22 commits May 15, 2026 21:15
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
Co-authored-by: Xiangyi Li <xiangyi@benchflow.ai>
@xdotli xdotli closed this May 17, 2026
@xdotli
xdotli deleted the cursor/rollout-sandbox-architecture-1d09 branch May 17, 2026 05:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants