Skip to content

evals: make agent mounting part of tool surfaces - #2618

Merged
miguelg719 merged 4 commits into
stg-2671-llm-exposure-contractfrom
refactor/tool-surface-agent-mount
Aug 6, 2026
Merged

evals: make agent mounting part of tool surfaces#2618
miguelg719 merged 4 commits into
stg-2671-llm-exposure-contractfrom
refactor/tool-surface-agent-mount

Conversation

@miguelg719

@miguelg719 miguelg719 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • keep the existing CoreTool abstraction as the canonical tool-surface definition
  • attach optional agent delivery and final-state capture to ToolStartResult, preserving one runtime lifecycle
  • replace LLMExposure with a lifecycle-free AgentMount value using the independent via: handles | mcp | cli delivery axis
  • keep native CoreTool.surface independent from agent delivery, so a code surface may be wrapped by MCP or CLI
  • leave task, start URL, and default console binding with the harness
  • define reserved harness bindings, CLI environment merge semantics, and capture-before-cleanup ordering

This is stacked on #2590. The revision deliberately defers broad naming/registry churn so the contract change remains a focused two-file diff.

Verification

  • pnpm --filter @browserbasehq/stagehand-evals typecheck
  • pnpm --filter @browserbasehq/stagehand-evals test:unit (52 files, 413 tests)
  • focused formatting check passed
  • reviewed independently with Claude Opus 5 against the inline repository diff

@changeset-bot

changeset-bot Bot commented Aug 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: d5b3421

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 21 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/evals/framework/context.ts Outdated
Comment thread packages/evals/core/contracts/tool.ts
Comment thread packages/evals/tests/core/tool-contract.test.ts
@miguelg719
miguelg719 force-pushed the refactor/tool-surface-agent-mount branch from bacadab to 80dc91f Compare August 6, 2026 05:19
@miguelg719
miguelg719 force-pushed the refactor/tool-surface-agent-mount branch from 80dc91f to 9e7da41 Compare August 6, 2026 05:42

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/evals/core/contracts/tool.ts Outdated
@miguelg719
miguelg719 merged commit d3ba958 into stg-2671-llm-exposure-contract Aug 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant