Skip to content

[Claimed #1978] fix: add variable substitution to keys step during cache replay#1983

Merged
pirate merged 2 commits intomainfrom
external-contributor-pr-1978
Apr 9, 2026
Merged

[Claimed #1978] fix: add variable substitution to keys step during cache replay#1983
pirate merged 2 commits intomainfrom
external-contributor-pr-1978

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot commented Apr 8, 2026

Mirrored from external contributor PR #1978 after approval by @pirate.

Original author: @a7med3liamin
Original PR: #1978
Approved source head SHA: 2149aa265a04dc37154d5a84411f3ab4d1045897

@a7med3liamin, please continue any follow-up discussion on this mirrored PR. When the external PR gets new commits, this same internal PR will be marked stale until the latest external commit is approved and refreshed here.

Original description

Fixes #1776

Problem

The keys tool has no variable substitution in either the live execution or cache replay paths. When the agent uses %variableName% tokens with the keys tool, the literal token string gets typed instead of the resolved value.

Fix

This PR combines two fixes into one:

1. Live execution (original fix by @trillville from #1777)

  • Accept variables parameter in keysTool (matching typeTool)
  • Call substituteVariables() before page.type() in the method === "type" branch
  • Pass variables to keysTool in createAgentTools
  • Update schema description to advertise available variables to the LLM
  • Return original token in result to avoid exposing sensitive values to LLM

2. Cache replay (new fix)

  • Import substituteVariables in AgentCache.ts
  • Pass variables through to replayAgentKeysStep
  • Call substituteVariables(text, variables) before page.type() in the replay path

Without fix #2, cached keys steps with method="type" replay by typing literal %variableName% tokens even when variables are provided, since replayAgentKeysStep had no access to the variables map.

Credit

The live execution fix (part 1) is from @trillville's work in #1777/#1813. We merged it here with the cache replay fix per @pirate's request to consolidate into a single PR.


Summary by cubic

Add variable substitution to the keys tool for both live execution and cache replay so %variableName% tokens are resolved before typing. This fixes cases where literal tokens were typed and brings parity with the type tool.

  • Bug Fixes
    • Pass variables into keys and call substituteVariables() before page.type(); update the input schema to list available variables.
    • In cache replay, forward variables to replayAgentKeysStep and substitute before typing to avoid replaying literal tokens.
    • Record and return the original tokenized value (not the resolved value) to avoid leaking sensitive data.

Written for commit abb3905. Summary will update on new commits. Review in cubic

Two fixes for %variableName% token handling in the keys tool:

1. Live execution: pass variables to keysTool, call substituteVariables()
   before page.type(), and update the schema description to advertise
   available variables to the LLM. Records the original token in the
   cache entry and returns it to the LLM to avoid exposing sensitive
   values. (Original fix by @trillville in #1777)

2. Cache replay: pass variables to replayAgentKeysStep in AgentCache
   and call substituteVariables() before page.type(). Without this,
   cached keys steps replay by typing literal %variableName% tokens
   instead of resolved values.

Fixes #1776

Co-Authored-By: trillville <trillville@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:mirrored An internal mirrored PR currently exists for this external contributor PR. labels Apr 8, 2026
@github-actions
Copy link
Copy Markdown
Contributor Author

github-actions bot commented Apr 8, 2026

This mirrored PR has been merged into main. The original external PR #1978 is now completed.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 8, 2026

🦋 Changeset detected

Latest commit: abb3905

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@browserbasehq/stagehand Patch
@browserbasehq/stagehand-evals Patch
@browserbasehq/stagehand-server-v3 Patch
@browserbasehq/stagehand-server-v4 Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.
Architecture diagram
sequenceDiagram
    participant Agent as Agent / LLM
    participant Tool as keysTool
    participant Util as substituteVariables
    participant Cache as AgentCache
    participant Browser as Browser Page

    Note over Agent,Browser: Live Execution Flow
    
    Agent->>Tool: execute(method: "type", value: "%password%")
    
    rect rgb(240, 240, 240)
        Note right of Tool: NEW: Resolve variables before typing
        Tool->>Util: substituteVariables("%password%", variables)
        Util-->>Tool: "secret_value_123"
    end

    Tool->>Browser: page.type("secret_value_123")
    
    Note right of Tool: NEW: Record template value to cache (not secret)
    Tool->>Cache: recordAgentReplayStep(text: "%password%")
    
    Tool-->>Agent: { success: true, value: "%password%" }

    Note over Agent,Browser: Cache Replay Flow (Subsequent Run)

    Cache->>Cache: replayAgentKeysStep(step, variables)
    
    alt step.method === "type"
        rect rgb(240, 240, 240)
            Note right of Cache: NEW: Substitute variables during replay
            Cache->>Util: substituteVariables("%password%", variables)
            Util-->>Cache: "secret_value_123"
        end
        Cache->>Browser: page.type("secret_value_123")
    else method === "press"
        Cache->>Browser: page.press(keys)
    end
Loading

@pirate pirate merged commit 8543c11 into main Apr 9, 2026
204 checks passed
@github-actions github-actions bot added external-contributor:completed The mirrored PR has been merged and the external contributor flow is complete. and removed external-contributor:mirrored An internal mirrored PR currently exists for this external contributor PR. labels Apr 9, 2026
@pirate
Copy link
Copy Markdown
Member

pirate commented Apr 9, 2026

thanks again @a7med3liamin! great addition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:completed The mirrored PR has been merged and the external contributor flow is complete. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

keys tool does not perform variable substitution

3 participants