[chore]: strengthen regex validation on act, observe for elementId

[miguelg719]

why

Stricter validation on LLM responses for chosen elementId to interact with

what changed

V3 includes only elementIds in the shape number-number to account for iframe ordinals. This PR ensures the LLMs have stricter validation on the elementId generation by changing the observe and act schemas to to enforce

z.string().regex(/^\d+-\d+$/)

test plan

• act evals
• observe evals
• combination evals
• regression evals

---

Summary by cubic

Enforce 'number-number' format for elementId in observe and act via regex (^\d+-\d+$) to prevent invalid IDs. Aligns with Linear STG-675 guardrail requirements for elementId formatting.

^{Written for commit 3b70cc1. Summary will update on new commits. Review in cubic}

[changeset-bot]

🦋 Changeset detected

Latest commit: 3b70cc1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

No issues found across 2 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

Architecture diagram

sequenceDiagram
    participant Caller
    participant Inference as inference.ts
    participant Zod as Zod Schema

    Note over Caller, Zod: Runtime Validation Flow for act() / observe()

    Caller->>Inference: Invoke operation with params
    
    Inference->>Zod: Define Schema
    Note right of Zod: NEW: Added .regex(/^\d+-\d+$/)
    
    Inference->>Zod: Validate input (elementId)
    
    alt NEW: elementId format mismatch
        Note right of Zod: Input like "[1-2]" or "123"<br/>fails regex check
        Zod-->>Inference: Throw Validation Error
        Inference-->>Caller: Error: Invalid elementId format
    else Valid elementId
        Note right of Zod: Input like "1-2" matches
        Zod-->>Inference: Validation Success
        Inference->>Inference: Execute Action logic
        Inference-->>Caller: Result
    end

Loading

[greptile-apps]

Greptile Overview

Greptile Summary

Added regex validation (/^\d+-\d+$/) to enforce the number-number format for elementId in both observe and act inference functions. This aligns with the existing EncodedId TypeScript type definition and prevents LLM responses from including invalid formats like square brackets or other malformed strings.

• Enforces validation at the Zod schema level in packages/core/lib/inference.ts:248 and packages/core/lib/inference.ts:386
• Matches the EncodedId type definition: ${number}-${number}
• Prevents runtime errors from malformed element IDs
• Consistent with existing test fixtures using format like 1-0

Confidence Score: 5/5

• This PR is safe to merge with no concerns
• The regex validation correctly enforces the number-number format that matches the EncodedId TypeScript type definition. The change is minimal, well-scoped, and prevents invalid elementId formats without affecting valid usage patterns
• No files require special attention

Important Files Changed

Filename	Overview
.changeset/clean-birds-fetch.md	Added changeset file for patch release documenting elementId validation enforcement
packages/core/lib/inference.ts	Added regex validation to enforce number-number format for elementId in observe and act schemas

Sequence Diagram

sequenceDiagram
    participant Client
    participant ObserveHandler
    participant Zod Schema
    participant LLM
    participant ActHandler

    Note over Client,ActHandler: Observe Flow
    Client->>ObserveHandler: observe(instruction, domElements)
    ObserveHandler->>Zod Schema: Define observeSchema with elementId.regex(/^\d+-\d+$/)
    ObserveHandler->>LLM: createChatCompletion with schema
    LLM-->>ObserveHandler: Return elements with elementId
    Zod Schema->>Zod Schema: Validate elementId matches /^\d+-\d+$/
    alt Valid format (e.g., "1-0")
        Zod Schema-->>ObserveHandler: Validation passes
        ObserveHandler-->>Client: Return parsed elements
    else Invalid format (e.g., "[1-0]", "1", "a-b")
        Zod Schema-->>ObserveHandler: Validation fails
        ObserveHandler-->>Client: Throw validation error
    end

    Note over Client,ActHandler: Act Flow
    Client->>ActHandler: act(instruction, domElements)
    ActHandler->>Zod Schema: Define actSchema with elementId.regex(/^\d+-\d+$/)
    ActHandler->>LLM: createChatCompletion with schema
    LLM-->>ActHandler: Return element with elementId
    Zod Schema->>Zod Schema: Validate elementId matches /^\d+-\d+$/
    alt Valid format (e.g., "1-0")
        Zod Schema-->>ActHandler: Validation passes
        ActHandler-->>Client: Return parsed element
    else Invalid format (e.g., "[1-0]", "1", "a-b")
        Zod Schema-->>ActHandler: Validation fails
        ActHandler-->>Client: Throw validation error
    end

Loading

[greptile-apps]

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}