observe/act returning unresolvable xpath locators

[dnwarnock]

I'm really impressed with the Stagehand and Browserbase setup! We're currently exploring the possibility of transitioning from our proprietary natural language action translation layer to Stagehand. To evaluate its suitability, I'm trying to verify Stagehand's functionality against some simulated webpages.

Unfortunately, I've noticed that some of the XPath locators returned by observer/act don't seem specific enough for Playwright to efficiently interact with the intended HTML elements. I was wondering if there are any configuration options I can adjust to obtain more specific locators?

I've also attempted to modify the timeoutMs option when calling act, but it appears to be consistently stuck at 30 seconds.

Any advice or insights on how to improve XPath specificity and address the timeoutMs issue would be greatly appreciated.

Thanks in advance!

`

var [topAction] = await p.observe({instruction: "Click the 'Statements & documents' link"});
undefined
topAction
{
description: "The 'Statements & documents' link in the main section of the page.",
method: 'click',
arguments: [],
selector: 'xpath=/div/a[1]'
}
var res = await p.act(topAction)
undefined
res
{
success: false,
message: 'Failed to perform act: locator.evaluate: Timeout 30000ms exceeded.\n' +
'Call log:\n' +
"\x1B[2m - waiting for locator('xpath=/div/a[1]').first()\x1B[22m\n",
action: "The 'Statements & documents' link in the main section of the page."
}
`

[dnwarnock]

To me it seems like the xpath return from observe isn't appending enough parent information in the path. If I grab the full xpath from the html manually I get /html/body/div[3]/div/div[1]/div[2]/main/div[3]/div/div/div/div/div[2]/div/div/div[1]/div/mds-navigation-bar/mds-navigation-bar-item[2]//div/a. I'm not sure how playwright is meant to resolve /div/a[1] to the link in question.

[dnwarnock]

I got some more details on how our simulator webpages were generated. We used SingleFile. I wonder if the large output from this tool could be tripping up Stagehand. I've notice better performance on live website vs our seemingly simpler simulated websites.

Update: The issue persists on both the live and simulated version of our websites

[kamath]

Hey! Thanks for trying Stagehand and opening this issue.

Do you know if there are iframes on the page? Have you tried using computer use models with stagehand.agent? Also, what LLM are you using?

[dnwarnock]

Thanks for the response! The issue was with iframes. Using a computer use model does get up pasted the issue.