-
Notifications
You must be signed in to change notification settings - Fork 82
Description
Is your feature request related to a problem? Please describe.
The accessibility tree generated by Stagehand does not currently include key details about the state of input elements on the page. As a result, LLMs cannot effectively extract critical page state when it's contained in these HTML input attributes.
(Note: all HTML snippets from this issue are available for review in context here: https://jsfiddle.net/21zp8rwb/1/)
checkedanddisabled
<section>
<h2>Checkboxes</h2>
<label><input type="checkbox" checked> Option A</label>
<label><input type="checkbox"> Option B</label>
<label><input type="checkbox" disabled> Option C</label>
</section>The Stagehand AX tree omits both checked and disabled states:
[69] section
[70] heading: Checkboxes
[12] checkbox: Option A
[13] checkbox: Option B
[14] checkbox: Option C
Both properties are supported AXProperties in the Chrome DevTools Protocol, so Stagehand should be able to expose them. The same issue affects radio inputs, which also use the checked attribute:
<section>
<h2>Radio Buttons</h2>
<label><input type="radio" name="color" value="red" checked> Red</label>
<label><input type="radio" name="color" value="green"> Green</label>
<label><input type="radio" name="color" value="blue"> Blue</label>
</section>Current output:
[74] section
[75] heading: Radio Buttons
[15] radio: Red
[16] radio: Green
[17] radio: Blue
- Input values w/o a text display
Inputs with internal values (e.g., range, color) are also missing value information in the AX tree.
<section>
<h2>Numeric Inputs</h2>
<label>Number: <input type="number" value="5" min="0" max="10"></label>
<label>Range: <input type="range" min="0" max="100" value="40"></label>
</section>Stagehand output:
[58] section
[59] heading: Numeric Inputs
[60] LabelText
[208] StaticText: Number:
[11] spinbutton: Number:
[209] StaticText: 5
[65] LabelText
[210] StaticText: Range:
[22] slider: Range:
The range input’s value (40) is not included in the AX tree, but is available in both the DOM itself (obviously) and as an AX Property.
Describe the solution you'd like
Stagehand should include relevant input AXProperties (e.g., checked, disabled, value) in its accessibility tree output. This would provide LLMs and other downstream consumers with a more complete and accurate representation of page state.
Two possible representations:
Option A – Nested property nodes:
[15] radio: Red
[...] checked: true
Option B – Inline property annotation:
[15] radio: Red (checked: true)
Either approach would be an improvement; Option A may be more structured and easier to parse programmatically.
Describe alternatives you've considered
No practical workarounds exist without reimplementing key portions of Stagehand’s accessibility tree logic.
Are you willing to contribute to implementing this feature or fix?
- Yes, I can submit a PR
- Yes, but I need guidance
- No, I cannot contribute at this time