Repository navigation
Independent test: AI coding agent vs Carbon, 13 of 15 runs shipped, two missed accessibility wiring #23147
pjhawksr
started this conversation in
Accessibility
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I’ve been developing a method to test design-system alignment and whether AI agents can produce working software with it. I ran it on Carbon Core v11 and would value a maintainer’s review before sharing it more widely.
The agent completed five tasks three times each using frozen public evidence and @carbon/react 1.114.0. It did not use Carbon’s MCP server, so the results make no claim about what the MCP might prevent.
13 of 15 runs passed. The two failures were repeats of the same form-flow issue: a visible TextInput error was not programmatically connected to the invalid field. The interfaces looked correct and passed axe; a Playwright check of the rendered relationship caught the problem. The passing run used invalidText, allowing Carbon to render and connect the message itself.
Two questions:
Evidence cutoff: August 22, 2026. I’m sharing this independently because the method is only useful if the team that owns Carbon can check it.
All reactions