POST /v1/heal takes a page source and whatever you know about an element, and
returns one locator that is guaranteed to resolve to exactly one node in the
page source you sent. Serves Selenium, Playwright and Appium (Android, iOS,
React Native), including shadow DOM.
uvicorn app.main:app --reloadcurl -s localhost:8000/v1/heal -H 'content-type: application/json' -d '{
"page_source": "<html>…</html>",
"existing_locators": [{"type": "css", "value": "#add-to-cart-backpack"}],
"description": "add the Sauce Labs Backpack to the cart"
}'{
"locator_type": "css",
"locator": "[data-test=\"add-to-cart-sauce-labs-backpack\"]",
"confidence": 0.908,
"match_count": 1,
"strategy": "test_id",
"rationale": "Dedicated test hook data-test='…'; survives markup and style changes.",
"platform": "web",
"framework": "selenium",
"llm_used": false,
"elapsed_ms": 5
}parse → shortlist candidate nodes → disambiguate → generate ladder → validate → answer
↑
LLM sits only here
The LLM never sees your page source and never writes a locator. Deterministic code prunes thousands of nodes down to ~8 candidates and asks the model one closed question: which index is the target? A 5 MB page produces a ~3 KB prompt, and the model cannot emit an invalid selector because it never writes one.
The model is also skipped whenever the deterministic ranking is already clear,
which is most of the time. With LLM_ENABLED=false the service still works — it
just makes more mistakes on genuinely ambiguous pages.
Nothing is returned unvalidated. Every candidate is evaluated against the
page source you sent and must (a) compile, (b) match exactly one node, and
(c) match the node. Match count alone is not enough: a unique selector can
still point at the wrong element. If nothing passes, you get a 422 explaining
why rather than a guess.
XPath is a last resort everywhere, and never absolute.
Web — data-testid → Playwright get_by_role → stable id →
aria-label/name/placeholder → exact text → relative to a stable ancestor
→ semantic classes → short anchored XPath.
Android — resource-id → content-desc → UiSelector().text() →
id+text combinations → XPath anchored on the nearest identified ancestor.
iOS — accessibility id → -ios predicate string → -ios class chain →
anchored XPath.
React Native — testID → accessibilityLabel → visible text → text-predicated
XPath for a container whose text RN split across children → stop.
Rejected outright, because they look stable in one snapshot and are not:
| Anti-pattern | Example |
|---|---|
| Generated ids | :r3:, ember1043, a bare UUID |
| Build-hashed classes | css-1x2y3z, Button_root__aB3dE |
| Absolute paths | /html/body/div[2]/button |
| Index chains | //div[3]/div[1]/span[2] |
| Data-dependent text | $29.99, 2026-07-25 14:03 |
Capture matters more than anything else here. driver.page_source and
page.content() return the light DOM only — shadow roots are simply absent, so
the element is not in your payload at all. Serialize with declarative shadow DOM:
document.documentElement.getHTML({ serializableShadowRoots: true,
shadowRoots: [...allOpenRoots] })Closed roots cannot be traversed by any tool, so they return 422 rather than a
guess.
| Framework | You get | Why |
|---|---|---|
| Playwright | css |
Its engine pierces open roots. Validated for uniqueness across every scope, not just the target's own. |
| Selenium / WebdriverIO | shadow_piercing_css |
A >>> chain, one segment per boundary, each unique inside its own root. |
XPath is never emitted behind a boundary — it cannot cross one, and a caller who ran it would silently get zero matches.
Walking a chain in Selenium:
def find_piercing(driver, chain: str):
head, *rest = [s.strip() for s in chain.split(">>>")]
element = driver.find_element("css selector", head)
for segment in rest:
element = element.shadow_root.find_element("css selector", segment)
return elementRN renders real native views, so the page source is ordinary Appium XML — but the
hierarchy is generic ViewGroup / XCUIElementTypeOther, deeply nested, and
reshapes between renders and RN versions.
| RN prop | Android | iOS |
|---|---|---|
testID |
resource-id |
name |
accessibilityLabel |
content-desc |
label |
There is deliberately no structural fallback. When a component has no
testID, no accessibilityLabel and no text, the service returns 422 naming
the element and telling you to add a testID. A ViewGroup[3]/ViewGroup[1] path
would break on the next render — returning it would be worse than admitting
failure.
Text split across sibling ReactTextView nodes is reassembled before matching. A
pressable that wraps <Text> has no text of its own, so the last rung matches it
by the fragments its children render:
//com.facebook.react.views.view.ReactViewGroup[.//*[@text='Add to'] and .//*[@text='cart']]
That is an XPath, but a text-predicated one — no index, no path through the hierarchy. It survives a re-render for the same reason the plain text rung does, and fails only when the rendered words change. It is still not a structural fallback.
| Field | Notes |
|---|---|
page_source |
Required. |
existing_locators |
The locator that broke. Optional, but the strongest evidence there is. |
description |
What the element is, in prose. |
remarks |
Context about the change ("moved into the new header"). Scored together with description. |
platform / framework / app_kind |
Auto-detected; pass them to override. |
options.allow_xpath |
false restricts output to CSS. |
options.test_id_attrs |
Override the test-hook attribute list. |
options.use_cache |
false forces a fresh heal. |
At least one of existing_locators / description is required — there is
otherwise no way to know which element you mean.
Both are better than either. If a positional locator still resolves after a
refactor, it may now point at a different element (a wrapper div was inserted,
rows were reordered). When a description is present the service cross-checks, and
the credit given to a surviving locator is scaled by that locator's own
robustness — a surviving [data-testid] is strong evidence, a surviving
//div[1]//button is nearly none.
A surviving locator whose element answers none of the description earns no
credit at all. An id that outlived the element it was written for is the
ordinary way a locator ends up gripping the wrong thing, and an averaged score
lets it: a zero description match dilutes rather than argues. So a description
the element contradicts counts against it, and no intact verdict is issued when
the evidence disagrees with itself — not between two supplied locators, and not
between a locator and the description.
Known gap. When an id is not merely renamed but swapped onto a sibling, the
stolen id corroborates itself: scoring reads the description against every
attribute, so the wrong element matches the words through the very id that moved
onto it. The stale evidence and the confirming evidence are the same string.
tests/mutations.py carries this as a failing case that counts against the heal
rate rather than being deleted to protect the number.
Copy .env.example to .env.
LLM_MODEL=gemini/gemini-2.5-flash
LLM_API_KEY=…
LLM_ENABLED=true
LLM_TIMEOUT_S=30The prefix on LLM_MODEL picks the provider and the rest is the catalog id
verbatim, so swapping providers is one line and no code:
| Provider | LLM_MODEL |
|---|---|
| Google AI Studio | gemini/gemini-2.5-flash |
| Anthropic | anthropic/claude-sonnet-5 |
| NVIDIA NIM | nvidia_nim/z-ai/glm-5.2 |
LLM_API_BASE is empty by default on purpose — LiteLLM already knows each
provider's endpoint, and a base carrying its own version path (Gemini's
/v1beta, an OpenAI-compatible /v1) can collide with the one LiteLLM appends.
Set it only to override: a self-hosted container, a proxy, or a gateway.
Params a provider does not support are dropped rather than sent — seed is the
usual casualty. Losing determinism beats losing the call.
The deterministic path answers in 0–2 ms and scores 12/13 on the mutation
harness. The model is consulted only when two candidates are genuinely close, and
on timeout the pipeline keeps its deterministic answer and sets
llm_used: false — so a slow or dead endpoint costs latency on the ambiguous
minority, never correctness.
Measured against gemini-2.5-flash, five runs of one index decision:
| One structured call | 5.3 s min, 5.7 s median, 7.8 s max |
| Full heal, model consulted | 3.3–6.2 s end to end |
| Structured-output tier | json_schema accepted first try, one call |
The 30 s / 60 s defaults leave roughly 4× headroom on purpose: a limit set near the median turns an ordinary slow response into a pointless tier fallback.
What was measured on hosted NVIDIA NIM is why the timeout machinery exists at all — the same code path, a provider two orders of magnitude slower:
| Model | One index decision |
|---|---|
deepseek-ai/deepseek-v4-pro |
132 s (json_schema), 191 s (json_object) |
z-ai/glm-5.2 |
exceeded 300 s; no answer |
Do not rely on the SDK's timeout. A litellm.completion(timeout=120) against
that endpoint was measured taking 363 seconds to raise. LLM_TOTAL_BUDGET_S
is therefore enforced here, on a daemon worker thread abandoned when the deadline
passes — it is the only limit that actually binds. The same request went from
363.2 s to 150.0 s under it, still returning the correct deterministic locator.
Two regression tests cover it. That guard is provider-independent and stays.
If you point LLM_MODEL at a reasoning model, raise both limits: minutes, not
seconds. Consider running the tie-break out of band entirely — LLM_ENABLED=false
on the request path, the model tier as an async or batch job.
If you write your own streaming client against a reasoning model, note that
these emit reasoning as delta.reasoning_content, not delta.content. A loop
printing only delta.content looks frozen for the whole reasoning pass. This
service does not stream; it reads message.content and ignores
reasoning_content, stripping inline <think> blocks and code fences.
Support varies per model behind an OpenAI-compatible endpoint, so the client
walks down three tiers — json_schema → json_object → prompted extraction —
remembers which one worked, and retries once on a parse failure. If none
succeeds it discards the model's answer and keeps the deterministic one.
Both DeepSeek v4 models accept json_schema and return conforming JSON. Gemini
and Anthropic models accept it too; the ladder exists for whatever does not.
Privacy: page_source can contain customer data, and a hosted model means
sending it to a third party. Set LLM_ENABLED=false and nothing leaves your
network at all — the deterministic path is the bulk of the accuracy. Or point
LLM_API_BASE at an in-network container.
Keyed on the request (locator + description + platform), not the page. A hit is re-validated against the page in hand and dropped if it no longer resolves, so a stale entry can never be served.
python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"See TESTING.md for the full strategy, commands, and release gate.
CI runs the offline suite and mutation harness on every push/PR to main.