Skip to content

fix: improve android fill verification diagnostics#495

Merged
thymikee merged 1 commit into
mainfrom
fix-android-fill-verification
May 11, 2026
Merged

fix: improve android fill verification diagnostics#495
thymikee merged 1 commit into
mainfrom
fix-android-fill-verification

Conversation

@thymikee
Copy link
Copy Markdown
Member

@thymikee thymikee commented May 11, 2026

Summary

Improve Android fill verification diagnostics for IME-captured input and masked password fields.

Details:

  • Detect when typed text lands in a keyboard/IME-owned input instead of the target app field and report target/actual input metadata.
  • Fail fast once verification shows IME capture instead of trying alternate text injection strategies against the wrong focused field.
  • Treat matching-length masked/password values as successful verification, with a defensive non-empty expected/actual guard.
  • Add a shared fill diagnostics helper for targetInput/actualInput, masked redaction, inferred password-node sensitivity, and expected/actual length metadata so other platforms can adopt the same contract.
  • Redact masked/password expected and node text from failure diagnostics while reporting lengths and textRedacted metadata.
  • Parse Android hierarchy package and password attributes for fill diagnostics, and tighten Android IME ownership detection to input-method package/resource prefixes.
  • Extract Android fill verification into a focused module so input-actions remains the action orchestrator.
  • Add versioned help and SkillGym guidance so agents stop retrying fill/type when diagnostics show Gboard handwriting/stylus capture.
  • Add an Agent Device Tester fixture on the Checkout form with stable android-ime-capture-fixture and field-ime-capture-target IDs, and point the SkillGym case at that fixture.

Touched files: 11. Scope stayed within Android input verification, shared fill diagnostics, CLI help guidance, Agent Device Tester fixture coverage, and focused tests.

Validation

  • pnpm format
  • pnpm exec oxfmt --write examples/test-app/src/lab-state.tsx examples/test-app/src/screens/FormScreen.tsx
  • pnpm exec vitest run src/platforms/android/__tests__/index.test.ts
  • pnpm exec vitest run src/platforms/__tests__/fill-diagnostics.test.ts src/platforms/android/__tests__/input-actions-fill.test.ts src/platforms/android/__tests__/index.test.ts
  • pnpm exec vitest run src/utils/__tests__/args.test.ts
  • pnpm check:fallow --base origin/main
  • pnpm build
  • pnpm check:quick
  • pnpm test-app:install
  • pnpm --dir examples/test-app typecheck
  • pnpm exec skillgym run ./test/skillgym/suites/agent-device-smoke-suite.ts --config ./test/skillgym/skillgym.config.ts --case android-fill-ime-capture-stop-retry-loop --runner codex-mini

Known gap: full all-runner SkillGym invocation could not run here because the configured claude runner binary is missing from PATH (spawn claude ENOENT).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 11, 2026

PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstackincubator.github.io/agent-device/pr-preview/pr-495/

Built to branch gh-pages at 2026-05-11 17:42 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@thymikee thymikee force-pushed the fix-android-fill-verification branch 8 times, most recently from 5560a25 to 11ad734 Compare May 11, 2026 17:41
@thymikee thymikee force-pushed the fix-android-fill-verification branch from 11ad734 to 9178676 Compare May 11, 2026 18:01
@thymikee thymikee merged commit 5df37ec into main May 11, 2026
17 of 18 checks passed
@thymikee thymikee deleted the fix-android-fill-verification branch May 11, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant