Skip to content

v0.2.0 — installable APK with live chat and phone use

Choose a tag to compare

@susyimes susyimes released this 26 Jul 17:56

This release ships an APK. Install it, paste your own API credential, and chat with a live model through the harness — or press Send with no credential at all and watch the offline scripted loop run locally.

Install

Download android-agent-harness-sample-0.2.0-debug.apk below. Android 8.0 (API 26) or newer.

The APK is debug-signed and sideload-only, so Android will ask you to allow an unknown source. A debug build is also debuggable, which means anyone with adb access to your unlocked device can read the credential it stores — use a low-limit, revocable key and press Clear credential when you are done.

What you can do with it

  • Chat with zero setup. No credential, no network: a deterministic scripted provider runs the full context → provider → tool → provider → transcript loop on device, and every tool call is visible in the transcript.
  • Chat with your own model. Paste the base URL, model, and credential of any OpenAI-compatible chat-completions endpoint (OpenAI, DeepSeek, and anything speaking the same protocol). The credential lives only in the app's private preferences and is sent only to the endpoint you configured.
  • Let the agent operate your phone. Phone mode gives the model an observe → one action → observe loop over a real Android accessibility tree: tap, set text, Back, swipe, scroll until text is visible, launch an app by name, wait for the screen to settle. Home is refused, because leaving the current app strands the task. Finishing requires evidence text actually visible on the screen being claimed.

How the dangerous parts are held down

  • High-risk actions stop for a human. Pay, purchase, transfer, delete, uninstall, order confirmation — and their Chinese equivalents — pause before executing. The approval panel is drawn by the accessibility service itself, so it stays visible over whatever app the agent is driving; an approval hosted by this app's own window would be invisible exactly when it matters.
  • The model cannot approve itself. A model-supplied confirmed=true is ignored entirely. Only a real tap on Allow approves; Deny, the visible countdown expiring, and a missing approval surface all count as refusal — and a refusal is reported back in wording that tells the model not to retry, so a denial cannot turn into an approval-fatigue loop.
  • No pixels leave the device. There is no screenshot, screen recording, or image input anywhere in the loop. The agent sees the accessibility node tree rendered as bounded text (60 nodes, 80 characters each), password fields redacted.

Honest limits: risk matching is keyword-based and can miss a control labelled in an unlisted language or not labelled at all, and with no vision fallback, canvas-drawn and WebView-heavy screens are largely invisible.

Under the hood

device-loop gained typed failures with candidate lists, a stale-target guard, and approval decisions that distinguish "the user refused" from "nobody answered". device-loop-android assigns content-derived node ids that survive scrolling (positional ids used to re-point the model at the wrong control), and walks the tree iteratively with depth and node caps — a self-referencing virtual tree previously raised StackOverflowError, which walks straight through a RuntimeException catch. provider-openai refuses redirects so an Authorization header is never replayed to a Location host.

217 tests. Verified on an Android 34 emulator: install, offline turn, accessibility service connect, phone mode toggle, keyboard-up send, no crashes.