On Device Ai

Evaluate the experimental on-device AI branch

On-device inference is a separate development direction from the recorded owned-host candidate. The reported feature branch uses LiteRT-LM in an isolated module, with model downloads requiring explicit consent and pinned size and checksum checks. Its source gate and larger APK are not the same artifact as the fifteen-megabyte 0.2.8-review candidate. This chapter describes how to evaluate that branch without marketing an unobserved model response as a shipped capability.

Keep branch identity visible

Record the branch revision, dependencies, APK hash and native ABI inventory. The reported branch gate has 175 Android tests, 174 passed and one skipped; the receipt checker's seven are not part of that number. Those are maintainer-reported branch facts until their preserved evidence is reviewed. Do not replace the recorded candidate's 164-test ledger with the larger number while continuing to link its old APK.

The reported implementation uses LiteRT-LM 0.8.0 to avoid changing pinned Kotlin and coroutine versions. Compatibility with a newer gallery library is therefore not established by the inspiration alone. A NOTICE credits Google AI Edge Gallery and clarifies non-affiliation. Credit describes provenance; it does not certify the app's runtime or grant permission for every downloaded model.

Review download consent

The user should see the source, license and expected download size before any model bytes are fetched. A finished download should require exact size and SHA-256 agreement. If the server requires a Hugging Face sign-in or license acceptance, that access requirement must be implemented or the model shown unavailable. Do not claim a download works because a model URL exists in source.

Checksums establish file identity, not safe behavior, inference quality or a license right. A pinned artifact can still be incompatible with the device or use more RAM than available. Download interruption needs cleanup and a clear retry policy. Model storage is a different concern from the small encrypted vault; do not pretend a multi-gigabyte model fits under the vault's per-object allowance.

Qualify actual inference

Install the branch build on an identified runtime, load a permitted model and ask a non-sensitive prompt. Record load time, completion behavior, memory limits, cancellation and error states. The reported implementation runs on CPU rather than a qualified GPU path. Avoid promising accelerator support or token throughput before measuring it. An emulator run can establish a bounded functional result, while physical-device thermals and sustained performance need device measurements.

The APK grows substantially because native inference libraries are bundled. Reducing ABIs can shrink an artifact but may remove emulator compatibility or devices from support. Make that decision with an explicit target matrix and build tests, not merely to improve a marketing file-size badge. A model does not become on-device private if the app quietly sends a fallback prompt to a server.

Decide whether to merge

Before promotion, resolve gated model access, actual response evidence, screenshot baseline changes, supported ABIs and license notices. Keep the existing owned-host path intact. Update the site only after the branch's capability and artifact identity are clear. Until then, Experimental branch — inference NOT_RUN is a useful, accurate label that communicates progress without converting future work into current product behavior.

---

Wiki home · Qualification updates · Sources and credits