Skip to content

Mobile benchmark 2.2 — install, start, automatic HF reports

Choose a tag to compare

@vinyteles vinyteles released this 05 Nov 17:07
· 25 commits to main since this release

Install the assigned APK and tap Start / resume selected. Version 2.2 creates a public HF results session automatically through the upload server. Participants need no Hugging Face account, token, repository setup, Python or Android Studio.

Batch Configurations in order Maximum answers
1 · gpu-known Energy mixed48 LiteRT GPU; LFM3 Q4_K_M Vulkan GPU 240
2 · gpu-new LFM3 mixed4 LiteRT GPU; original Gemma mobile LiteRT GPU 240
3 · gemma Original Gemma Q8 Vulkan GPU if RAM permits; Gemma mobile LiteRT CPU 240
4 · lfm-cpu LFM3 Q4_K_M CPU 120
5 · energy-cpu Energy int8 CPU; Energy mixed48 CPU 240

The five APKs share their package and signing key. Installing another batch updates the app and preserves downloaded models and answers. All configurations retain the same 120 prompts and 4096/512 token settings from 2.1. Requires Android 12+ and ARM64; models download separately.

Open Report upload status to copy the public results link. Keep the app foreground and wait for Hugging Face synced before ending the reservation when possible. Received by server means the checkpoint is durably queued; publication to HF follows asynchronously. Both phone and server retry network failures. Local ZIP export/import remains available.

Fresh installations, runs and batch changes create separate upload sessions. Resuming the same run/batch reuses its session. If an existing installation/run/batch is reused for another reservation, choose New upload session while stopped.

Verification passed: 28-test Android suite, 55 server tests, actual Android → public HTTPS relay → HF uploads, independent checkpoint/hash/download checks, and live server restart recovery. Test reports are labelled synthetic and do not establish model quality or physical GPU speed.

Initial ingress is a temporary Cloudflare quick tunnel. The app follows the current URL from the public HF index while verifying the server identity pinned in GitHub. It retains checkpoints during outages. Server credential renewal remains an operator task; no HF credential is embedded in the APKs.

Run instructions · Public session index · Validation evidence

Download your assigned batch APK below. SHA256SUMS contains all five checksums.