Skip to content

llama.android : Rewrite Android binding (w/o cpu_features dep) #17413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 274 commits into
base: master
Choose a base branch
from
Open

llama.android : Rewrite Android binding (w/o cpu_features dep) #17413

wants to merge 274 commits into from
+2,586 −1,267

Conversation

@naco-siren
Copy link

@naco-siren naco-siren commented Nov 20, 2025

  1. Setup dynamic native library loading in the :lib native submodule to:
    a. enable Aarch64 acceleration up to SME2
    b. enable x86_64 acceleratiion up to AMX
    c. support both Android and ChromeOS
  2. Rewrite the C++ layer with:
    a. automatic message role formatting
    b. system prompt injection
    c. context overflow handling
    d. batch decoding
  3. Rewrite the JNI bridge to expose:
    a. engine states
    b. new API (such as system prompt)
  4. Add utilities and helpers to:
    a. parse GGUF metadata
  5. Miscellaneous performance optimizations
  6. Rewrite the basic sample app

Architectural diagram
512337972-87574ed8-8810-4754-b5bd-fca606e5497e

Performance comparison
Against Google AI Edge Gallery on the same Pixel 9:

512345275-355b5941-65c2-4088-8622-44979954caef.mp4

Production app
The full-featured production app "Arm AI Chat" is published on Google Play, it's built with the same binding and works on both Android and ChromeOS:
512358773-ba1a4412-df71-4f3f-bb12-e1bc97cd12ad

@hanyin-arm
UI: app navigation
32608fb
@hanyin-arm
UI: implement basic UI components
4dd755e
@hanyin-arm
util: implement performance monitor; wrap it with a viewmodel
46bd638
@hanyin-arm
util: implement user preferences utility
5ad6591
@hanyin-arm
UI: implement core flow's screens
7e5c80c
@hanyin-arm
UI: add a new MainActivity; update manifest
ca2b777
@hanyin-arm
[WIP] DI: implement simple local vm factory provider
a7ae8b7
@hanyin-arm
UI: disable triggering drawer via gesture; enable alert dialog on bac…
648b978 
…k navigation inside conversation and benchmark
@hanyin-arm
UI: allow drawer's gesture control only on Home and Settings screens;…
65c09b2 
… enable alert dialog on back navigation inside conversation and benchmark
@hanyin-arm
UI: split a nested parent settings screen into separate child setting…
a7ee3d3 
…s screens
@hanyin-arm
UI: polish system prompt setup UI
5868eaa
@hanyin-arm
Deps: bump Kotlin plugin; introduce KSP; apply in :app subproject
4046cd1
@hanyin-arm
DB: setup Room database
5596d52
@hanyin-arm
data: introduce repo for System Prompt; flow data from Room to VM
4848bf9
@hanyin-arm
bugfix: properly handle user's quitting conversation screen while tok…
75c986a 
…ens in generation
@hanyin-arm
UI: rename ModeSelection to ModelLoading for better clarity
5568184
@hanyin-arm
UI: update app name to be more Arm
64ebdc6
@hanyin-arm
UI: polish conversation screen
3b499ac
@hanyin-arm
data: code polish
fddf060
@hanyin-arm
UI: code polish
e8b84c6
@hanyin-arm
bugfix: handle user quitting on model loading
6b341b0
@hanyin-arm
UI: locks user in alert dialog when model is unloading
e47e3b7
@hanyin-arm
vm: replace token metrics stubs with actual implementation
2a41c0e
@hanyin-arm
UI: refactor top app bars
5e4972e
@hanyin-arm
nit: combine temperatureMetrics and useFahrenheit
af0d68d
@hanyin-arm
DI: introduce Hilt plugin + processor + lib dependencies
65741a7
@hanyin-arm
DI: make app Hilt injectable
564b095
@hanyin-arm
DI: make viewmodels Hilt injectable
a1f6e7e
@hanyin-arm
DI: replace manual DI with Hilt DI
0afd087
@hanyin-arm
UI: optimize AppContent's composing
ea11ee3
hanyin-arm and others added 15 commits October 28, 2025 11:39
@hanyin-arm
llama: update the app's package name and namespace
8897b78
@hanyin-arm
app: bump ksp version
7c2e6d0
@hanyin-arm
app: remove deprecated SystemUIController from accompanist by migrati…
42e3972 
…ng to EdgeToEdge
@hanyin-arm
app: extract AppContent from MainActivity to a separate file in ui pa…
f833c3a 
…ckage
@hanyin-arm
lib: add File version for GGUF Magic number verification
f10d1ab
@hanyin-arm
lib: perform engine state check inclusively instead of exclusively
3644082
@hanyin-arm
lib: change LlamaTier to ArmCpuTier
266fc31
@hanyin-arm
lib: remove kleidi-llama related namings
cadaf80
@hanyin-arm
cleanup: remove Arm AI Chat/Playground app source code; replace with …
f94efba 
…the basic sample app from https://github.com/hanyin-arm/Arm-AI-Chat-Sample

Note: the full Google Play version of AI Chat app will be open will be open sourced in another repo soon, therefore didn't go through the trouble of pruning the history using `git filter-repo` here.
@hanyin-arm
[WIP] doc: update main and Android README docs; add self to code owners
f10a45f
@hanyin-arm
lib: revert System.load back to System.loadLibrary
3fa3c15
@hanyin-arm
jni: introduce a logging util to filter different logging levels on d…
33987b5 
…ifferent build types
@hanyin-arm
lib: enable app optimization
e765543
@hanyin-arm
doc: replace stub Google Play app URL with the actual link add screen…
0bbe3ba 
…shots; add my GitHub ID to maintainer list
@naco-siren
Remove cpu_features
254cd84
@naco-siren naco-siren mentioned this pull request Nov 20, 2025
Open
@loci-dev loci-dev mentioned this pull request Nov 20, 2025
Closed
@github-actions github-actions bot added documentation Improvements or additions to documentation android Issues specific to Android examples ggml changes relating to the ggml tensor library for machine learning labels Nov 20, 2025
@naco-siren
Copy link
Author

naco-siren commented Nov 20, 2025 •
edited
Loading

just merged and resolved conflicts with changes introduced 3 days ago from cb44fc8

@naco-siren
Copy link
Author

naco-siren commented Nov 20, 2025 •
edited
Loading

Not sure why one test case in ggml-ci-x64-amd-vulkan fails...
@slaren This PR doesn't touch Vulkan or AMD at all. Would appreciate it if you could take a quick look, thank you!

Vulkan.csv L9456

"Vulkan0","SUM","type=f32,ne=[11,5,6,3],permute=[0,2,1,3]","support","1","yes","Vulkan"

@naco-siren naco-siren changed the title llama.android : Rewrite Android binding without cpu_features dependency llama.android : Rewrite Android binding (w/o cpu_features dep) Nov 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov Awaiting requested review from ggerganov ggerganov is a code owner

@slaren slaren Awaiting requested review from slaren

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

android Issues specific to Android documentation Improvements or additions to documentation examples ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@naco-siren @hanyin-arm