Repository navigation
Proposal: a Llama app for iPhone and iPad #124
thom-dev-fr
started this conversation in
Ideas
Replies: 1 comment
|
Hi @erusev, a gentle follow-up. I know this is a big one to review. Just one question to unblock me: would an iOS app have a place in this repo at all? A one-word answer is perfect, and "no" is a completely fine answer: it helps me plan either way. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi!
I'd like to propose an iOS/iPadOS version of Llama, which I'd contribute and maintain.
Immediate ask: Is iOS support a direction you'd be open to exploring, and would you prefer it in this repository or a separate one?
The initial scope would be model installation and on-device chat using the existing WebUI and unmodified llama.cpp. There's a working preview below. Native chat, agent features, and App Store distribution would be separate discussions; no commitment to those is needed now.
Who I am and why I'm here
I'm a software engineer at a large French tech company. On my own time, I've been building and maintaining BonzAI for over a year: a hobby iOS app that runs local inference on iPhone and iPad with llama.cpp. It uses the engine's multimodal and tool-calling support to run agentic scenarios on the device.
BonzAI runs on a fork of llama.cpp. I'm preparing a clean version of that work for upstream (ggml-org/llama.cpp#29981). Nothing in this proposal depends on the fork: the branch below builds unmodified llama.cpp
b11517.When I found Llama-macOS, I saw that it shares BonzAI's philosophy: local models, simple to use, and close to llama.cpp. I think this project is the right open-source home for the iOS work I've done on BonzAI.
Vision
iPhones and iPads keep getting more powerful. I'd like Llama to let people use that power without friction: offline inference first, then agentic scenarios such as MCP servers and Apple Shortcuts. The iOS app would share as much as possible with the Mac app, including model handling, Hugging Face resolution, the catalog and, ideally, the llama.cpp version.
What exists today
thom-dev-fr/Llama-macOS@ios-preview, rebased onmainLlama-ios-preview-demo-chat.mov
Llama-ios-preview-demo-decision.mov
What the app does:
MobileServer.cppserves the router API the WebUI expects (/models,/models/load,/models/sse, …) from a single process. It hands requests to the unmodified llama.cpp handlers, and only one model is in memory at a time.llama://installlinks from llama.app.llama fit-params, and only offers context sizes that fit the device.models.inifor diagnosis.Maintenance
I'd take responsibility for iOS-specific fixes, testing on physical devices, and checking compatibility with llama.cpp updates. For changes to shared code, I'd include macOS build and test results. I'd work with you on the scope and review process before opening PRs.
Proposed split
The branch is too large to review in one go (~5,900 lines). I propose two PRs, and every commit builds on its own.
PR 1: Extract platform-independent code into a
LlamaCorepackage (touches macOS, ~1,400+ / 800−, mostly moves)Models, HF cache, download plan, catalog,
llama://links and compatibility move into a local Swift package with unit tests. No behavior change intended for the Mac app. This PR only makes sense if you want the iOS app; I'm not proposing it as a standalone refactor.PR 2: The iOS app (iOS only, ~4,500 lines in 3 commits, meant to be reviewed commit by commit)
models.ini, memory fit, Settings./v1/systemone.The last commit is separable: I can drop it from PR 2 and propose it later.
Possible future work (outside this proposal)
These are directions I'd be interested in exploring later, subject to your feedback. The initial contribution would not depend on them.
From BonzAI, not yet in this branch:
New:
Questions
If there's interest in principle, we can agree on the contribution split before I open any PRs. Feedback on these two questions alone would be enough to guide my next step.
Disclosure
This branch was written with the help of an AI coding assistant . I directed, reviewed and tested every change, and I'm responsible for it like any code I submit.
I won't open any PR until you say so. Thanks for reading!
All reactions