Phi is an experimental, research-oriented assistant designed to run entirely on a modern Apple silicon Mac. It demonstrates how compact language models and pragmatic engineering can deliver fast, private, and extensible AI capabilities without cloud dependencies. Built for students, developers, and researchers exploring on-device NLP, Phi prioritizes privacy, modularity, and practical utility: secure local notes, summarization, study planning, and conversational Q&A served over an HTTP interface that integrates easily with web UIs, scripts, and automation.
- Privacy-first: user data and inference remain on-device by design.
- Local-first: inference runs locally with pluggable, on-device LLM backends.
- Research-friendly: minimal dependencies and a modular architecture make it ideal for experimentation with inference, prompting, and model adapters.
- Practical: focused features for productivity and study workflows (notes, summaries, planning) delivered over a programmatic HTTP surface.
- Language: Rust (performance + reliability)
- Platform: macOS (developer-friendly, local-first deployment)
- Execution model: on-device LLM inference exposed via an HTTP JSON API (optional web UI)
- Build: Cargo
- Primary goals: local LLM experimentation, private note-taking, and study-assistant features
What it already does
- HTTP JSON API (and optional web UI) for conversational and programmatic access
- Pluggable local language model integration (adapter pattern in
src/assistant/model.rs) - Persistent local note storage with a simple storage API
- Question answering and text summarization
- School-focused assistance: study planning, concise summarization, and task scaffolding
- Structured logging and robust error handling for safe iteration
Planned / near-term
- Native macOS integrations (menu bar utilities, notifications)
- Voice input/output
- Encrypted local storage option
- Richer personal knowledge base integration and caching
Key design principles
- Minimal trusted surface: keep models and user data local and auditable.
- Modular backends: swap local LLMs or inference engines without changing assistant logic.
- Simple, auditable code: Rust + clear module boundaries for safe experimentation.
Core modules
src/main.rs— HTTP server entrypoint and routersrc/assistant/— assistant core logic and skillsmodel.rs— language model integration / adapterskills.rs— skill implementations (notes, Q&A, summarization, planners)
src/storage/— local persistence (notes, storage primitives)src/utils/— logging, configuration, and helpers
Why Rust?
- Performance for on-device inference experiments
- Memory safety without a GC
- Strong static typing for reliability and maintainability
- Excellent tooling (Cargo) for fast iteration
Why local models?
- Privacy: user data never leaves the device
- Offline availability and robustness
- Full control for tuning and customization
- Strong educational value: direct exposure to inference, prompt engineering, and system design
- Model size and responsiveness are constrained by local hardware — very large models may be impractical on some Macs.
- No live internet knowledge updates — responses reflect the local model’s knowledge and any integrated local KB.
- Encryption / secure storage is not currently implemented — notes remain local and unencrypted unless you add an encrypted storage layer.
- Native macOS UI and voice features are planned but not yet implemented.