A fully-local, privacy-first AI assistant with a drop-in skill system.
Facet runs a complete agentic assistant entirely on your own machine: tool calling, long-term memory, workspace search, and multimodal input. No cloud, no API keys, no data leaving your computer. Every capability is a facet: a single Python file you drop into a folder.
Powered by Google Gemma 4 E2B via llama.cpp, on CPU alone.
Asking a question that needs a tool. The call is shown inline with the arguments the model passed, and the panel on the left is the machine actually running the model.
Most "AI assistant" projects are thin wrappers around a cloud API. Facet is the opposite: a self-contained agent stack you fully own.
- 100% local: inference via llama.cpp on your CPU; works offline (except the optional web-search skill)
- Agentic tool use: the model decides when to call tools, chains up to 10 calls per turn, and streams the final answer
- Drop-in skills: add a capability by creating one
.pyfile; hot-reload without restarting - Long-term memory: a local Chroma vector store the assistant can search and save to, across sessions
- Workspace search (RAG): index folders, code, and PDFs, then ask questions about them
- Safety built in: shell commands require interactive approval, every tool call is written to an audit log, and the REST API supports bearer-token auth
- Three interfaces: Rich terminal CLI, a React web app, and a FastAPI REST API with SSE streaming
- Multimodal: image and audio input (optional ~1 GB projector download)
┌─────────────────────────────────────────┐
│ llama-server (Gemma 4 E2B, :8080) │
└────────────────────┬────────────────────┘
│ OpenAI-compatible API
┌────────────────────┴────────────────────┐
│ Facet core │
│ │
│ agent_loop ── tool-call orchestration │
│ skill_registry ── dynamic .py loader │
│ vector_memory ── Chroma long-term mem │
│ workspace_index ── RAG over files/PDFs │
│ approval + audit ── tool safety layer │
│ media ── image/audio encoding │
└───┬──────────────┬──────────────┬───────┘
│ │ │
┌─────┴────┐ ┌──────┴───────────────────┐
│ CLI chat │ │ FastAPI + SSE :8000 │
│ (Rich) │ │ │
│ │ │ serves the React web │
│ │ │ client from web/dist │
└──────────┘ └──────────────────────────┘
The agent loop sends your message plus all skill schemas to the model. If the model returns tool calls, Facet dispatches them through the skill registry (with approval and audit checks), feeds the results back, and repeats until the model produces a plain-text answer, which streams token by token to whichever interface you're using.
Requirements: Linux x86_64 · Python 3.12+ · 8+ GB RAM (16 GB recommended) · ~7 GB disk · no GPU needed
The web app additionally needs Node.js 20+ to build its client once. The CLI and REST API do not.
# 1. One-time system deps
sudo apt install -y cmake
curl -LsSf https://astral.sh/uv/install.sh | sh
# 2. One-time setup: builds llama.cpp, downloads the model (~5 GB),
# installs Python deps. Optionally grabs the multimodal projector.
./scripts/setup.sh
# 3. Run it
./scripts/start.sh # Terminal chat
./scripts/start.sh web # Web app on http://localhost:8000
./scripts/start.sh api # Same server, without building the client firststart.sh manages the llama-server lifecycle for you. It starts the server, waits until it's ready, and shuts it down when you exit.
| Skill | What it does |
|---|---|
web_search |
Search the web via DuckDuckGo |
run_python |
Execute Python in an isolated, resource-limited subprocess |
read_file |
Read local files (sandboxed to your home directory) |
summarize_text |
Condense a block of text into bullet points |
system_info |
CPU, RAM, disk, uptime, top processes |
git_context |
Read-only git status / diff / log |
terminal |
Run shell commands (requires interactive approval) |
memory |
Search and save long-term vector memory |
workspace_search |
Semantic search over your indexed files |
# skills/my_skill.py
SCHEMA = {
"type": "function",
"function": {
"name": "my_skill",
"description": "A clear description so the model knows when to use this.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Input for the skill"}
},
"required": ["query"],
},
},
}
def execute(query: str) -> str:
return f"Result: {query}"Then /reload in the CLI (or POST /skills/reload). The model discovers and uses the skill based on its description alone. A broken skill never crashes the agent, since the registry validates schemas at load time and wraps execution in error handling.
Facet has two kinds of memory:
- Conversation memory: a sliding window of the current chat (trimmed safely so tool-call pairs are never orphaned)
- Vector memory: a persistent local Chroma store (
~/.gemma/memory) with sentence-transformer embeddings
You> /memory add My favourite editor is Helix
You> /workspace index ~/Projects/my-app # index code, docs, PDFs
You> How does my-app handle authentication? # assistant searches the index itself
The assistant can also search and save memories on its own via the memory and workspace_search skills, so facts persist across sessions without you managing them.
Local doesn't have to mean unguarded:
- Tool approval:
terminalcommands pause and ask you to approve each command before it runs (terminal_approval_required: true). The REST API surfaces this as anapproval_requiredresponse / SSE event instead of executing silently. - Audit log: every tool call (skill, args, approval status, duration, outcome) is appended to
~/.gemma/tool_audit.jsonl. Inspect it with/auditorGET /audit. - API hardening: optional bearer-token auth (
api_token), configurable CORS origins, and request-size limits. - Sandboxed execution:
run_pythonruns in a resource-limited subprocess with a hard timeout.
A full-featured Rich terminal client: streaming Markdown rendering, persistent history, tab completion, and slash commands.
| Command | Description |
|---|---|
/image <path> / /audio <path> |
Attach media to the next message |
/paste |
Paste an image from the clipboard (X11/Wayland) |
/memory search|add|forget |
Manage long-term memory |
/workspace index <path> |
Index files for semantic search |
/audit |
Recent tool audit entries |
/save / /load / /list |
Persist and restore conversations |
/export |
Export the conversation as Markdown |
/retry |
Regenerate the last response |
/keep / /unkeep |
Keep the last attachment for the whole chat, or release it |
/system <prompt> |
Change the system prompt mid-chat |
/multi |
Multi-line input mode |
/diagnostics |
Model, server, skill, and memory health |
/skills · /reload · /clear · /help · /quit |
The usual suspects |
Smart auto-attach: type ~/photo.jpg What's in this? and the file is detected and attached automatically.
./scripts/start.sh web → http://localhost:8000. A React client (Vite, TypeScript, Tailwind) served by the same FastAPI process that serves the API, so there is one port and no CORS in production.
- Token-by-token streaming, with tool calls shown inline as you can expand them to see the arguments the model passed
- Slash commands with a keyboard-driven palette: type
/, move with arrow keys, Tab or Enter to complete - Drag, drop, or paste images and audio straight into the composer
- A live panel for what the machine is actually doing: llama-server health, model, context size, thread count, vision on/off
- Light and dark themes, and fonts bundled with the app so it renders with no network at all
Typing / filters the commands as you go. They run against the REST API in the browser, so the model is never called for them.
Everything the CLI can do from a slash command, the web app can too, apart from the ones the browser already covers: attaching is a button, multi-line is Shift+Enter, and quitting is closing the tab.
The first start.sh web builds the client, which needs Node.js 20+. To work on the client itself, run the API with ./scripts/start.sh api and npm run dev in web/ for hot reload on :5173.
If api_token is set in config.yaml, the app asks for the token on first load and keeps it in the browser.
The earlier Gradio UI is still there as ./scripts/start.sh gradio on :7860.
./scripts/start.sh api → interactive docs at http://localhost:8000/docs.
# Chat
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{"message": "What is my system RAM?", "session_id": "s1"}'
# Streaming (SSE): token, skill, approval_required, done, and error events
curl -N -X POST http://localhost:8000/chat/stream \
-H "Content-Type: application/json" \
-d '{"message": "Name three planets.", "session_id": "s1"}'
# With an image
curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{"message": "Describe this.", "session_id": "s1", "media_paths": ["/abs/path/photo.jpg"]}'Other endpoints: GET /skills, POST /skills/reload, GET /health, GET /diagnostics, GET /audit, GET /memory/search, POST /memory, DELETE /memory/{id}, POST /workspace/index, POST /approvals/{id}/approve|deny.
If api_token is set in config, all endpoints except /health require Authorization: Bearer <token>.
Gemma 4 E2B natively accepts images and audio. Download the projector (~986 MB) during setup (or re-run ./scripts/setup.sh), and start.sh enables it automatically.
- Images: jpg, jpeg, png, gif, webp, bmp
- Audio: wav, mp3, ogg, flac, m4a, aac
Without the projector, everything else works in text-only mode.
How long an attachment stays in context. Base64 media is large, so older
attachments are replaced with a placeholder to keep memory and context down.
The memory_keep_recent_media most recent ones (2 by default) keep their real
payload, which is enough to ask follow-up questions about what you just sent.
To keep one for an entire conversation, use /keep — it pins the last
attachment so it is never swapped out, however many turns later you ask about
it. /unkeep releases it.
Edit config.yaml, or override any value with a GEMMA_-prefixed environment variable.
ctx_size: 4096 # Context window (drop to 2048 if RAM is tight)
threads: 6 # CPU threads for inference
temperature: 1.0 # Google's recommended defaults for Gemma 4
top_p: 0.95
top_k: 64
max_tool_iterations: 10 # Max skill calls per turn
skill_timeout: 30 # Seconds before a skill is killed
memory_backend: "chroma"
memory_dir: "~/.gemma/memory"
embedding_model: "sentence-transformers/all-MiniLM-L6-v2"
terminal_approval_required: true
tool_audit_log_path: "~/.gemma/tool_audit.jsonl"
api_token: null # Set to require bearer auth on the REST APIfacet/
├── core/
│ ├── agent_loop.py # LLM ↔ skill orchestration (sync, async, streaming)
│ ├── skill_registry.py # Dynamic skill loader with hot reload
│ ├── inference.py # OpenAI-compatible client → llama-server
│ ├── memory.py # Sliding-window conversation memory
│ ├── vector_memory.py # Chroma-backed long-term memory
│ ├── workspace_index.py # File/PDF chunking + indexing (RAG)
│ ├── approval.py # Tool approval flow
│ ├── audit.py # JSONL tool audit log
│ ├── media.py # Image/audio encoding
│ ├── text_format.py # Output cleanup for terminal rendering
│ └── config.py # YAML + env settings (pydantic-settings)
├── skills/ # ← drop new skills here
├── cli/ # Rich terminal chat + Gradio web UI
├── web/ # React web client (Vite + TypeScript + Tailwind)
├── api/ # FastAPI REST API
├── tests/ # pytest suite
├── scripts/ # setup.sh (one-time) / start.sh (every time)
├── models/ # GGUF files (gitignored)
└── config.yaml
ggml-org/gemma-4-E2B-it-GGUF, Q8_0 quantization (4.97 GB)
- Google DeepMind, April 2026 · Apache 2.0 (commercial use OK)
- 2.3B effective parameters, 128K context, native function calling, multimodal
- Reference performance (i5-1145G7, CPU-only): ~5-10 tokens/sec, ~5-10 s first-prompt latency
uv sync # install dependencies
uv run pytest # run the test suiteTests cover agent safety (approval + audit), API auth, media/memory/workspace handling, and output formatting.
| Symptom | Fix |
|---|---|
| Model "thinks" forever, empty answers | llama-server must run with --reasoning off (start.sh does this) |
| Swap thrashing / system slowdown | Reduce ctx_size to 2048 in config.yaml |
| A skill never gets called | Sharpen its description. The model picks skills by description alone |
| llama-server won't start | Check /tmp/llama-server.log |
| Disk full | Run du -sh models/ llama.cpp/ to see usage (both are gitignored) |