Strategic Intelligence, Logistics, Voice & Integrated Assistant
SILVIA is a locally-hosted, voice-enabled AI operating system for engineers, makers, and power users. All inference runs on your machine via Ollama. No cloud required. No data leaves your network.
SILVIA is not a chatbot. It is a personal command center that integrates your entire digital and physical environment:
- Talk to your infrastructure — check CPU load, SSH into nodes, send commands to drones and robots
- Control your desktop — launch apps, close processes, search files, open KiCad projects
- Manage hardware projects — track inventory, check build readiness, log orders, import BOMs
- Stay organized — tasks, reminders, calendar events, project health reports
- Monitor proactively — Watch Officer alerts on telemetry spikes and node outages
- Use your voice — full wake word → STT → LLM → TTS pipeline, everything local
┌──────────────────────────────────────────────────────────┐
│ BROWSER / UI │
│ React SPA (Vite) — Command Center · Hardware Board │
│ Infrastructure · Mission Panel · Voice Interface │
└─────────────────────┬────────────────────────────────────┘
│ HTTP REST + WebSocket
┌─────────────────────▼────────────────────────────────────┐
│ FastAPI Backend :8000 │
│ │
│ ┌────────────┐ ┌─────────────────┐ ┌──────────────┐ │
│ │ Planner │ │ HW Assistant │ │ Node/Service │ │
│ │ qwen2.5:3b │ │ (regex router) │ │ Registry │ │
│ └─────┬──────┘ └────────┬────────┘ └──────┬───────┘ │
│ │ │ │ │
│ ┌─────▼──────────────────▼──────────────────▼────────┐ │
│ │ Tool Layer │ │
│ │ Desktop · Node · Watch · Tasks · Hardware · Voice │ │
│ └─────────────────────────────┬──────────────────────┘ │
│ │ │
│ ┌─────────────────────────────▼──────────────────────┐ │
│ │ SQLite Databases │ │
│ │ nodes.db · hardware.db · missions.db │ │
│ └────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
│ Ollama HTTP API
┌─────────────────────▼────────────────────────────────────┐
│ Local LLMs (Ollama) │
│ gemma3:4b · phi4-mini-reasoning · qwen2.5:3b │
└──────────────────────────────────────────────────────────┘
| System | Status | Description |
|---|---|---|
| Voice Pipeline | ✅ | Wake word → STT → LLM → TTS, fully local |
| Desktop Awareness | ✅ | App discovery, file indexing, launch preferences |
| Node Registry | ✅ | Register, monitor, and command remote machines |
| Service Registry | ✅ | Map services (NAS, media, robot) to nodes |
| Capability Registry | ✅ | Execute named capabilities (media.play, motion.forward) |
| Watch Officer | ✅ | Telemetry alert rules with info/warning/critical severity |
| SSH Terminal Launch | ✅ | Opens Windows Terminal tab for any registered node |
| Hardware Inventory | ✅ | Parts registry with auto-categorization |
| Hardware Projects | ✅ | Project-part links with 10-status model |
| Build Readiness | ✅ | Check if current stock covers all project requirements |
| Procurement Engine | ✅ | Low-stock alerts, reorder thresholds, delivery tracking |
| BOM Import | ✅ | CSV and KiCad BOM import |
| Vision Inventory | ✅ | Image → component detection → inventory update |
| Tasks & Calendar | ✅ | Tasks, reminders, calendar events |
| Project Registry | ✅ | Maker project tracking with health reports |
| Semantic Memory | ✅ | Search past conversation history by meaning |
| Brain63 Integration | ✅ | Read-only Obsidian vault access |
| Hermes Engine | ✅ | Multi-step autonomous task execution |
| Scheduled Tasks | ✅ | Recurring background tasks |
| Morning Briefing | ✅ | Daily status from live data — no hallucination |
| Multi-Brain Debate | ✅ | SARASWATI + LAKSHMI + DURGA consensus decisions |
| External Notifications | 🔜 | Discord/email for Watch Officer alerts |
| Telemetry History | 🔜 | Historical charts per node |
| Fleet Commands | 🔜 | "Land all drones" — multi-node simultaneous |
- Python 3.11+
- Node.js 18+
- Ollama installed and running
- Windows 10/11 (macOS/Linux: core features work, desktop control is Windows-only)
git clone https://github.com/your-account/SILVIA.git
cd CMD-CTR
# Backend dependencies
pip install -r backend/requirements.txt
# Frontend dependencies
cd frontend && npm install && cd ..ollama pull gemma3:4b # Main conversation model
ollama pull phi4-mini-reasoning # Multi-brain reasoning
ollama pull qwen2.5:3b # Tool planning
# Optional — for local vision analysis
ollama pull llavacp .env.example .env
# Edit .env with your settings (see Configuration section)# Terminal 1 — Backend
python -m uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
# Terminal 2 — Frontend (dev)
cd frontend && npm run devOpen http://localhost:5173 in your browser.
For production frontend: cd frontend && npm run build → serves from dist/ on port 8001.
Create .env in the project root:
# ── Required ──────────────────────────────────────────
OPENWEATHER_API_KEY=your_key_here
TIMEZONE=Asia/Kolkata
# ── Brain63 (Obsidian vault — READ-ONLY) ──────────────
BRAIN63_VAULT_PATH=C:\Users\YourName\Documents\GitHub\Brain63
# ── Voice: Speaches (optional, recommended) ───────────
# Speaches must be on a different port than SILVIA (8000 conflicts)
SPEACHES_URL=http://localhost:9000
SPEACHES_STT_MODEL=rtlingo/mobiuslabsgmbh-faster-whisper-large-v3-turbo
SPEACHES_TTS_MODEL=speaches-ai/Kokoro-82M-v1.0-ONNX
SPEACHES_TTS_VOICE=af_aoede
# ── Vision (Phase 12F) ────────────────────────────────
# Option A: Anthropic Claude Vision (best accuracy)
ANTHROPIC_API_KEY=sk-ant-...
VISION_PROVIDER=auto # auto | anthropic | ollama
VISION_MODEL_ANTHROPIC=claude-haiku-4-5-20251001
# Option B: Ollama (no key needed, pull llava first)
VISION_MODEL_OLLAMA=llava
VISION_CONFIDENCE_THRESHOLD=0.65
# ── Notifications (optional) ──────────────────────────
NOTIFICATION_WEBHOOK_URL=https://discord.com/api/webhooks/...
NOTIFICATION_WEBHOOK_FORMAT=discord # discord | slack | json
NOTIFICATION_MIN_SEVERITY=critical # warning | critical
# ── Authentication (optional) ─────────────────────────
# Leave empty for localhost-only use (no auth required)
API_KEY=
# ── Web search (optional) ─────────────────────────────
SEARXNG_URL=http://localhost:8080CMD-CTR/
├── backend/
│ ├── app/
│ │ ├── api/ # FastAPI route handlers
│ │ │ ├── hardware.py # Hardware Board API (40+ endpoints)
│ │ │ ├── nodes.py # Node registry API
│ │ │ └── ...
│ │ ├── core/ # App startup, auth middleware, WebSocket
│ │ ├── models/ # Pydantic request/response schemas
│ │ ├── orchestration/ # Assistant router and mode dispatch
│ │ ├── services/ # All business logic
│ │ │ ├── hardware_service.py # Inventory/project/order DB
│ │ │ ├── hardware_assistant_service.py # Chat-based HW control
│ │ │ ├── hardware_vision_service.py # Image analysis (Phase 12F)
│ │ │ ├── node_service.py # Node registry DB
│ │ │ ├── watch_service.py # Watch Officer alerts
│ │ │ ├── mission_service.py # Tasks, reminders, calendar
│ │ │ ├── hermes_service.py # Multi-step task execution
│ │ │ ├── conversation_service.py # LLM conversation loop
│ │ │ └── semantic_memory_service.py # Conversation search
│ │ ├── tools/
│ │ │ ├── planner.py # Tool dispatch (all SILVIA commands)
│ │ │ ├── node_tool.py # Node operations
│ │ │ └── time_tool.py # Time utilities
│ │ └── world/ # RSS, world model
│ ├── voice/ # STT, TTS, VAD, wake word detector
│ ├── config.py # All configuration constants
│ ├── main.py # FastAPI entry point
│ └── requirements.txt
├── frontend/
│ ├── src/
│ │ ├── components/ # UI panels and widgets
│ │ ├── lib/api.js # All backend API calls (single source of truth)
│ │ └── pages/ # Top-level page components
│ └── package.json
├── docs/ # All documentation
├── prompts/ # LLM system prompts (SARASWATI, LAKSHMI, DURGA)
└── .env # Local config — never commit this file
| Document | What it covers |
|---|---|
| Architecture | System diagrams, data flow, LLM routing |
| Features | Every feature with components and examples |
| Commands | Complete command reference by category |
| Hardware Board | Inventory, projects, orders, BOM, Vision |
| Hardware Assistant | Chat routing, all HW commands |
| Infrastructure | Nodes, telemetry, Watch Officer, SSH |
| Node Registry | Node schema, types, agent protocol |
| Service Registry | Services mapped to nodes |
| Capability Registry | Executable capabilities |
| Voice System | Wake word, STT, TTS, VAD |
| Desktop Awareness | App control, file search |
| Project Registry | Maker project tracking |
| Inventory Registry | Hardware inventory system |
| Roadmap | Completed phases and future plans |
| Developer Guide | How to add tools, services, endpoints |
| Troubleshooting | Common issues and fixes |
- Windows-primary — app discovery and SSH terminal launch use Win32 APIs
- Voice requires microphone — wake word detection uses the default audio input device
- Vision requires setup — needs either
ANTHROPIC_API_KEY+pip install anthropicorollama pull llava - Brain63 is read-only — SILVIA cannot create or edit Obsidian notes
- No cloud backup — all data lives in local SQLite files (
backend/*.db) - Ollama must be running — all LLM inference fails gracefully if Ollama is offline
| Phase | Status | Description |
|---|---|---|
| 1–4 | ✅ Done | Core assistant, multi-brain debate, voice pipeline |
| 5–7 | ✅ Done | Nodes, Watch Officer, Hermes engine, robotics protocol |
| 8–9 | ✅ Done | Proactive intelligence, morning briefing |
| 10 | ✅ Done | Universal node capability layer |
| 12A–12F | ✅ Done | Hardware Board — inventory through vision analysis |
| 13 | 🔜 Planned | Historical telemetry charts |
| 14 | 🔜 Planned | Fleet commands (multi-node simultaneous) |
| 15 | 🔜 Planned | External notifications (Discord/email) |
| 16 | 🔜 Planned | Full API authentication |
MIT — see LICENSE
SILVIA is a personal project.