Repository navigation
Home
A full custom OS and assistant stack built on top of Rokid AR glasses — with root access and Rokid's own software almost entirely stripped out and replaced. It's a voice/vision AI assistant platform: glasses + an Android companion phone + a backend of agents and speech/vision services, all self-hosted.
Tech enthusiasts and developers who want to build their own thing on real AR hardware, not a locked-down consumer product. Some features aren't polished for a regular consumer — this set is meant for people ready to keep building on top of it (or contribute later, if they want).
Built to be forked and extended, not just used. The whole stack was developed test-driven through scripts, so between the existing code and the per-device docs you have a real reference to build from — not a blank page. You can build almost anything on top of it: new glasses apps, new AI agents, new device tools, your own integrations.
Rokid's own software was almost entirely removed and replaced, so the system is fully incompatible with Rokid's phone companion or CXR libraries — by design, to escape their technical debt and give you a clean base to build on.
On the glasses, hands-free
- Voice AI assistant — wake-word or button activated; ask questions and run actions hands-free on the HUD.
- Real-time two-way translation — translates a live conversation in both directions, on the glasses and the phone.
- Copilot — an always-on conversation assistant that follows along and surfaces helpful cards.
- Teleprompter — scrolling script on the HUD.
- Navigation — turn-by-turn with a minimap, journey planning, transit steps, and ETA, using both Google Maps and Yandex Maps.
- Capture — photos, video, and AR-screen recordings by voice.
- Person recognition — identify people via face/gait.
Through the phone companion
- Desktop & mouse control — drive a desktop over WebRTC, including head-motion → mouse.
- AI chat — full conversation UI, with a photo you just took auto-attached to your question.
- Notifications, alarms, to-dos, and scheduled autonomous "jobs" the assistant runs for you.
- Telegram — read and act on your messages by voice.
Built to extend
- Add your own AI agents — they self-register with the orchestrator; no redeploy to add one.
- Multi-language — English and Russian speech in/out, plus on-device translation.
- Fully self-hosted — your own backend and keys; nothing phones home except the AI models and map APIs you choose.
Open source? Yes — mostly. A few repos holding some features stay private (not listed here);
Direct hardware access (camera frames, mics, IMU, buttons/touch, HUD)? Yes — root access, and the existing apps already use all of it, so the code is there to copy. Though some SoC still remains inaccessible, this won't prevent you from building most of things.
Replaces the CXR stack, or a compatibility layer? Full replacement, no compatibility. Rokid is just hardware now.
Local API / WebSocket / IPC for third-party apps (assistant, maps, notifications, vision)? Not out of the box — the stack is designed to be self-contained and self-sufficient (except AI models, maps APIs, and partially translation/speech transcription), and the deployment is strictly isolated. But since it's open, nothing stops you from building an integration. The OS wasn't originally meant to be shared, so that's a designed drawback.
iPhone? No — the companion app is Kotlin/Android. I don't have an iPhone, and this wasn't originally meant for release.
Build custom AI agents / integrate external services? Yes — you'll be able to do almost anything.
Am I free to use, modify, and redistribute it however I want? Yes.
How do I contribute? Open a pull request.
Where can I ask questions or get help? Join the Discord: discord.gg/EYbsMwrhb2. It's the official Rokid community server, but there are developers there who can help with this custom OS.
- Architecture — port allocation, service map, data flow, build + deploy.
- Installation — stand up the whole stack on bare metal.
- Features — what the phone and glasses clients actually do, with screenshots and clips.
| Project | Repo | Purpose | Port |
|---|---|---|---|
| Orchestrator | jaskier-os/orchestrator | Central router/classifier/dispatcher; owns @orchestrator/sdk | 10001 |
| Communicator | jaskier-os/communicator | LLM API gateway, OpenAI-compatible -> Anthropic Claude | 10000 |
| Anthropic STT | jaskier-os/anthropic-stt | Speech-to-text via Anthropic | 10016 |
| Transcriber | jaskier-os/transcriber | Audio transcription (faster-whisper); built, not cluster-deployed | 10003 |
| Kokoro TTS | jaskier-os/kokoro-tts | Text-to-speech (English) | 10007 |
| Tera TTS | jaskier-os/teratts-tts | Text-to-speech (Russian; GLaDOS voice) | 10018 |
| Translator | jaskier-os/translator | NLLB-200 multilingual translation | 10015 |
| OCR | jaskier-os/ocr | OCR / text extraction from images | 10006 |
| Project | Repo | Purpose | Port |
|---|---|---|---|
| Chat History Agent | jaskier-os/chat-history-agent | Conversation history store (WS agent + REST API) | 10014 |
| ClickUp Agent | jaskier-os/clickup-agent | ClickUp integration (MCP + agentic wrapper) | 10012 |
| Vision Agent | jaskier-os/vision-agent | Image analysis | 10005 |
| PC Agent | jaskier-os/pc-agent | NL -> shell with safety + remote control; built, not cluster-deployed | 10004 |
| Project | Repo | Purpose | Port |
|---|---|---|---|
| ReID Worker | reid/reid-worker | Camera feed -> person/face/gait recognition; posts to reid-db-handler | 9284 |
| ReID DB Handler | reid/reid-db-handler | Only component allowed to touch the ReID/FAISS database | 3001 |
| ReID Analytics | reid/reid-analytics | Aggregates reid-db-handler data; recognized-people dashboard | 3400 |
| Project | Repo | Purpose | Port |
|---|---|---|---|
| Client Desktop | jaskier-os/client-desktop | Desktop relay client (Python + PyQt6) | - |
| Client Glasses | jaskier-os/client-glasses | Rokid AR glasses listener app (Kotlin) + firmware | - |
| Client Phone | jaskier-os/client-phone | Android companion app (Kotlin) + glasses APK relay | - |
| Component | Image | Purpose | Port |
|---|---|---|---|
| MongoDB | mongo:7 | Orchestrator session/chat store | - |
| PostgreSQL | postgres:16-alpine | ReID relational store | - |
| FlareSolverr | ghcr.io/flaresolverr/flaresolverr | Cloudflare/JS challenge solver | - |
jaskier-os — whole-system documentation.