v4 — 'The Orb': Zen Mode, Computer Control, Marketplace & More

v4 ??? "The Orb" ????

A JARVIS-class desktop AI that runs entirely on your hardware ??? now with a radical minimal UI, real computer control, and total-privacy options.

??? Headline Features

???? Zen Mode ??? The default UI is now a single monochrome orb + chat. Everything else hides behind ???B. Ambient floating widgets orbit the orb; hover or drag to expand.

??????? Computer Control ??? Friday can open apps, focus/list/close windows, type, press keys, and click your desktop. "Organize my desktop and summarize" is a single autopilot goal. Every action asks for your confirmation first.

???? Plugin Marketplace ??? Install and remove community plugins in one click from Settings.

???? Custom Tool Builder ??? Describe a tool in natural language and Friday generates, registers, and persists it. No code required.

???? Local RAG Pipeline ??? Ingest documents and retrieve them with sentence-aligned chunking + lexical reranking over the three parallel memory engines.

???? Knowledge Graph ??? Entities and relations extracted from your chats; every session seeds context ??? "Last time you were working on???".

???? Blackout Mode ??? One toggle for total privacy: network tools blocked, local Ollama forced, PRIVATE seal on the orb.

???? More

• Onboarding ??? first-launch "Hello, I'm Friday" orb greeting with flagship suggestions
• Single-command start ??? python main.py --ui or npm run friday boots API + frontend and opens your browser
• Global hotkey ??? Ctrl+Alt+F summons Friday from anywhere
• Shareable Moments ??? capture canvas + message snapshots as share cards
• Holodeck v2 ??? floating draggable cards with gesture openness
• Session continuity via knowledge graph + diary

???? What's Inside

Frontend	React 19 ?? Three.js ?? Vite ?? Tailwind CSS v4 ?? TypeScript
Backend	Python 3.11+ ?? Quart (async)
LLM	OpenRouter ?? OpenAI ?? Ollama ?? any OpenAI-compatible API
Memory	TF-IDF + Vector + Embeddings + RAG + Knowledge Graph
Control	pyautogui ?? pywin32 (Windows)
Tests	278 Python ?? 107 TypeScript ??? CI green on both suites

???? Privacy

Your data stays yours. Memory lives in local JSON, voice/wake-word runs offline in-browser, and Blackout mode guarantees zero outbound AI traffic.

???? Thanks

To every contributor, stargazer, and user of Friday ??? this release is for you.

v3 — Personality, Automations, Vision & Holodeck

v3 — Voice Personality · Automations · Vision · Holodeck

Voice Personality

3 distinct personas — J.A.R.V.I.S. (calm British wit), FRIDAY (warm proactive), Cortana (knowledgeable formal). Each has a unique system prompt, TTS rate, and pitch. Switch anytime via settings or Cmd+K.

Automations

Schedule recurring actions with cron expressions. Create via natural language. Background engine checks every 30s. Toggle, run manually, or delete from the Intelligence panel.

Screen & Camera Vision

Analyze your screen or webcam via LLM vision. Ask "what's on my screen" for instant description. Optional OCR via pytesseract. Background SSE push on screen changes.

Holodeck

Real-time 3D data visualization using Three.js — animated metric bars (CPU/RAM/LAT/TOK), ambient particle fields, orbital rings. Camera follows hand gestures when webcam is active.

Other

• Ambient Voice conversation mode (natural back-and-forth, auto-send on pause)
• Morning Pulse briefing (weather, news, crypto, calendar, email — zero LLM cost)
• SSE reconnection with exponential backoff

v2 — Performance Overhaul + Voice + Gestures + Google Integrations

v2 — Performance overhaul, SSE push, async backend, polished UI