The future belongs to those who build their memory today.
Jarvis is a personal AI system I built to turn my whole digital life into one unified, queryable knowledge base. Most AI tools can read an email or a calendar entry when you ask. Jarvis instead keeps persistent context across everything: who I met, what we discussed, what I am working on, and what matters to me. The difference is like showing someone your photo album versus having a friend who was actually there for every moment.
This repository is a public showcase. Jarvis predates OpenClaw, and the focus has always been the same: solid data capture first, and a clean foundation on top of which functions and agents can be layered later. The full system runs my private infrastructure and is not open source. What you find here is the architecture, the story, and a curated set of the components that are genuinely useful to others, with all personal data and secrets removed.
Jarvis has three parts:
- Knowledge Hub : a central Postgres database (Supabase) that stores and connects everything.
- AI Agents : background services that capture, process, and act on the data.
- Interfaces : how I interact with it day to day (Claude Desktop via MCP, Notion, Telegram).
See docs/ARCHITECTURE.md for the full diagram and data flow.
1. Aggregate everything. Conversations, emails, messages, meetings, notes, health, screen time. Every signal matters, and the longer the system runs, the smarter it gets. This compounds. Everything is built modularly so new models, data sources, and services just plug in.
2. Multiply productivity. Perfect recall of any conversation or commitment. Contextual awareness of who I am meeting and what we last discussed. Proactive support that surfaces the right information at the right moment, instead of me chasing it.
3. Strengthen memory and learning. The risk of a second brain is that the first one atrophies. So Jarvis is a two-way street: it resurfaces topics, quizzes me, and flags knowledge gaps. The goal is not to replace memory but to train it.
Capture and digest. I send a voice memo over Telegram, or feed in a meeting recording. The pipeline transcribes it (WhisperX on GPU), analyses it with Claude, routes it to the right bucket (meeting note, reflection, task), extracts contacts and action items, and stores everything in the Knowledge Hub. A confirmation with a summary comes back.
Pre-meeting briefings. Fifteen minutes before a calendar event I get a briefing: who I am meeting and their background, when we last spoke and about what, recent email threads, open action items, and suggested talking points.
Evening summary and journaling. Every evening: a summary of the day, reflection prompts based on what actually happened, and a preview of tomorrow.
Query anything via Claude Desktop. Claude connects straight to the Knowledge Hub over MCP, so I can ask things like "what themes keep coming up in my reflections this month?" or "based on this investor's background and my deck, what should I prepare for?"
Notion as a frontend. A daily workspace over the same data, synced bidirectionally, so changes anywhere flow back to the hub.
| Typical AI plus connectors | Jarvis |
|---|---|
| Reads emails when asked | Everything interconnected |
| No memory between sessions | Persistent, growing knowledge |
| You provide most of the context | Context is always there |
| Reactive only | Proactive briefings and summaries |
| Generic responses | Personalised to my history and style |
These are the parts of the system that stand on their own and are useful to others. Each has been stripped of credentials and personal data; configuration is shown via .env.example files.
mcp-server/: a schema-aware Model Context Protocol server that exposes a personal Postgres knowledge base to Claude Desktop and other MCP clients, with safe read and human-approved write tools.audio-pipeline/: the voice and meeting digestion pipeline. Audio in, WhisperX transcription (with speaker identification), Claude analysis, structured data out.utilities/: small device and automation helpers that feed the system or reduce friction:voice-hotkey/: push-to-talk voice capture on the desktopchrome-extension/: a browser capture extensionir-bridge/: a small infrared device bridgelight-scheduler/: lighting automationboox-dialog-killer/: an Android accessibility helper for Boox e-ink deviceswindows-notifier/: desktop notifications from the agents
The system was built in phases, capture first, intelligence later.
- Phase 1, Data Hub. Unified Supabase database. Contact, task, email, and calendar sync. Transcription pipeline writes here first. Done.
- Phase 2, First Steps. Telegram interface, Claude Desktop MCP, daily digest, pre-meeting briefings, follow-up reminders. Done.
- Phase 3, RAG and Deep Context. Vector search over the full history, so Jarvis can reason across everything. In progress.
- Phase 4, Real Jarvis. Real-time voice, actions on my behalf with a human-in-the-loop, proactive nudges, and a spaced-repetition learning loop. Future.
The live system holds a lot of personal data, so the working repository is private by design. This showcase exists to share the architecture and the reusable pieces, not the data. Nothing here contains real credentials, tokens, or personal records. If you spot anything that looks like it should not be public, please open an issue.