Your own AI assistant. On your laptop. In code you can read in an afternoon.

Meet Waku — a local-first personal assistant that shows the four pillars behind every serious agent: Harness · Loop · Memory · Eval/LLM-Ops. No frameworks hiding the good parts. Built for Sean's AI Stories.

• Local-first. Your memory is one SQLite file. Open it. Read it. It's yours.
• Memory is the hero. Semantic + episodic + procedural — with a gate that decides whether to remember, and a pass that decides what to keep.
• The loop is ~95 lines of plain Python. Step through it.
• Watch it think. A local dashboard lights up every message as it flows through the harness.
• Eval built in. Deterministic tests and LLM-as-judge, side by side, with a release gate.

The system-design whiteboard from the Sean's AI Stories series. For the code-accurate version (every box → a file it maps to), see The whiteboard maps to the code below.

▶ Watch the 20-min code walkthrough: You Can Build Your Own Local AI Agent — every part of this repo, live: the loop, the memory pillars, the evals, the Telegram gateway, and the "Waku Waku" voice wake word.

---

• Video walkthrough: You Can Build Your Own Local AI Agent In 20 Min
• YouTube: Sean's AI Stories
• X: @ShenSeanChen
• LinkedIn: shen-sean-chen
• Instagram: @sean_ai_stories
• Discord: Sean's Discord · AutoManus Discord

• ☕️ Buy me a coffee: Cafe Latte
• 🚀 Launch Full-Stack App Template: launch-mvp-stripe-nextjs-supabase — NextJS + Supabase + Stripe, everything you need to ship a SaaS
• 💻 My AI startup: AutoManus.io — an AI sales lead manager for made-to-order product sales. We embed where conversations happen (WhatsApp, email, web chat) to capture inbound, automate follow-ups, and kill CRM busywork. Pre-seed backed by Character VC.

git clone https://github.com/ShenSeanChen/waku-agent && cd waku-agent
uv venv && uv pip install -e .          # create the env + install the `waku` command
cp .env.example .env                    # pick a provider, paste ONE key
uv run waku                             # talk to your Waku in the terminal
uv run waku dashboard                   # …or the browser cockpit → localhost:7777

uv run waku … needs no venv activation. Three ways to run it:

waku and waku dashboard are two doors into the same Waku. The dashboard is a tiny web server on your machine — chat in the browser, that process runs the turn. Nothing leaves your laptop. Set TELEGRAM_BOT_TOKEN and it starts your bot too. (make dashboard works as well.)

Now try it. "Remember that Alex prefers morning meetings." Quit. Restart. "Book a catch-up with Alex on Friday." → it remembers, and books 9am. Your memory is one file: .waku/state.db.

Use the model you already pay for. Anthropic (default), OpenAI, Gemini, DeepSeek, MiniMax, Kimi, GLM, or OpenRouter (one key, hundreds of hosted models) — set WAKU_PROVIDER=, paste the key, done. One dialect in the loop; a ~60-line adapter handles the rest.

waku dashboard          # starts a local server → http://localhost:7777

A small web server you own (127.0.0.1, no cloud). The browser is just the UI — the same process runs every turn. This is the fastest way to get the system.

A chat dock sits on every tab. Type or speak, and watch it flow through the harness on the Overview diagram: gate lights up → loop calls a tool → reply comes back → memory updates. The frontend is plain static files. No build step.

Each tab is one pillar, linked to the real files:

The sidebar and chat dock are drag-resizable and hideable, and the chat has New chat + history like any chat app.

Type these in the chat dock (or make run) and watch the dashboard light up:

The money shot is the World Cup one. In one turn, Waku searches the web a few times, reasons over the results, and books every remaining match — 8 loop iterations, live. Needs a free TAVILY_API_KEY (paste it in Settings). Watch the LOOP box pulse per cycle. That's loop engineering, on tape.

Those are products you use. This is a codebase you own — the loop, the memory schema, the gate, the eval harness, all yours to read and change. Understand this repo, and you understand what the products do under the hood.

Versus the big open-source assistants (OpenClaw, Hermes)? Same architecture, 1/100th the code. Products vs. a readable blueprint.

Every whiteboard from the videos lives in docs/whiteboards/ as an editable .excalidraw source — download one, drop it on excalidraw.com, and remix it for your own team:

New charts land here with every video. If they help you, a star keeps them coming — and sponsoring gets new whiteboards early.

This diagram renders straight from the README (it's Mermaid text, not an image — edit it in a PR):

flowchart LR
  GW["Gateway<br/>cli · telegram · voice · dashboard"] --> WM["Working memory<br/>SOUL.md + memory + history"]
  WM --> LLM
  subgraph LOOP["The Loop — loop/agent.py"]
    LLM["LLM"] -->|tool call| TOOLS["Tools<br/>create_event · list_events<br/>search_web · save_note · …"]
    TOOLS -->|result| LLM
  end
  LLM -->|reply| REPLY["Reply"] --> GW
  GATE{{"Retrieval gate<br/>does this turn need memory?"}} -. only if needed .-> WM
  MEM[("Memory — state.db<br/>SQLite + FTS5<br/>semantic · episodic · procedural")] --> GATE
  REPLY -. save chat .-> MEM
  MEM -->|every N chats| CONS["Consolidate → facts"] --> MEM
  REPLY --> OPS["LLM Ops<br/>trace → eval → gate → release"]
  OPS -. improved prompt/config .-> WM
  WM -.- WATERMARK["waku-agent · Sean's AI Stories · @ShenSeanChen"]:::wm
  classDef wm fill:none,stroke:none,color:#9aa0aa,font-size:11px;

Architecture of waku-agent — built on Sean's AI Stories (@ShenSeanChen). Code is MIT; this diagram is licensed CC BY-NC-SA 4.0 — reuse it with credit to the channel, not for commercial resale.

Every box is one module (full version with every file path: docs/architecture.md):

A note on MEMORY.md vs state.db. Some assistants (e.g. Hermes) keep long-term memory as a single MEMORY.md markdown file. Waku keeps the queryable source in state.db (the facts and episodes tables, keyword-searchable via FTS5) and regenerates a human-readable .waku/MEMORY.md mirror after every turn — so you get both: a real file you can open, backed by a sturdy database. The dashboard's Memory tab is the friendly view; the Database tab shows the raw state.db tables.

Yes, there's a real agent loop, and it's ~95 lines of plain Python — no LangGraph, no hidden control flow:

while not done:
    response = llm(messages, tools)      # reason
    if response wants tools:
        results = run(tool_calls)        # act
        messages += results              # observe
    else:
        done                             # reply to the human

Two guardrails end every turn: the model stops asking for tools (natural end), or it hits max_iterations (hard stop — it never spins forever). That's "loop engineering": the exit conditions, the tool round-trip, and feeding results back as working memory.

How to show it on camera:

1. Type "schedule a swim with Sergey Saturday at 5pm" in the chat dock and watch the LOOP box on the Overview diagram light up: reason → create_event → reason → reply.
2. Open the Loop tab — every turn is listed with its gate decision, each tool call, the iteration count, tokens, and dollar cost. A tool-using turn shows iter 2 (reason, act, then reason again to reply); a plain answer shows iter 1.
3. Open the Ops tab (or .waku/traces/<today>.jsonl) to read that same turn as raw events in order: turn_start → gate → llm → tool → llm → turn_end. That's the loop, on tape.

The multi-tool loop (the money shot). One tool is a loop; chaining tools is where loop engineering earns its name. Try:

"Search for the World Cup games still left to play and add each one to my calendar."

The agent loops across two tools: search_web reads the web, it reasons over the results, then calls create_event once per match — several iterations in a single turn. You'll see iter 4, iter 5… on the Loop tab and the LOOP box pulse for each cycle. search_web works keyless via DuckDuckGo but that endpoint rate-limits bots, so for a clean take set a free TAVILY_API_KEY (see .env.example).

1. The retrieval gate. Most agents hit their memory store on every turn. That's slow, and worse — irrelevant memories bias answers. Here a cheap model first answers one question: does this message need memory at all? Watch it in the terminal:

you > what's 2+2?
  gate · skip — pure math
you > when am I meeting Alex?
  gate · retrieve — references user's plans

2. Deterministic eval vs LLM-as-judge. "Did it create the right calendar event?" is a unit test — 0 or 1, no model judges it (make eval). "Was the reply helpful?" is a judged score with a threshold (make eval-judge). Conflating the two is the most common eval mistake; here they're separate suites you can diff. make gate runs both as a release gate.

Three commands, two kinds of eval — the LLM-Ops half of the system:

make eval          # deterministic: "did the right tool fire?" — 0 or 1, no model judges it
make eval-judge    # LLM-as-judge: "was the reply helpful?" — a scored %, needs a key
make gate          # the release gate: deterministic must pass 100%, judge must clear threshold

Deterministic tests are plain pytest in evals/deterministic/; judged ones use DeepEval in evals/judge/. Keeping them apart is the whole point — conflating "did it do the thing" (a unit test) with "was it any good" (a scored judgement) is the most common eval mistake.

Where the results show: the terminal, and the dashboard's Ops tab — the release-gate verdict, an eval-history table (one row per make gate, so you can see it grow), the actual per-turn gate decisions, and the raw traces inline.

The bug workflow (this is the discipline you show on camera): when you catch a bug by using the thing live, you fix it AND add a deterministic case so it can never come back. A real example from this repo: the agent didn't know the current time and asked for it before scheduling "in 30 minutes" → fixed in session.py, locked forever by test_working_memory.py. Run make gate → green → the eval history records the run.

Spend is permanent: every LLM call's tokens are appended to .waku/usage.jsonl — an append-only ledger that a demo reset never wipes. The Ops tab shows the all-time cost, tokens, and a per-day / per-provider breakdown (dollar cost is estimated from tokens, which are the ground truth). So the number you show on camera is your real running total, not a per-session guess.

Tracing is always on: every turn appends readable lines to .waku/traces/<date>.jsonl (zero setup) — a trace is just "what happened, in order." For span-waterfall views:

pip install -e '.[tracing]'
make trace                                            # Phoenix at localhost:6006
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 make run

Langfuse cloud speaks the same OTel toggle.

python scripts/demo_seed.py --yes      # resets .waku to a tidy, curated state (--yes required)

It backs up your current .waku first, then seeds a few clean facts, one episode, and one event — Sergey's standing Saturday 5 PM swim. The chat log and traces start empty, so when you type live the Loop, traces, and Gateway inbox fill up in front of the viewer. The memory/Data/Tools tabs already have tidy content to explain. Edit the seed lists at the top of the script to taste.

uv pip install -e '.[voice]'
waku voice        # hands-free: always-listening for "waku waku"

Hands-free by default. waku voice listens for the wake word "waku waku" — a tiny Whisper model scans the mic; when it hears the phrase, the big model takes over for your command and speaks the reply. Change or disable it:

WAKU_WAKE_WORD="hey waku"  waku voice     # any phrase, no training
WAKU_WAKE_WORD=""          waku voice     # push-to-talk instead (Enter, speak, Enter)

The matcher is ~15 transparent lines with a deterministic eval; it accepts cross-script variants ("waku waku,わくわく"). A trained openWakeWord model is the efficient v2 upgrade.

A beautiful voice. Out of the box it uses macOS say — and Waku auto-picks the nicest voice you have, preferring a downloaded Premium/Enhanced one (System Settings ▸ Accessibility ▸ Spoken Content ▸ System Voice) over the robotic built-ins. For the real neural upgrade, install Kokoro — a fully local, offline British-butler voice that's picked up automatically, no env var needed:

uv pip install '.[voice-neural]'          # neural Kokoro (bm_george); pulls torch (~2GB)

Override either engine with WAKU_VOICE (a say voice name, or a Kokoro voice like bf_emma).

pip install -e '.[telegram]'
# message @BotFather, /newbot, put the token in .env, then:
make telegram

Text your bot from anywhere and your laptop runs the turn — long-polling, so no public URL or webhook. Set TELEGRAM_ALLOWED_USER to lock it to just you.

WAKU_APPLE_TOOLS=1 make brief      # macOS; grant the permission prompts once

Waku reads your real Calendar.app (including events invited by email) and recent Apple Mail, cross-references your memory, and writes a focus-first briefing with clickable message:// links. Cron it for a morning greeting:

30 7 * * *  cd ~/waku-agent && make brief

It runs through the normal harness, so it animates on the dashboard like any turn.

The agent has tools to keep itself useful — no black box:

• manage_memory — correct or forget a fact when you say it's wrong.
• update_soul — save a standing preference you give it (lives in SOUL.md).
• create_skill — when you teach it a repeatable workflow, it offers to save it as a skill (written to .waku/skills/, live the same session).

You can also edit any of this by hand on the dashboard's Memory tab (edit/delete facts, rewrite SOUL.md) or in Settings (switch provider/model, paste keys — BYOK, kept in your local .env, never sent to the browser).

pip install -e '.[mcp]'

Create .waku/mcp.json and any Model Context Protocol server's tools appear to the agent, namespaced <server>_<tool> (and in the dashboard's Tools ▸ MCP tab):

{"servers": [{"name": "fs", "command": "npx",
  "args": ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]}]}

Node-free demo — a tiny self-contained Python MCP server ships in the repo:

cp examples/mcp.demo.json .waku/mcp.json   # points at examples/mcp_demo_server.py
make dashboard                               # demo_word_count / demo_reverse_text appear in Tools

Same pattern scales to any server, yours or a vendor's — no changes to Waku's code.

Skills are procedural memory: markdown instructions loaded only when relevant.

python -m waku skill install https://github.com/<someone>/<repo>/blob/main/skills/<skill>/SKILL.md

Contribute one — it's just a markdown file. Copy skills/TEMPLATE.md, PR it into skills/community/. CI validates the frontmatter. See CONTRIBUTING.md.

The waku command is installed with the package; the make targets are equivalent aliases.

These live in waku/tools/experimental.py, OFF by default — WAKU_EXPERIMENTAL=1 registers them.

Sub-Agents is now LIVE. delegate_task hands a coding job to pi — Mario Zechner's minimal open-source coding agent — through its headless print mode (pi -p "task"). Waku stays the orchestrator (memory, context, evals); pi is the specialist contractor (read/bash/edit/write). Try it:

npm install -g --ignore-scripts @earendil-works/pi-coding-agent
WAKU_EXPERIMENTAL=1 uv run waku
# "have pi fix the failing test in ~/my-project"

The full pi transcript lands in .waku/outbox/delegate-*.log; tune the budget with WAKU_DELEGATE_TIMEOUT (default 300s).

The rest are still deliberate skeletons — the intent is drawn so the diagram maps to something, but nothing is over-promised (they report "coming soon", and the dashboard's Tools tab lists them under Coming soon):

The point of a teaching repo is a readable core; these come alive one at a time, tested.

launch-rag · launch-agentic-rag · launch-agent-skills · launch-mcp-demo · launch-DeepResearch-Backend

Star the repo, join the Discord, and grab a good first issue — gateway adapters (WhatsApp, Discord), memory backends, and community skills are all designed to be first PRs.

MIT — see LICENSE. Built by @ShenSeanChen (YouTube · X).