Update: July 19, 2026.
Hi there! I'm Elias, the dude who made Lee-AI.
The current codebase will remain open source under its existing license — feel free to build on it in any way you see fit, commercially or otherwise. Going forward, new development will be proprietary.
...do whatever you want with this repo — it's MIT.
A live AI shop assistant on any website — a visible cursor that talks, points, frames products, draws, and operates the page for your visitors. 🛍️🖱️
Discord · Quickstart · API keys · Adapter · Features · Voice · Live example
Ask it anything in plain language — by voice or text. Its cursor glides across your page with a shimmering star trail, frames the right product in marching red dashes, peeks into the basket, sketches a worked-out price comparison on a pinned sticky note, guides multi-step errands with a visible mission plan, and honestly reports everything it actually did.
Step 1 — run the brain (one file, your terminal; API keys never touch the browser):
ANTHROPIC_API_KEY=sk-ant-... npx -y github:eliasschwalme/avatar-cursor
# → avatar-cursor brain listening on http://localhost:3100/api/avatarStep 2 — add two lines to your site:
<script src="https://cdn.jsdelivr.net/gh/eliasschwalme/avatar-cursor@main/dist/avatar-cursor.js"></script>
<script>
AvatarCursor.init({ brainUrl: "http://localhost:3100/api/avatar" });
</script>That's it. Generic mode scans your page's links, buttons, inputs, and product cards automatically — no build step, no framework, works on any website. Visitors get the launcher button bottom-left; opening it starts a hands-free voice conversation (browser mic, no extra keys) or they just type.
Features unlock progressively — only the first row is required:
| Key | Required? | What it unlocks | Where to get it | Rough cost |
|---|---|---|---|---|
ANTHROPIC_API_KEY |
Yes | The brain — understanding, pointing, actions, ink | console.anthropic.com | pennies/day at typical traffic |
ELEVENLABS_API_KEY |
No | Her real voice (without it: the browser's built-in voices) | elevenlabs.io → Profile → API keys | free tier covers testing |
ELEVENLABS_VOICE_ID |
No | Which voice she speaks with | any voice in your ElevenLabs library | — |
LIVEKIT_URL + API_KEY + API_SECRET |
No | The voice-agent tier: studio STT/TTS + true barge-in (voice-agent/) | cloud.livekit.io → Keys | free tier is generous |
DEEPGRAM_API_KEY |
No | (voice-agent) STT that hears your product names correctly | console.deepgram.com | $200 free credit |
| Runway API key + avatar id | No | (voice-agent) a live lip-synced video face — hard-off by default | dev.runwayml.com | per session-second; opt-in only |
Where keys live — read this before launch: keys go in server-side env
only (avatar-cursor-server and the voice worker read them from env / a
gitignored .env). They are never placed in browser code, never committed, and
never sent to the widget — that's the whole reason the tiny server exists. Use
the .env.example files as templates; if a key ever leaks (pasted in a chat,
committed by accident), rotate it immediately at the provider console.
Everything is optional; each field you add makes it measurably better.
Full reference with rationale: src/adapter.ts.
AvatarCursor.init({
brainUrl: "https://yourdomain.com/api/avatar",
adapter: {
// Teach the scanner your product cards + landmarks
scan: {
items: { selector: "[data-product]", label: (el) => el.dataset.name },
handles: { search: "#search", cart: "#cart-button", checkout: "#checkout" },
},
// Exact app state beats pixel-guessing — the single biggest accuracy win
getState: () => ({ cart: myCart.summary(), page: location.pathname }),
// Your live basket → unlocks the cursor-pinned BASKET PEEK + grounds the
// decision-tree windows in the visitor's real picks
cart: {
items: () => myCart.items.map(i => ({ name: i.name, qty: i.qty, price: i.priceFmt })),
summary: () => ({ subtotal: "…", delivery: "…", total: "…" }),
checkout: () => myCart.openDrawer(),
},
// Site-specific verbs the model may use ("apply_coupon", "book_fitting", …)
verbs: {
apply_coupon: {
description: "apply a coupon code (value=code)",
run: async (_id, code) => myCart.applyCoupon(code) ? "success" : "failed | invalid code",
},
},
persona: {
name: "Ari",
subtitle: "Personal shopper",
style: "Discreet, knowledgeable, one or two short sentences.",
knowledge: "Free delivery over £250. Returns within 30 days. Departments: …",
cursor: "🛍️", // any emoji mascot (default: the arrow pointer)
// face ladder (all optional): portrait image, or two pre-rendered clips
// portrait: "/ari.jpg",
// video: { idle: "/ari-idle.mp4", talking: "/ari-talking.mp4" },
},
},
sounds: true, // full mp3 palette ships built-in; false = silence,
// or per-event overrides: { click: "/my-click.mp3" }
socialProof: { min: 10, max: 150, label: "People Watching" }, // or false
voice: { agent: false }, // true = the LiveKit voice-agent tier (see below)
});- Points & frames — glides to the product and locks a marching-ants red frame around it (the frame tracks the element through scrolling/re-layout)
- Live social proof — "+N People Watching" pill beside the cursor on items
- Basket peek — a glowing ring under the cursor opens a glass mini-bag: live items, totals, quick-action dots, go-to-checkout
- Decision tree — "why this one?" opens the animated how-I-decided glance; "deeper look" opens the big resizable outcome-tree window
- Ink — hand-drawn underlines/boxes, highlighter
MARKs over key facts, freehand shapes, cursive on-screen handwriting - Math notes — worked-out price comparisons handwritten line by line on a pinned translucent sticky note, placed in the emptiest free screen space
- Missions — multi-step goals get a visible plan (✓/→/○ ledger card), honest per-step verification, redo-once on failure, and a hard money gate: it can never reach checkout/payment without the visitor's explicit ask
- Page camera — "show me closer" zooms the page in around the target and holds until released
- Draw-to-ask — visitors hold ⌥ and circle anything; the assistant recognizes what's inside the loop and explains it
- Scroll controls, self-resize, sounds — up/down buttons on the page edge, "make yourself bigger" drag-resizes the widget grip theatrically, and a full mp3 sound palette (glide loop, clicks, chimes) ships in the box
- Honest by construction — every action's real effect is measured and reported back to the model; deterministic guards (name-guard, stuck-guard, wall clock, deadman, money gate) enforce what the prompt teaches
| Tier | Needs | What you get |
|---|---|---|
| Browser voice (default) | nothing | hands-free mic on widget open, auto-restarting turns, energy-based barge-in, browser/ElevenLabs TTS |
Voice-agent (voice: { agent: true }) |
LiveKit + Deepgram + ElevenLabs keys | studio pipeline: vocabulary-boosted STT, word-timed TTS, clause-streamed replies (she starts speaking on the first sentence), true echo-cancelled barge-in |
| Live video face | + Runway key, AVATAR_VIDEO=on |
real-time lip-synced avatar in the widget stage |
Run the worker: see voice-agent/README.md.
Cost controls, built in: closing the widget disconnects the session
(nothing streams while closed) · the in-widget Video / Voice-only pill cuts
Runway per-visitor · AVATAR_VIDEO is hard-off by default — a cloned repo
can never bill a video vendor by accident · the brain's cache pre-warmer is
opt-in (PREWARM=on).
- Deploy
server/index.mjs(ornpx -y github:eliasschwalme/avatar-cursor) behind your domain — setANTHROPIC_API_KEY(+ voice keys if used); pointbrainUrlat it - Optionally run
voice-agent/(any Python host) with your LiveKit project - Add the two script lines + your adapter to the site
- Key hygiene: env-only,
.envgitignored, rotate anything ever exposed; never ship a key in frontend code or this repo sounds,socialProof, and every window are individually configurable — ship only what fits your brand
- Honest actions — never claims success it can't prove
- Deterministic guards over prompt hope — the prompt teaches, the runtime enforces
- Grounding beats pixels — your app's own state outperforms screenshot-reading
- Progressive integration — zero-config works; each adapter field makes it better
- Assistant-side everything — no changes to your site's code, ever
Three moving parts, one hard rule: API keys never touch the browser.
- Widget (
dist/avatar-cursor.js) — everything the visitor sees, in one dependency-free file: the panel, the animated cursor and its verbs (point, frame, scroll, sticky notes, missions), the adapter that reads your page, and the voice client. - Brain (
server/index.mjs) — a small Node proxy that holds your Anthropic key, streams replies, mints LiveKit tokens, and rate-limits the public endpoint. Runs anywhere Node runs:npx -y github:eliasschwalme/avatar-cursor. - Voice worker (
voice-agent/agent.py, optional) — a LiveKit agent that gives her real ears, a studio voice, and a live face (Deepgram STT · ElevenLabs TTS · Runway avatar). It never reasons — transcripts route through the widget into the same brain turn as typed messages, so voice and text stay one conversation. - The subtle edge: the worker only starts the Runway session — Runway then joins the LiveKit room as its own participant and publishes the face video straight to the browser. That's why the avatar face requires a cloud LiveKit URL: Runway connects from the internet and can't reach a localhost room.
Turn-by-turn sequence diagrams, the cost-control valves, and the full dependency web live in ARCHITECTURE.md.
PRs welcome. If you're using Claude Code, it already knows the codebase — just tell it what you want to build and point it at CLAUDE.md.
Got feedback? DM me on X @elias or hop into the Discord.
Avatar-cursor only sees your screen on the integrated website when you talk - screenshots are never stored.