A warm voice companion that talks with Indian elders, and quietly keeps their family in the loop.
Saathi calls you Ramesh ji, never tum. It waits when you pause to find a word. It does not correct your memory. And if you say your chest hurts, it stops being a companion and starts being an alarm — your family knows within seconds.
All five phases are built. npm run verify runs 45 unit tests and 36
endpoint checks; everything passes.
| Phase | Scope | Status |
|---|---|---|
| 1 | WebRTC voice engine, empathy persona, safety shield, DPDP foundation | ✅ Built |
| 2 | Post-call memory extraction, pre-call context brief, family dashboard | ✅ Built |
| 3 | Autonomous PSTN calling agent — no app required | ✅ Built |
| 4 | Nostalgia podcasts, memory capsules, regional Indian voices | ✅ Built |
| 5 | Care-home portal, clinical export, cost optimisation | ✅ Built |
What each phase delivers, and what is honestly still missing, is in docs/ROADMAP.md. One item blocks production: the consent notice does not yet disclose that audio is processed outside India.
You need Node 20+ and an OpenAI API key with Realtime access.
npm install
cp .env.example .env.local # then fill in OPENAI_API_KEY
npm run devOpen http://localhost:3000 and press Baat kijiye. Saathi greets you first.
Two variables are enough to hear it speak:
OPENAI_API_KEY=sk-...
SESSION_SIGNING_SECRET=$(openssl rand -hex 32)Without Supabase configured the app runs in demo mode against a built-in elder profile (Ramesh ji, a retired schoolteacher from Kanpur). Calls work and latency is measurable; nothing is persisted.
Create a Supabase project in the ap-south-1 (Mumbai) region — this is a DPDP data-residency requirement, not a preference. Then:
psql "$DATABASE_URL" -f supabase/migrations/0001_core_schema.sql
psql "$DATABASE_URL" -f supabase/migrations/0002_rls_policies.sql
psql "$DATABASE_URL" -f supabase/seed.sqlAdd the Supabase keys and NEXT_PUBLIC_ELDER_ID to .env.local.
TELEPHONY_PROVIDER=console (the default) logs alerts instead of sending them,
so nobody's actual mother gets a 3am SMS during development. For real delivery
set twilio or exotel — Exotel is the better choice in India, but note that
TRAI requires DLT registration or the operator will silently drop your messages.
npm run verify # typecheck + all tests + endpoint checks (needs npm run dev)| Suite | Covers |
|---|---|
npm run test:unit |
27 — WAV/chunking, cost, routing, prompt-safety |
npm run test:bridge |
18 — audio codec, anti-aliasing, bridge↔app contract |
npm run verify:phase1 |
17 — auth boundaries, emergency latency, VAD config |
npm run verify:phases |
19 — Phase 2–5 authorization and webhook signatures |
Things that need a human or real infrastructure — median voice latency,
no-cutoff-on-pause, a live PSTN call, Sarvam synthesis — are listed with their
commands at the end of npm run verify:phases, and in
docs/PHASE1.md.
┌─ Browser ────── WebRTC audio (direct) ──────┐
│ │
Elder ──────────────┤ ▼
│ OpenAI Realtime
└─ Any phone ──PSTN──► Twilio ──ws──► bridge/ ─┘
(μ-law ↔ PCM16)
│
▼
Next.js (Vercel, bom1)
│
┌──────────────────────────┼──────────────────────────┐
▼ ▼ ▼
Supabase Postgres Exotel / Twilio Sarvam Bulbul
(ap-south-1) SMS + alert calls regional TTS
│
├─ family dashboard /dashboard
└─ care-home portal /facility
Two ways in, one brain. A browser call holds the peer connection with OpenAI
directly — audio never touches our servers, which removes a hop of latency
and means we are not paying to relay bytes for every concurrent elder. A phone
call cannot do that, so bridge/ relays it instead, resampling 8kHz μ-law to
24kHz PCM per frame.
Both paths converge on the same persona, the same memory brief, the same
report_emergency tool, and the same post-call extraction.
The OPENAI_API_KEY never leaves the server. The browser gets an ephemeral
ek_… secret, scoped to one session config, expiring in ten minutes.
The browser drives two privileged actions: writing transcript turns and firing
an emergency escalation that sends real SMS to real people. A plain callId in
a request body would let anyone POST an emergency for any elder whose id they
guessed. So /api/session mints an HMAC token binding the browser to one
(callId, elderId) pair, and every privileged route verifies it — see
src/lib/session-token.ts.
The VAD is deliberately patient. OpenAI defaults to 500ms of silence before deciding you have finished speaking. Saathi waits 900ms, with 400ms of prefix padding. Older speakers pause mid-sentence to find a word or to breathe, and being cut off by a machine is the fastest way to make someone stop using it. The cost is ~400ms of added turnaround. It is worth it.
Saathi never corrects the elder's memory. If they say their granddaughter is in Class 8 and our notes say Class 9, Saathi goes with Class 8. Being right is worth nothing here. This is validation therapy, and it is the single most important line in src/lib/realtime/persona.ts.
Emergencies do not ask permission. The persona instructs Saathi to fire
report_emergency mid-conversation, without checking first, on any acute
symptom. A false alarm costs a family one worried phone call.
The alert row is written before any SMS is sent. If telephony fails we still have a durable record. Losing the record because a gateway timed out would be the worst outcome available.
The UI is built for 70-year-old eyes. 22px base type, 88px minimum touch targets, AAA contrast, and a warm cream palette rather than clinical white, which glares under Indian tubelights. Every state is words and colour and motion — never colour alone.
src/
app/
page.tsx elder call screen (no login, ever)
dashboard/ family: trends, symptoms, transcripts, audio
facility/ care home: triage board, clinical view
api/session|turns|emergency Phase 1 call plumbing
api/memory/extract Phase 2 extraction worker
api/telephony/* Phase 3 dispatch, webhooks, bridge context
api/media/* Phase 4 podcasts, capsules, signed URLs
api/facility/* Phase 5 clinical export, batch dispatch
api/consent | api/elder/data DPDP consent, access, erasure
hooks/useRealtimeCall.ts browser WebRTC engine
lib/
realtime/ persona, tools, elder-tuned VAD
memory/ extraction schema, worker, context brief
telephony/ Twilio/Exotel, TwiML, webhook signatures
voice/ Sarvam TTS, WAV joining, script chunking
media/ podcast + capsule generation, storage
clinical/ 30-day summaries, FHIR R4
cost/ pricing, pipeline routing, compression
facility/ RBAC and staff access logging
session-token.ts HMAC call + attempt binding
bridge/ standalone media bridge (deploy separately)
supabase/migrations/ 0001–0006, schema + RLS
scripts/ verification harnesses
docs/ PHASE1, DPDP, ROADMAP
The full list, ordered by how much each matters, is in docs/ROADMAP.md. The ones worth knowing before you go further:
- Cross-border transfer is not disclosed in the consent notice. Data at rest is pinned to Mumbai, but audio streams to OpenAI outside India and the notice does not say so. This blocks production, and it is a copy change plus a legal review, not an engineering project.
- Non-Hindi consent notices are unreviewed drafts. A notice the elder did not truly understand is not consent.
- No retention policy. Data persists until someone calls the erasure endpoint. DPDP s.8(7) wants better.
- The bridge does not reconnect. A dropped model socket ends the call in silence; it should speak an apology first.
- Every price in
cost/pricing.tsis an assumption. Reconcile against real invoices before quoting a care home. - ABDM export is FHIR-shaped, not certified. Real ABDM needs HIP registration and sandbox certification — largely a non-engineering project.