A consent layer for AI agents acting on behalf of the people who can least supervise them.
Stead is a phone-first care companion for Ruby — 84, lives at home, has cerebral palsy — built so that every actor and every AI agent acts only inside scoped, revocable, owner-held, audited consent. The care app is the wedge; the consent layer is the product.
Agents are starting to act on people's behalf. They need the same scoped, revocable, audited governance we'd demand of a human aide. Stead is OAuth/IAM for agents — demoed in the highest-stakes, most human context there is.
General AI literally cannot hear Ruby. Off-the-shelf speech models were never trained on dysarthric (cerebral-palsy) speech, so they mis-hear her — and the millions like her are invisible to the AI revolution. The fix isn't a bigger general model. It's a sovereign one: trained on her data, owned by her family, never folded into a hyperscaler's training set. And none of it is possible without on-demand, dedicated GPU you control. That's Nebius.
Everything in this project starts here. On a Nebius H100 we:
- fine-tuned a personalized speech-pathology model for Ruby's cerebral-palsy speech — the training
loss fell 2.12 → 0.53 on her own recordings, so her agent finally understands her (
model/); - on the same on-demand GPU, the path to robot policy training (MARS learning new skills) — speech pathology today, embodied skills next.
For vulnerable populations this is the unlock: train a model on your own data, sovereignly, without surrendering it. No Nebius → no Ruby model → none of the rest works. Nebius is the foundation the whole thing stands on.
Here's the chain that matters: because Nebius gave Ruby a voice the machine can understand, Composio can turn that voice into connection. Once her agent hears her, Composio lets it act for her across the real world — calendar, email, calls — every action scoped, revocable, and owner-held by her family. The fine-tuned model is the key; Composio is the door it opens. For someone with cerebral palsy, that chain — be understood → then act — is the difference between isolation and connection.
Train sovereign on Nebius → serve sovereign on a DGX Spark (on-prem / edge), so day to day her voice and intent never leave her control.
Nebius makes the sovereign model possible → the model gives Ruby a voice → Composio turns that voice into action → the DGX Spark keeps it sovereign. AI for the people it forgot.
Sister project: Plumbline — the open-source intent-boundary analyzer that hardens any agent acting on someone's behalf (it found, and we fixed, real authority bugs in Stead itself).
- Ruby's screen — her agency: she can't navigate a dashboard, but she can press a big picture of a pizza, and her agent orders it for her — within Mom's limits.
- Mom's dashboard — her control: approve/deny requests, set the spending cap, revoke in one tap, and watch the live consent ledger.
The bridge between them is consent.
- Acts within a grant — Ruby's agent orders dinner ≤ Mom's cap → places a real outbound phone call (Vapi) to the restaurant, in a chosen voice.
- Escalates instead of overstepping — over the cap, it does not fail; it parks a request and asks Mom.
- Owner approves / denies — live, on her phone; approval mints a scoped grant, denial blocks the action.
- Attenuating delegation — when the agent spawns a sub-agent (e.g. the MARS robot for a selfie), the sub-grant can only narrow, never widen (least privilege).
- Everything audited — every grant / act / halt / approve streams to a consent ledger (Neon) and to Langfuse.
- The voice agent only orders within the amount that's allowed. It will not place an order a single cent over Mom's live cap. An over-cap request is paused and routed to Mom for approval — the agent never raises its own limit.
- Ruby's home address is never given out — only verified. On the call, the agent does not read out or hand over her address; it confirms the delivery address the vendor already holds (a yes/no verification), so a personal detail is never broadcast to place an order.
- Owner-held & revocable — only the owner (Mom) can grant, change, or revoke. Revocation takes effect on the agent's very next action.
- Audited end-to-end — no consequential action happens without a logged consent decision behind it.
▶ Live app (Cloud Run — always-on, Neon-backed): https://stead-650440848480.us-central1.run.app
| Page | Link |
|---|---|
| 🏠 Control panel (Mom's dashboard) | https://stead-650440848480.us-central1.run.app/ |
| 🤖 MARS — the interaction point | https://stead-650440848480.us-central1.run.app/mars.html |
| 🎙️ Talk to Stead (voice → action) | https://stead-650440848480.us-central1.run.app/voice.html |
| 🔗 Connect Mom + Ruby's calendar/email | https://stead-650440848480.us-central1.run.app/connect.html |
The public instance is call-safe + spend-safe (Vapi/card keys stripped from the cloud box) — the consent engine, the Neon-backed shared brain, the MARS mock, and the live Claude voice agent are all real and verifiable. Sister tool: Plumbline — the open-source intent-boundary analyzer that hardens any agent acting on someone's behalf.
📹 Demo short coming — it'll be embedded here.
The pizza order also runs from the robot: robot/skills/order_pizza.py + robot/skills/log_mood.py
deploy to ~/skills/, and robot/agents/ruby_agent.py to ~/innate-os/agents/. Ruby tells MARS how
she's feeling → it's logged to her care timeline; she says she's hungry → MARS offers Tony's and places
the consent-gated call. Point the robot at the laptop with STEAD_URL=http://<laptop-ip>:8770.
| Piece | What it is |
|---|---|
Consent engine (orchestration-kit/consent_agent/) |
Bolo-shaped grant / check / revoke / act + request → approve/deny + attenuating delegate. Owner-held, audited. In-memory backend keeps tests deterministic; a Neon Postgres "shared brain" backend lets the app and agent (separate processes) see the same live consent state. |
Care app (app/) |
Installable PWA (manifest + service worker), stdlib server on :8770. Ruby's day + the consent ledger live in Neon; Claude powers "Ask Stead." Ruby's kiosk is at /ruby.html. |
Voice / phone (agent/tools/phone.py) |
Real outbound calls via Vapi (Vapi-native voice, or an ElevenLabs cloned voice). |
Robot (robot/, Vocalizer) |
MARS senses Ruby's day and speaks/acts over rosbridge. |
| Observability | Local audit always on; Langfuse is the optional external mirror. |
python orchestration-kit/grade.py exits 0 only when the whole tier/spiral passes — each tier independently demoable, every lower tier kept green:
T1 RBAC · T2 care companion · T3 live app · T4 act-on-behalf (HALT on overstep) · T5 voice through MARS · T6 request → approve → attenuating delegation
The tiers were built by an autonomous build-verify loop (orchestration-kit/workflow/build-verify-loop.js): grade → build the lowest red tier → have independent agents adversarially verify the fix didn't game the tests or regress a banked tier → re-grade.
pip install -r orchestration-kit/requirements.txt
cp .env.example .env # fill in your keys (see below)
python app/server.py # http://localhost:8770 · Ruby's screen: /ruby.html
python orchestration-kit/grade.py # the scorecard (exits 0 when done)Stead runs against Neon (already cloud), so the only thing tying it to a laptop was the local process. Host the server and it's permanently live — app and robot point at one stable URL:
render.com → New → Blueprint → connect this repo (reads render.yaml)
→ paste your secrets in the dashboard (never committed)
→ deploy → https://stead-XXXX.onrender.com
Then on the Jetson: export STEAD_URL=https://stead-XXXX.onrender.com and the robot orders through the
cloud, not the laptop. (Render free tier sleeps when idle; Starter/Railway/Fly keep it warm.) The server
binds $PORT automatically. Procfile + requirements.txt work on any PaaS with the same start command.
All secrets live in .env (gitignored — never commit real keys). See .env.example:
ANTHROPIC_API_KEY— "Ask Stead" + the agentDATABASE_URL— Neon Postgres (Ruby's day + consent ledger); falls back to local SQLite if unsetVAPI_API_KEY,VAPI_PHONE_NUMBER_ID,VAPI_TARGET_PHONE— outbound callsSTEAD_VOICE_PROVIDER(vapi|11labs),STEAD_VOICE_ID,ELEVENLABS_API_KEY— call voiceROSBRIDGE_URL— the MARS robot (e.g.ws://<robot-ip>:9090)LANGFUSE_PUBLIC_KEY/LANGFUSE_SECRET_KEY— optional tracing
Built in a day. Demo-grade, not production — but every consent decision is real.