Practice conversations. Analyze real interactions. Build confidence that sticks.
Several of us have tutored and worked with children, and over time we noticed something: neurodivergent kids struggled to feel welcome in group settings in ways that felt social, not academic. They were smart, engaged learners, but navigating conversations and reading the room was hard for them. We wanted to build something that could help.
Special education programs at K-12 schools are chronically underfunded, which means neurodivergent students often don't get the structured support they need to develop social skills. The tools that exist are either locked behind expensive therapy or designed for clinical settings, not everyday practice.
Stanford built an AI social coach focused on teaching empathy to autistic children, and 71% of users showed measurably increased empathy responses after using it. That told us the approach works: a personal AI coach you could practice with anywhere, anytime, without a therapist in the room.
CoCo is an iOS app that gives you a personal AI coach for social skills. It works across three tracks.
Track 1: Daily Interaction Log - Record a voice memo of any real conversation. A local AI model (via ZETIC) processes it entirely on-device and surfaces a sentiment score, social cues, strengths, and specific improvement areas.
Track 2: AI Coach - Based on your logs, your coach breaks down patterns, explains what happened, and lets you practice that exact situation. You can also pick from general personas:
- Alex - small talk, low-stakes catch-ups
- Morgan - behavioral questions, professional pressure
- Jordan - first introductions, talking about yourself
- Sam - setting boundaries, disagreeing respectfully
Tap Reflect after any session for one strength, one improvement, and one exact line to try next time.
Track 3: Progress Dashboard - Tracks sentiment scores, improvement streaks, and skill growth over time. Earn points and badges for logging interactions, completing sessions, and hitting personal bests.
| Layer | Technology |
|---|---|
| iOS framework | Swift / SwiftUI |
| Android framework | React Native + Expo |
| Voice AI | ElevenLabs Conversational AI + LiveKit WebRTC |
| On-device analysis | ZETIC Melange (social cue detection, sentiment) |
| Android AI | Gemini |
| Backend / Auth / Storage | Supabase |
| Navigation | Native iOS Navigation / Expo Router (file-based) |
The practice flow gives each persona a custom system prompt built from the user's onboarding profile. Their name, anxiety level, goals, and focus areas all feed directly into how the AI behaves in that session.
The recording flow pipes audio into ZETIC Melange running on the device's NPU, then pushes the structured analysis to Supabase. No audio is sent to the cloud.
Daily scenarios are generated by cross-referencing the user's profile against their prior sessions, so suggestions are always grounded in recent gaps rather than generic advice.
Finding the right idea - There are too many generic hackathon projects out there. We wanted to build something with a real impact, and the education sector kept coming up. It usually focuses on large groups, but the people who need the most help are individuals, specifically the ones working on skills that no classroom teaches.
ZETIC + Swift - We wanted on-device inference for privacy reasons but couldn't get ZETIC working with React Native. After reaching out to one of their advisors, we learned we'd need to switch to Swift. Privacy was non-negotiable for us given that users are logging personal conversations, so we built the main app in Swift for iOS.
External API error handling - ElevenLabs and other APIs were returning generic error codes with no useful context, so we had to build out our own error handling layer to get reliable behavior.
Free tier limits - ElevenLabs' free tier kept hitting its cap mid-session, so we had to cycle through API keys to keep things running during the demo.
- Built a full end-to-end voice practice loop where you go from tapping to live audio in under 5 seconds, through the full reflect and save flow
- On-device social cue analysis that runs entirely on the user's phone with zero audio ever leaving the device
- A persona system that adapts to the user's stated goals and recalibrates daily based on what recent sessions show
- A UI that actually feels calm and human, because we knew that if the app felt clinical or cold, people with social anxiety just wouldn't open it
- A Stripe-inspired character design system that keeps the visual language consistent and polished across every screen, making the app feel intentional rather than thrown together
- Real-time voice AI on mobile is genuinely hard. Latency alone is a problem, but WebRTC state management and graceful error recovery on top of that means every edge case needs its own solution
- On-device inference with ZETIC Melange is fast enough to feel instant, and that matters a lot when the pitch of your app is that nothing leaves the phone
- The interface itself has to feel safe. For a mental health-adjacent app, design is not a layer on top of the product, it is part of what makes the product work at all
- Android support - The Android app is currently in React Native. We plan to port it to native Kotlin so we can bring the ZETIC bridge, and full on-device analysis, to Android users
- Spaced repetition - Resurface specific scenarios on a schedule based on long-term retention curves, so weak areas actually get drilled over time
- Therapist mode - A read-only dashboard where a coach or therapist can review a client's progress alongside them
swift react-native zetic-melange elevenlabs supabase claude-api webrtc npu on-device-inference