A hackathon project that analyzes nursery video footage to detect crying, movement, adult presence, and generates AI-powered summaries of baby sleep patterns.
Built for OpenAI Build Week — track: Apps for Your Life. Powered by OpenAI GPT-5.6 (vision + language) and built with Codex. Codex Session:
019f6a4a-a5e3-70d0-a562-577310890273
cd backend
chmod +x run.sh
./run.sh
# Or manually:
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
python app/main.pyServer runs at http://localhost:8000
cd frontend
python -m http.server 8001Visit http://localhost:8001
┌──────────────────────────────────┐
│ Frontend (HTML/CSS/JS) │
│ - Starfield night sky │
│ - Drag-drop upload UI │
│ - Results dashboard │
└──────────────┬────────────────────┘
│ HTTP REST + CORS
↓
┌──────────────────────────────────┐
│ Backend (FastAPI + Python) │
│ - /api/upload │
│ - /api/analysis/{id} │
│ - /api/alert-rules │
└──────────────┬────────────────────┘
│
┌─────────┼─────────┐
↓ ↓ ↓
Video Audio OpenAI
Proc. Anal. GPT-5.6
(OpenCV) (librosa) (Vision+LLM)
- Primary Night:
#060B1A,#0B1330,#121C43 - Sky Blues:
#6FA8DC,#A9D8F5 - Baby Blue:
#E4F3FC - Accents:
- Crying:
#F5B8C4 - Movement:
#C9B6F2 - Adult:
#8FE3C0 - Sleep:
#6FA8DC
- Crying:
cradleAI/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI entry
│ │ ├── config.py # Settings
│ │ ├── api/
│ │ │ └── routes.py # API endpoints
│ │ ├── services/
│ │ │ ├── video_processor.py # OpenCV
│ │ │ ├── adult_detector.py # YOLOv8 person detection
│ │ │ ├── audio_analyzer.py # librosa crying detection
│ │ │ ├── openai_analyzer.py # OpenAI GPT-5.6
│ │ │ ├── sleep_interpreter.py # Signal fusion + night narrative
│ │ │ └── detection_engine.py # Event merge + timeline
│ │ ├── models/
│ │ │ └── schemas.py # Pydantic models
│ │ └── database/
│ │ ├── db.py # SQLite
│ │ └── models.py # SQLAlchemy
│ ├── requirements.txt
│ ├── .env.example
│ └── run.sh
├── frontend/
│ ├── index.html
│ ├── css/
│ │ └── main.css # All styles
│ └── js/
│ ├── api.js # Backend calls
│ └── ui.js # Interactions
├── docs/
│ ├── SETUP.md
│ ├── API.md
│ └── FEATURES.md
├── README.md
└── .gitignore
- ✅ Video Upload — Drag & drop interface with progress tracking
- ✅ Crying Detection — Audio pitch/energy/timbre analysis, plus GPT-5.6 vision cues
- ✅ Movement Detection — OpenCV frame analysis with noise filtering
- ✅ Adult Detection — YOLOv8 person detection, plus GPT-5.6 vision
- ✅ Sleep Interpretation — Fuses signals into caregiver visits, wake-ups vs restlessness, cry episodes, and sleep state (
sleep_interpreter.py) - ✅ Timeline Generation — Chronological event list with correlated details ("responded 40s after crying began")
- ✅ AI Summary — GPT-5.6 narrative with a deterministic detector fallback
- ✅ Sleep Quality Score — 0-100 calculated from interpreted events
- ✅ Night History — Past uploads with status, click through to reload any analysis
- ✅ Beautiful UI — Night theme, staged upload/processing/dashboard flow
OPENAI_API_KEY=your_key # GPT-5.6 (vision + language)
DATABASE_URL=sqlite:///cradle_ai.db
UPLOAD_DIR=./uploads
MAX_VIDEO_SIZE_MB=500
CORS_ORIGINS=http://localhost:3000,http://localhost:8000,http://localhost:8001,http://localhost:5173SQLite with 3 tables:
- Videos — Upload metadata
- Analyses — Detected events + summary
- AlertRules — User configuration
# requirements.txt already set up
# Set env vars on Render dashboard
# Push to main → auto-deploys# Static HTML/CSS/JS → zero config
# Just update API_BASE to your backend URL- Video player with timeline scrubbing
- PDF export of reports
- Multi-night comparison
- Parent notifications (email/SMS)
- Mobile app (React Native)
- Sleep coaching recommendations
Team: review this section before submitting — expand it with any additional Codex sessions you used.
Codex Session ID: 019f6a4a-a5e3-70d0-a562-577310890273
We used OpenAI Codex as our primary development partner and GPT-5.6 as the product's AI engine:
- Where Codex accelerated us: Codex implemented the GPT-5.6 vision integration (
analyze_video_frameswith strict JSON-schema output over sampled frames), connected the upload-to-analysis flow end to end (background processing with its own DB session, status polling, error surfacing), fixed the frontend API client so it runs in a plain browser, and hardened the upload pipeline (chunked streaming writes, size limits, cleanup on failure). - Key decisions we made (not Codex): the OpenAI-only architecture, the standard event schema shared by every detector, the multi-detector design (OpenCV movement + YOLOv8 adult presence + librosa audio crying + GPT-5.6 vision as a complementary signal), the night-timeline UX and staged upload flow, and the sleep-quality scoring model.
- How GPT-5.6 contributes to the product: GPT-5.6 vision analyzes sampled nursery frames and returns structured, confidence-scored events for adult presence, visible crying cues, and sleep state (
app/services/openai_analyzer.py). These vision cues run alongside dedicated detectors — OpenCV frame-diff for movement, YOLOv8 for adult presence, and audio pitch/energy/timbre analysis for crying — merged into one timeline (app/services/detection_engine.py), then reasoned over bysleep_interpreter.pyinto caregiver visits, wake-ups vs. restlessness, and correlated insights. GPT-5.6 also writes the final night summary from those interpreted events and stats; without an API key the app falls back to a deterministic summary built from the same data, so the pipeline degrades gracefully rather than failing.
OpenAI Build Week — track: Apps for Your Life
Submission deadline: July 21, 2026, 5:00 PM PT
Team (3 members): @marinemee · @Intechgent · @stolen-yellowteeth
MIT — Go build something amazing! 🌙