CareSight is an AI-powered healthcare assistant designed specifically for elderly, disabled, and neurodiverse users. It simplifies medical information, provides accessible interfaces, and includes comprehensive care coordination features.
- Real-time OCR processing with Tesseract.js
- Simplifies medical documents into plain language
- Text-to-speech for easy listening
- Supports prescriptions, lab results, and medical forms
- Google Gemini AI integration with medical-focused prompts
- Simple, elderly-friendly explanations
- Voice input and output capabilities
- Educational information with healthcare provider reminders
- Larger text and buttons for easy reading
- High contrast mode for better visibility
- Slower, patient voice synthesis
- Simplified navigation with essential features
- Real EmailJS integration for caregiver alerts
- SMS simulation (ready for Twilio integration)
- Emergency and general alert types
- One-click contact system
- Real-time AI health monitoring with MediaPipe
- Hand gesture recognition and tremor detection
- Posture analysis and monitoring
- Heart rate estimation from facial analysis
- Eye tracking and attention monitoring
- Medication reminder system
- Health metrics tracking with AI insights
- Privacy-first local processing (no data sent to servers)
- Speech-to-text input for hands-free operation
- Text-to-speech output with elderly-friendly settings
- Browser-native Web Speech API integration
- No external dependencies
- Clone and install dependencies:
git clone <repository-url>
cd caresight
npm install- Set up environment (optional):
cp .env.example .env.local
# Edit .env.local with your API keys (see DEPLOYMENT.md)- Run development server:
npm run dev- Open in browser: Navigate to http://localhost:3000
- Advanced AI Health Monitoring with MediaPipe integration
- Hand gesture recognition and tremor detection
- Posture analysis and real-time feedback
- Heart rate estimation from facial analysis
- Eye tracking and attention monitoring
- Enhanced Document Scanner with Tesseract.js OCR
- Real-time medical document processing
- Smart classification (prescriptions, lab reports, medical notes)
- Medication and dosage extraction
- Doctor name and patient information detection
- AI chat with Google Gemini integration
- Text-to-speech with elderly-friendly settings
- Speech-to-text for voice input
- EmailJS caregiver alerts with dual alert types
- Care Mode with accessibility features
- 3D heart model visualization
- Comprehensive demo page
- Production-ready fallbacks
- Real SMS integration with Twilio
- User authentication system
- Database integration for data persistence
- Advanced health device integrations
- Multi-language support
- Framework: Next.js 14 with TypeScript
- UI: Tailwind CSS with Framer Motion
- AI: Google Generative AI (Gemini) + MediaPipe AI Models
- OCR: Tesseract.js (client-side processing)
- Health Monitoring: MediaPipe (hands, pose, face mesh, holistic)
- 3D: Three.js with React Three Fiber
- Speech: Web Speech API (native browser)
- Email: EmailJS integration
- Accessibility: WAI-ARIA compliant
caresight/
├── app/
│ ├── api/
│ │ ├── chat/route.ts # AI chat endpoint
│ │ └── upload/route.ts # OCR processing
│ ├── components/
│ │ ├── AdvancedHealthMonitoring.tsx # AI health monitoring
│ │ ├── DocumentScanner.tsx # Enhanced OCR processing
│ │ ├── EmergencyContacts.tsx # Caregiver alerts
│ │ ├── HealthMonitoring.tsx # Medication & metrics
│ │ ├── ThreeScene.tsx # 3D heart model
│ │ └── ...
│ ├── context/
│ │ └── CareModeContext.tsx # Accessibility state
│ ├── chat/page.tsx # AI assistant
│ ├── upload/page.tsx # Document scanner
│ ├── demo/page.tsx # Feature showcase
│ └── dashboard/page.tsx # Health overview
├── public/
│ └── heart.glb # 3D heart model
├── .env.example # Environment template
└── DEPLOYMENT.md # Production guide
Visit the /demo page for an interactive showcase of all features:
- Document scanning simulation
- AI chat examples
- Care mode demonstration
- Emergency alert system
- Voice interaction samples
Required environment variables (all optional with fallbacks):
# AI API (Optional - fallback responses available)
GEMINI_API_KEY=your_gemini_api_key
# EmailJS (Optional - demo mode available)
NEXT_PUBLIC_EMAILJS_SERVICE_ID=your_service_id
NEXT_PUBLIC_EMAILJS_TEMPLATE_ID=your_template_id
NEXT_PUBLIC_EMAILJS_USER_ID=your_user_idnpm install -g vercel
vercel --prodSee DEPLOYMENT.md for comprehensive deployment instructions including:
- Netlify deployment
- Traditional hosting
- Twilio SMS integration
- Production optimization
CareSight is designed with accessibility as a core principle:
- Visual: High contrast mode, large text, clear navigation
- Motor: Large touch targets, simple interactions
- Cognitive: Simplified language, patient voice synthesis
- Auditory: Visual indicators, text alternatives for audio
- Privacy: Client-side OCR processing, no data retention
- Accuracy: Educational information only, provider consultation encouraged
- Accessibility: WCAG 2.1 AA compliant design
- Security: HTTPS required, secure API handling
This project was developed for hackathon purposes and is ready for production deployment. See DEPLOYMENT.md for setup instructions.
This project is built for healthcare accessibility and is available for educational and production use.
CareSight - Making healthcare accessible for everyone! 🏥❤️
Built with ❤️ for elderly, disabled, and neurodiverse users