An interactive English learning platform powered by OpenAI's Realtime API and Agents SDK, designed to provide immersive oral and spoken English practice through AI-powered conversation scenarios and listening comprehension exercises.
- Real-time voice conversations with AI tutors in authentic campus scenarios
- 30-second practice sessions with immediate feedback and scoring
- Multiple conversation contexts including:
- Library study buddy planning
- Dorm move-in welcome conversations
- Club fair exploration and networking
- Dining hall casual chats
- Roommate harmony discussions
- Immersive listening scenarios with realistic campus announcements and briefings
- Interactive quiz questions to test comprehension
- Visual context with campus images to enhance learning
- Multiple difficulty levels covering:
- University orientation briefings
- Library workshop announcements
- Career center internship preparation
- Campus events roundups
- Academic advising reminders
- Voice Activity Detection (VAD) for natural conversation flow
- Real-time audio streaming with low latency
- Intelligent feedback system with scoring and improvement tips
- Multiple voice options (Alloy, Verse) for varied learning experiences
- Comprehensive evaluation after each practice session
- Node.js 18+
- OpenAI API key
- Modern web browser with WebRTC support
-
Clone the repository
git clone <repository-url> cd 51EnglishTutor
-
Install dependencies
npm install
-
Set up environment variables
# Copy the sample environment file cp .env.sample .env # Add your OpenAI API key echo "OPENAI_API_KEY=your_api_key_here" >> .env
-
Start the development server
npm run dev
-
Open your browser Navigate to http://localhost:3000
- Context: Planning a study session with a classmate
- Skills: Polite requests, time coordination, study preferences
- Duration: 30-second practice sessions
- Voice: Verse (conversational, friendly)
- Context: Greeting first-year students during move-in day
- Skills: Small talk, introductions, dorm life discussions
- Duration: 30-second practice sessions
- Voice: Verse (welcoming, mentor-like)
- Context: Exploring campus organizations and interests
- Skills: Interest expression, organization inquiries, networking
- Duration: 30-second practice sessions
- Voice: Verse (guide-like, enthusiastic)
- Context: Casual conversation while waiting in line
- Skills: Food discussions, weekend plans, casual small talk
- Duration: 30-second practice sessions
- Voice: Verse (casual, friendly)
- Context: Discussing routines and boundaries with roommates
- Skills: Collaborative communication, conflict resolution, negotiation
- Duration: 30-second practice sessions
- Voice: Verse (mediator-like, understanding)
- Content: Campus orientation events and procedures
- Quiz Focus: Check-in locations, event timing, resource access
- Duration: ~18 seconds of audio + quiz time
- Voice: Alloy (clear, informative)
- Content: Research skills workshop announcements
- Quiz Focus: Registration procedures, workshop details, timing
- Duration: ~18 seconds of audio + quiz time
- Voice: Alloy (professional, clear)
- Content: Internship preparation series information
- Quiz Focus: Session topics, scheduling, target audience
- Duration: ~18 seconds of audio + quiz time
- Voice: Alloy (motivational, clear)
- Content: Weekend campus events and activities
- Quiz Focus: Event timing, locations, activity details
- Duration: ~18 seconds of audio + quiz time
- Voice: Alloy (engaging, informative)
- Content: Course planning meeting instructions
- Quiz Focus: Deadlines, locations, required materials
- Duration: ~18 seconds of audio + quiz time
- Voice: Alloy (professional, clear)
- Frontend: Next.js 15 with React 19
- AI Integration: OpenAI Realtime API + Agents SDK
- Audio Processing: WebRTC for real-time audio streaming
- Styling: Tailwind CSS for responsive design
- TypeScript: Full type safety throughout the application
// Example: Oral Conversation Agent
new RealtimeAgent({
name: 'oral_library_study_buddy',
voice: 'verse',
instructions: 'You are meeting a classmate in the library...',
handoffs: [],
tools: [],
handoffDescription: 'Library study planning partner',
})- Quiz Generation: Dynamic multiple-choice questions
- Audio Timing: Precise control over when questions appear
- Visual Context: Campus images to enhance comprehension
- Immediate Feedback: Real-time answer evaluation
- Voice Activity Detection: Automatic speech recognition
- Codec Support: Wide-band Opus (48kHz) and narrow-band PCMU/PCMA (8kHz)
- Low Latency Streaming: Optimized for natural conversation flow
- Audio Download: Session recording capabilities
- Scenario Selection: Dropdown menus for choosing learning contexts
- Real-time Transcript: Live conversation display with message history
- Event Logging: Comprehensive activity tracking and debugging
- Audio Controls: Playback, recording, and voice activity settings
- Progress Tracking: Visual feedback on quiz performance
- Mobile Optimized: Touch-friendly interface for all devices
- Accessibility: Screen reader support and keyboard navigation
- Visual Feedback: Clear indicators for conversation states and quiz results
OPENAI_API_KEY=your_openai_api_key_here- Voice Selection: Choose between Alloy and Verse voices
- Session Duration: Adjustable practice session lengths
- Quiz Timing: Configurable delay before showing questions
- Audio Quality: Toggle between high and low bandwidth codecs
- Warm-up: Brief scenario introduction
- Practice Session: 30-second focused conversation
- Evaluation: Immediate scoring and feedback
- Improvement Tips: Specific suggestions for enhancement
- Context Setup: Visual and audio introduction
- Listening Phase: Focused audio content delivery
- Quiz Assessment: Multiple-choice comprehension check
- Feedback: Immediate correct/incorrect response with explanations
npm run build
npm start- Ensure all environment variables are properly configured
- Set up HTTPS for WebRTC functionality in production
- Configure appropriate CORS settings for your domain
We welcome contributions to improve the English learning experience! Please:
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests if applicable
- Submit a pull request
- Follow TypeScript best practices
- Maintain accessibility standards
- Test across different browsers and devices
- Update documentation for new features
This project is licensed under the MIT License - see the LICENSE file for details.
For questions, issues, or feature requests:
- Open an issue on GitHub
- Check the documentation for common solutions
- Review the troubleshooting section below
- Ensure your browser supports WebRTC
- Check microphone permissions
- Verify your OpenAI API key is valid
- Check your internet connection
- Verify firewall settings allow WebRTC traffic
- Try refreshing the page
- Close other browser tabs to free up resources
- Check your browser's audio processing capabilities
- Ensure you're using a modern browser version
Built with ❤️ for English learners worldwide
Transform your English speaking and listening skills through immersive, AI-powered conversations and exercises designed for real-world success.