TopNotesAI - Production Ready

Convert YouTube educational lectures into HUMAN-STYLE student notes.

🎯 Production Optimizations

Free-Tier Optimized

• ✅ Downloads only first 10 minutes of content
• ✅ Extracts maximum 8 frames using scene detection
• ✅ Sequential AI calls (no parallel requests)
• ✅ Automatic retries with exponential backoff
• ✅ Rate limiting (max 3 concurrent jobs)
• ✅ Auto-cleanup of old jobs (1 hour expiry)

Reliability Features

• ✅ Background job processing (no timeouts)
• ✅ Real-time progress tracking
• ✅ Graceful degradation (continues without diagrams/frames if they fail)
• ✅ Comprehensive error handling with user-friendly messages
• ✅ Fallback notes structure if AI fails
• ✅ Automatic resource cleanup

📋 Prerequisites

1. Node.js (v18 or higher) - Download
2. Groq API Key (FREE) - Get one here

Note: FFmpeg and yt-dlp are included automatically via npm packages!

🚀 Setup Instructions

1. Install Dependencies

npm install

2. Configure Environment

Create a .env file in the root directory:

PORT=5000
GROQ_API_KEY=your_groq_api_key_here

3. Run the Application

npm start

Open your browser to: http://localhost:5000

🏗️ Architecture

Backend Flow

User submits URL
    ↓
Job created (returns immediately)
    ↓
Background Processing:
  1. Download first 10min audio + video (360p)
  2. Transcribe audio (Whisper-v3)
  3. Analyze up to 3 frames (Vision AI)
  4. Generate notes (Llama-3.3-70b)
  5. Create diagrams (Mermaid → SVG)
  6. Render PDF (Puppeteer)
    ↓
User downloads PDF
    ↓
Auto-cleanup after 10s

API Endpoints

• POST /api/generate-notes - Start job (returns jobId)
• GET /api/status/:jobId - Check progress
• GET /api/download/:jobId - Download PDF
• GET /health - Server health check

🛡️ Error Handling

Graceful Failures

• No audio detected → Returns transcript-only notes
• Frame extraction fails → Continues with audio analysis
• Diagram generation fails → Skips diagrams, continues PDF
• AI timeout → Automatic retry (3 attempts)
• PDF generation fails → Clear error message to user

Rate Limiting

• Maximum 3 concurrent jobs
• Jobs expire after 1 hour
• Auto-cleanup every 15 minutes

📊 Free-Tier Limits

Groq API (FREE Tier)

• Whisper: 30 requests/min
• Vision: 30 requests/min
• LLM: 30 requests/min

Our Optimizations

• Sequential processing (never parallel)
• 1 second delay between frame analyses
• Exponential backoff on retries
• Maximum 3 frames analyzed per video
• Maximum 3 diagrams generated

🐛 Troubleshooting

"Connection error" or "ECONNRESET"

• Cause: Temporary network issue with Groq API
• Solution: The app automatically retries 3 times. If it persists, wait 1 minute and try again.

"Server is busy"

• Cause: 3+ jobs already running (free-tier protection)
• Solution: Wait 1-2 minutes for current jobs to complete.

"Job not found"

• Cause: Job expired (>1 hour old)
• Solution: Submit a new request.

PDF not downloading

• Cause: Browser popup blocker
• Solution: Allow popups for localhost:5000

📁 Project Structure

topnotesai/
├── frontend/
│   ├── index.html          # UI
│   ├── app.js              # Job submission & polling
│   └── assets/
├── backend/
│   ├── server.js           # Express server
│   ├── routes/
│   │   └── notes.js        # Job orchestration
│   ├── services/
│   │   ├── videoProcessor.js   # yt-dlp + FFmpeg
│   │   ├── aiAnalyzer.js       # Groq API (sequential)
│   │   ├── diagramGenerator.js # Mermaid CLI
│   │   └── pdfGenerator.js     # Puppeteer
│   ├── templates/
│   │   └── notes-template.html # PDF layout
│   └── temp/               # Auto-cleaned
├── package.json
├── .env
└── README.md

🎨 Features

Human-Style Notes

• Active recall headings (question format)
• Marginalia with "Aha!" moments
• Emphasis markers (underline, circle, bold)
• Cheat sheet with key formulas
• Golden rule summary

Technical Excellence

• SVG diagrams (crisp at any zoom level)
• 2-column PDF layout
• Google Fonts (Inter + Caveat)
• Glassmorphism UI
• Real-time progress updates

📝 Development

Run in Development Mode

npm run dev

Environment Variables

PORT=5000                    # Server port
GROQ_API_KEY=gsk_xxx        # Your Groq API key
NODE_ENV=production         # Optional

🚢 Deployment

Free-Tier Hosting Options

• Backend: Render, Railway, Fly.io
• Frontend: Vercel, Netlify

Important Notes

1. Set environment variables on your hosting platform
2. Ensure NODE_ENV=production
3. Free-tier servers may sleep after inactivity (first request slower)

📄 License

MIT License - Feel free to use and modify!

🙏 Credits

• Groq for free-tier AI APIs
• yt-dlp for YouTube downloading
• FFmpeg for video processing
• Mermaid.js for diagram generation
• Puppeteer for PDF rendering

---

Made with ❤️ for students who want better notes