Multimodal Scout

A smart content discovery platform that automatically finds, curates, and helps you bookmark the latest multimodal AI research papers and industry articles. Built with FastAPI, Next.js, and PostgreSQL, powered by Google Gemini AI.

🌐 Live Demo: https://multimodal-scout.app/

✨ Features

• 🤖 AI-Powered Curation: Auto-discovers and summarizes multimodal AI research and industry content
• 💬 HN Comment Insights: Smart analysis of Hacker News discussions for registered users
• 🔍 Smart Search: Advanced filtering, real-time search, and "Discovery Mode" for exploration
• 📚 Personal Library: Secure bookmarking with editing, export, and management
• 🌙 Modern UI: Clean, responsive interface with dark mode support
• ⚡ Real-time Updates: Live content processing with progress tracking

📚 Documentation

• 📖 User Guide - Complete walkthrough of all features and controls
• 🛠️ Development Guide - Local setup, testing, and workflows
• 🔗 API Reference - Complete REST API documentation
• ⏰ Automation Guide - Pipeline and content processing
• ☁️ Cloud Setup - Google Cloud deployment guide

🚀 Quick Start

Prerequisites

• Docker and Docker Compose
• Google Gemini API key
• Firebase project with Google Sign-In enabled
• Google Cloud CLI (for cloud deployment)

Local Development

1. Clone & Setup:

   git clone https://github.com/yingzha/multimodal-scout.git
   cd multimodal-scout
   
   # Add your Gemini API key and Firebase config to .env

2. Start All Services:

   docker-compose -f docker/docker-compose.yml up -d

   This launches:

   • 🗄️ PostgreSQL (port 5432)
   • 🖥️ Backend API (port 8000)
   • 🌐 Frontend (port 3000)
   • ⏰ Cron Pipeline (every 30 min)

3. Access Applications:

   • Main App: http://localhost:3000
   • API Docs: http://localhost:8000/docs

Production Deployment

1. Prepare Environment:

   # Store secrets in Secret Manager
   echo -n 'YOUR_KEY' | gcloud secrets create google-api-key --data-file=- --project=YOUR_PROJECT_ID
   echo -n 'YOUR_KEY' | gcloud secrets create firebase-api-key --data-file=- --project=YOUR_PROJECT_ID
   
   # Set up infrastructure and database (run once)
   gcloud/setup-infrastructure.sh YOUR_PROJECT_ID us-central1
   gcloud/setup-db-instance.sh YOUR_PROJECT_ID us-central1-a

2. Deploy Services:

   # Build and deploy all services
   gcloud/deploy-services.sh YOUR_PROJECT_ID us-central1

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                            Frontend (Next.js)                           │ 
│                     Real-time UI + Authentication                       │
└─────────────────────────┬───────────────────────────────────────────────┘
                          │ REST API + SSE
                          ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                            Backend (FastAPI)                            │
│                    API Endpoints + Pipeline Logic                       │
└─────┬─────────────────────────────┬─────────────────────────────────────┘
      │                             │
      ▼                             ▼
┌──────────────┐            ┌───────────────────┐
│  PostgreSQL  |            |    Google Gemini  │
│   Database   │            │      AI API       │
│              │            │                   │
│ • Bookmarks  │            │ • Summarization   │
│ • Content    │            │ • Categorization  │
│ • Users      │            │ • Comment Insights│
│ • Cache      │            │ • Smart Filters   │
└──────────────┘            └───────────────────┘

┌─────────────────────────────────────────────────────────────────────────┐
│                         Automated Pipeline                              │
│                        (Every 30 minutes)                               │
│                                                                         │
│ Content Discovery  →  AI Processing  →  Storage & Indexing              │
│                                                                         │
│ • Hacker News         →  • Summarization   →  • PostgreSQL              │
│ • Substack Feeds      →  • Comment Insights →  • Search Embeddings      │
│ • Hugging Face        →  • Categorization  →  • Cache Management        │
│                       →  • Quality Filter  →  • 5-min TTL Caching       │
└─────────────────────────────────────────────────────────────────────────┘

🔧 Configuration

• Local dev: Configure .env with your API keys and Firebase config
• Cloud deployment: Secrets stored in Google Secret Manager, Firebase config derived from project ID

☕ Support

If you find this project helpful, consider buying me a coffee to support continued development!