AI Image Search Application

An intelligent image search system powered by CLIP and FaceNet models

🎯 Overview

This application allows you to search through your image collection using natural language queries like "Neelam giving speech" or "beach picnic". It combines the power of OpenAI's CLIP model for semantic image understanding and FaceNet for face recognition.

✨ Key Features

• 🔍 Natural Language Search: Search images using descriptive text queries
• 👤 Face Recognition: Identify specific people in your image collection
• 🖼️ Multiple Format Support: JPG, PNG, HEIC, HEIF, WebP, BMP, TIFF
• 📱 HEIC Support: Full support for Apple's HEIC format with automatic fallbacks
• 🎛️ Adjustable Parameters: Fine-tune search sensitivity and result count
• 📱 Responsive UI: Modern web interface that works on desktop and mobile
• ⚡ Real-time Results: Fast search with similarity scoring

🏗️ Architecture

• Frontend: Next.js 15 with React 19 and Tailwind CSS
• Backend: FastAPI with Python
• AI Models:
  • CLIP (ViT-L/14@336px) for text-to-image similarity
  • MTCNN for face detection
  • FaceNet (InceptionResnetV1) for face recognition
• Image Processing: PIL/Pillow with HEIC support via pillow-heif

📋 Prerequisites

• Python 3.8+ with pip
• Node.js 16+ with npm
• Git for cloning the repository
• 4GB+ RAM (recommended for model loading)

🚀 Quick Start

1. Clone the Repository

git clone https://github.com/your-username/ai_image_search.git
cd ai_image_search

2. Complete Setup (Recommended)

Run the automated setup script that installs all dependencies and creates sample data:

python scripts/quick-setup.py

3. Install HEIC Support (Optional but Recommended)

For full HEIC format support:

python scripts/install-heic-support.py

4. Start the Backend Server

python scripts/run-backend.py

You should see:

✅ All models loaded successfully!
✅ HEIC support available via: pillow-heif (recommended)
✅ Generated embeddings for X images
✅ Generated embeddings for X known faces
INFO: Uvicorn running on http://0.0.0.0:8000

5. Start the Frontend (New Terminal)

# Option 1: Use setup script (handles common issues)
python scripts/setup-frontend.py

# Option 2: Manual installation
npm install --legacy-peer-deps
npm run dev

# Option 3: Windows batch file
scripts\setup-frontend.bat

6. Access the Application

• Main Application: http://localhost:3000
• API Documentation: http://localhost:8000/docs
• Supported Formats: http://localhost:8000/supported-formats

📁 Project Structure

ai-image-search/
├── **Images**/                     # Your searchable image collection
│   ├── events/
│   ├── personal/
│   └── work/
├── **known_faces**/               # Reference photos for face recognition
│   ├── person1.jpg
│   └── person2.jpg
├── backend/
│   └── main.py               # FastAPI backend server
├── app/
│   ├── page.tsx              # Main React component
│   └── layout.tsx            # App layout
├── components/ui/            # Reusable UI components
├── scripts/                  # Setup and utility scripts
│   ├── quick-setup.py        # Complete automated setup
│   ├── install-heic-support.py  # HEIC format support
│   ├── run-backend.py        # Start backend server
│   ├── setup-frontend.py     # Frontend setup helper
│   └── create-folders.py     # Create directory structure
└── README.md

🖼️ Supported Image Formats

Fully Supported:

• JPG/JPEG - Standard photo format
• PNG - Lossless image format
• WebP - Modern web format
• BMP - Bitmap format
• TIFF - High-quality format

HEIC/HEIF Support:

• HEIC - Apple's modern format (iPhone photos)
• HEIF - High Efficiency Image Format

The system automatically detects and uses the best available HEIC decoder:

1. pillow-heif (recommended) - Full HEIC support
2. pyheif (fallback) - Alternative HEIC library
3. opencv-python (fallback) - Basic image processing

🖼️ Adding Your Images

For Searchable Images:

1. Place images in the Images/ folder
2. Organize in subfolders (optional): Images/events/, Images/personal/, etc.
3. All formats supported: JPG, PNG, HEIC, HEIF, WebP, BMP, TIFF

For Face Recognition:

1. Place reference photos in the known_faces/ folder
2. Use descriptive filenames: john_doe.jpg, jane_smith.png
3. Each image should contain a clear face

Refresh Embeddings:

After adding new images, click the "Refresh Embeddings" button in the web interface or restart the backend server.

🔍 Usage Examples

Search Queries to Try:

• "John giving speech" - Find images of a specific person doing an activity
• "beach picnic" - Find images of outdoor dining scenes
• "team meeting" - Find group meeting photos
• "office workspace" - Find workplace environments
• "mountain hiking" - Find outdoor adventure photos
• "family dinner" - Find dining/family gathering photos

Search Parameters:

• Max Results: 1-10 (default: 5)
• Similarity Threshold: 0.0-1.0 (default: 0.2)
  • Lower values = more results, less strict matching
  • Higher values = fewer results, stricter matching

🛠️ Manual Setup (Alternative)

If the quick setup doesn't work, follow these manual steps:

Backend Setup:

# Install Python dependencies
pip install fastapi uvicorn python-multipart pillow torch torchvision tqdm numpy
pip install git+https://github.com/openai/CLIP.git
pip install facenet-pytorch

# Install HEIC support (optional)
pip install pillow-heif pyheif opencv-python

# Create folders
mkdir Images known_faces backend

# Start backend
python backend/main.py

Frontend Setup:

# Install Node.js dependencies
npm cache clean --force
npm install --legacy-peer-deps

# Start development server
npm run dev

🐛 Troubleshooting

Common Issues:

Backend Issues:

• "No module named uvicorn": Run pip install uvicorn fastapi
• CUDA out of memory: The app will automatically use CPU if GPU memory is insufficient
• No images found: Make sure images are in the Images/ folder with supported formats
• HEIC not working: Run python scripts/install-heic-support.py

HEIC-Specific Issues:

• HEIC images not loading: Install pillow-heif: pip install pillow-heif
• HEIC conversion errors: Try alternative: pip install pyheif
• macOS HEIC issues: Ensure Xcode command line tools are installed

Frontend Issues:

• "next is not recognized": Run npm install next react react-dom
• PowerShell execution policy: Use Command Prompt instead of PowerShell, or run Set-ExecutionPolicy RemoteSigned
• npm install fails: Try npm install --legacy-peer-deps or npm install --force

Connection Issues:

• Backend offline: Make sure backend is running on port 8000
• CORS errors: Backend allows localhost:3000 by default
• Port conflicts: Change ports in the configuration if needed

Getting Help:

1. Check the browser console for frontend errors
2. Check the terminal running the backend for Python errors
3. Visit http://localhost:8000/docs to test the API directly
4. Test HEIC support: http://localhost:8000/supported-formats
5. Ensure both servers are running simultaneously

🔧 Configuration

Environment Variables:

The application works out of the box, but you can customize:

• Backend Port: Modify port=8000 in backend/main.py
• Frontend Port: Modify next.config.mjs or use npm run dev -- -p 3001
• Model Device: Automatically detects CUDA/CPU, or set device = "cpu" in backend/main.py

Model Configuration:

• CLIP Model: Currently uses ViT-L/14@336px
• Face Detection: MTCNN with 160px image size
• Face Recognition: InceptionResnetV1 pretrained on VGGFace2

HEIC Configuration:

The system automatically detects and uses available HEIC libraries in this order:

1. pillow-heif (best performance and compatibility)
2. pyheif (alternative implementation)
3. opencv-python (basic fallback)

📊 Performance

Model Loading Times:

• First startup: 30-60 seconds (downloading models)
• Subsequent startups: 5-10 seconds
• Embedding generation: ~1-2 seconds per image
• HEIC processing: ~2-3 seconds per image (first time)

Hardware Requirements:

• Minimum: 4GB RAM, CPU-only
• Recommended: 8GB+ RAM, NVIDIA GPU with 4GB+ VRAM
• Storage: ~2GB for models + your image collection

Format Performance:

• JPG/PNG: Fastest processing
• HEIC/HEIF: Slightly slower due to conversion
• WebP/TIFF: Standard processing speed

🤝 Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make your changes and test thoroughly
4. Commit your changes: git commit -m 'Add feature'
5. Push to the branch: git push origin feature-name
6. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• OpenAI CLIP for semantic image understanding
• FaceNet PyTorch for face recognition
• pillow-heif for HEIC support
• FastAPI for the backend framework
• Next.js for the frontend framework
• v0.dev for rapid UI development

📞 Support

If you encounter any issues:

1. Check the troubleshooting section above
2. Review the logs in your terminal
3. Test the API directly at http://localhost:8000/docs
4. Check HEIC support at http://localhost:8000/supported-formats
5. Create an issue on GitHub with:
   • Your operating system
   • Python and Node.js versions
   • Image formats you're trying to use
   • Complete error messages
   • Steps to reproduce the issue

---

Happy searching! 🔍✨