PokeFace

Find your Pokémon match by analyzing facial features. This project uses computer vision and machine learning to identify which Pokémon you look most like based on facial structure, expressions, and visual characteristics.

Technical Overview

PokeFace combines web scraping, computer vision, and transformer models to create a face-matching system. The pipeline scrapes Pokémon data and images, extracts visual features using CLIP embeddings, and compares them with human facial features to find matches.

Interesting Techniques

• CLIP (Contrastive Language-Image Pre-training) integration using Transformers for multi-modal feature extraction
• Face detection with OpenCV's Haar Cascades
• Web scraping with respectful rate limiting using BeautifulSoup and requests
• Weighted similarity scoring algorithm that considers confidence levels and feature importance
• Streaming downloads with chunked file processing for efficient image handling

Technologies and Libraries

• OpenAI CLIP - Vision transformer for understanding images and text together
• PyTorch - Deep learning framework powering the CLIP model
• OpenCV - Computer vision library for face detection and image processing
• Transformers - Hugging Face library providing pre-trained CLIP models
• BeautifulSoup4 - HTML parsing for web scraping
• Pillow - Image processing and format conversion

Project Structure

pokeface/
├── images/                    # Downloaded Pokémon sprites
├── extract_features.py
├── match_face.py
├── pokemon_data.json
├── pokemon_features.json
├── requirements.txt
└── scrape_pokemon.py

images/ - Contains scraped Pokémon sprite images organized by Pokédex number and name. Images are downloaded directly from PokemonDB during the scraping process.

The project follows a three-stage pipeline: scrape_pokemon.py gathers data and images, extract_features.py processes Pokémon images to create feature vectors, and match_face.py analyzes user photos to find similar Pokémon matches.