Poké-Pi-Dex

Poké-Pi-Dex is our deep learning / computer vision related project for nostalgic poké-weebs.

We recreated a Pokédex clone, which recognizes pictures of Pokémon from the first generation, using a Convolutional Neural Network. It's built on Raspberry Pi4 with LCD display, PiCamera, speaker and some other components attached.
The case is made of recycled cardboard. 🌱

View Report · Presentation · Project Work · Request Feature|Report Bug · Italiano

Table of Contents

1. Demo
2. Features
3. Usage
   • Prerequisites
   • Installation
4. Tools
5. Resources
6. Roadmap
7. License
8. Contacts

Demo

Watch our demo on YouTube!

Demo video

Features

• Main menu with functionalities and About panel.
• Pokémon-image acquisition from app.
• Pokémon prediction of captured pic, with an excellent accuracy, using a Convolutional Neural Network.
• Different objects recognition, such as cards, plushies, figures.
• Pokémon info (name, id, types, description and stats) and the previous and following evolution stages of Pokémon recognized.
• Audio description of the predicted Pokémon.
• Play Pokémon cry.
• Settings view where users can edit parameters, such as language (English and Italian), fullscreen, volume.
• Debug mode to check the values of the prediction.
• Easter egg, try and find it!

Usage

To use the application follow these steps: TO-DO

Prerequisites

TO-DO

Installation

TO-DO

Tools

• PyCharm
• Anaconda
• Docker
• Jupyter Notebook
• Sketchup browser

Resources

• Pokémon data:
  • Old Dataset
  • Our New Dataset
  • Details
  • Cry audio files
  • Description audio files
• Camera calibration with OpenCV
• Tensorflow-lite inference
• Tensorflow-lite conversion
• Tensorflow-lite post-training quantization

Roadmap

• Dataset
  • find a decent dataset for the neural network
  • fix (cut pictures) and extend it
• Classifier
  • CNN with 3 conv layers and 2 FC layers
  • data augmentation (random flip, rotation, contrast, brightness)
  • try dropout
  • try batch norm
  • loss and accuracy graphs
  • test real life pics
  • improve old CNN
• Application
  • Pokémon repository
    • find .json file and load it into a dictionary
    • check and fix it
    • create Pokémon class
  • video input
    • create a separate class
    • make a function that can take a frame from the picamera (to test)
    • display the image inside a canvas
  • GUI structure
    • create a main menu
    • create an about panel
    • create a main app view divided in 2 (left-side for video input, right-side for Pokémon details)
    • create a settings panel
  • button to get the current frame
  • labels and entry for the Pokémon details (stats with dynamic bars and different colors)
  • add buttons to scroll between multiple evolutions (example: Eevee has different evolutions)
  • change the "type(s)" entry (from text to image)
  • button to play cry
    • collect cry audio files
  • description voice reading
    • collect description audio files with a bot
  • different language update
  • debug mode
• Raspberry Setup
  • buy components
    • LDC display
    • PiCamera
    • power supply (powerbank)
    • speaker
    • push buttons
    • type-C elbow adapter
    • A/D converter (ADS1115)
  • integrate components
    • LCD display
    • PiCamera
    • power supply
    • speaker
    • buttons
    • analog joystick
  • prepare OS (disable password, enable interfaces, ...)
• App Deployment
  • prepare environment (install python3 and required packages)
  • clone the repo
  • test the app
• Case Prototype
  • project
  • cardboard cutout
  • painting
• Report
  • setup a LaTeX document
  • draft a possible subdivision into chapters
  • write the report
• Presentation
• Demo Video
• Extra & Future Developments
  • use a more comples neural network with the new dataset
  • use new form of data augmentation
  • add an amplifier to speaker
  • insert one or more white LEDs near the camera lens
  • add settings option to enable/disable the video freezing after taking a picture
  • finish the 3D model and print it
  • extend the Pokédex to the following Pokémon generations
  • porting of the application to mobile systems (Android, iOS) - Flutter + Firebase

License

Distributed under the GPLv3 License. See LICENSE for more information.

Contacts

• TryKatChup
• Mikyll