Skip to content

RegiFamily Shiny for CNN Training program and image process program

Notifications You must be signed in to change notification settings

ahoucbvtw/RegiFamily_shiny

Repository files navigation

RegiFamily Shiny or NotShiny

I was build a simple CNN to identify the Legendary TitansRegirock, Registeel, Regice, Regieleki, Regidrago these 5 pokemons in Switch Game Pokemon Sword and Shield.

Then use this model to encounter their shiny style automatically.

Result_NotShiny:

alt text

Result_Shiny:

alt text

List

Requirements

  • Environment
Version & Specification
OS Windows10
GPU GeForce GTX 1050Ti
NVIDIA Driver 456.71 - WHQL
Tensorflow-GPU 2.3.1
CUDA & cuDNN 10.1 & v7.6.5
  • Necessary for Python Packages
    • tensorflow-gpu 2.3.1

      This Package is must important to use GPU for high performance numerical computation.

      pypi : tensorflow-gpu

      • Install Command : pip install tensorflow-gpu
    • pyserial 3.4

      This Package can control our arduino equipment.

      pypi : pyserial

      • Install Command : pip install pyserial
    • pytesseract 0.3.6

      This Package can let us to Identify the word.

      pypi : pytesseract

      • Install Command : pip install pytesseract
      • After pip pytesseract, go to Github and download/install for your PC. → UB-Mannheim/tesseract
      • Go to installed path to find tesseract.exe. Then add this file's path to your Environment Variable.
    • opencv-python 3.4.11.39

      This Package can let us to video stream from Switch and image process.

      pypi : opencv-python

      • Install Command : pip install opencv-python
    • numpy 1.18.5

      This Package is for image process.

      pypi : numpy

      • Install Command : pip install numpy

Make Training Pictures

  1. 【 Video Capture Device 】

    I use HDMI→USB video capture device as my Switch's video input.

    Video Capture Card

  2. 【 Record videos : Video_save.py

    I use Video Capture Device as my Switch's video input, and use cv2.VideoWriter_fourcc to save videos.

    Because of I saved the video as MP4, openh264-1.7.0-win64.dll this file was necessary.

    openh264 Download here

  3. 【 Save Pictures from videos : Pictures_save.py

    Set how frequency to get pictures and take frames to numpy array from videos.

    According to I discovered the cave's light (background) was different in morning and night, I decided to gather all the morning and night pictures to enhance my identity rate.

    alt text

    • Shiny RegiFamily

      Because of I never encounter the shiny RegiFamily, I found the shiny RegiFamily from YouTube as my Training pictures.

    • NotShiny RegiFamily

      I recorded all the RegiFamily normal color with Video Capture Device morning and night the time in the game.

      Then the frame was setted 1 when I was saved training pictures.

Training

I build two models and the Dence layers was the same but convolutional layers was different for training.

First was use Transfer Learning (VGG16), second was slef-build convolutional layers. Use these two models to compare witch one is more simple, fast, high accuracy.

According to the result, the VGG16 is the best model in this project.

Why not try more other famous models? The reason is use VGG16 to identify this project is enough. In actual fact, its identity rate was vary high even closed to 100% when I was used.

Regirock training loss & accuracy :

alt text alt text

Regice training loss & accuracy :

alt text alt text

Registeel training loss & accuracy :

alt text alt text

Regieleki training loss & accuracy :

alt text alt text

Regidrago training loss & accuracy :

alt text alt text

But Regice and Regieleki are different. Other RegiFamily's shiny type has large of difference from normal type, only Regice and Regieleki there two of types are vary close, even human eyes can not identify without any information.

alt text

Thus, I use image process to catch definitely appeared the shiny light when encountered shiny Pokemon. shiny_detect.py

I use cv2.inRange to catch the ShinyLight, and use cv2.dilate to make my target more cleary, then use cv2.findContours to find the contour point and count it.

If the ShinyLight contour point is more than 6 even 8, it must be encounterd the shiny Regice certainly.

m1 = cv2.inRange(m1, (240,240,240),(255,255,255))
m1 = cv2.dilate(m1, np.ones((30,23)))
z, a, b = cv2.findContours(m1, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)

alt text alt text

alt text

Demo Video

  • CNN (VGG16)

    • Regirock Auto Shiny Farming :

      d

    • Registeel Auto Shiny Farming :

      d

  • Image Process (OpenCV)

    • Regice Auto Shiny Farming :

      d