AI PHYSICAL CHESS -STILL IN PROGRESS ♟️ 👁️

An AI agent you can play with, physically, using computer vision techniques. > Live demo here.

Table of Contents 🔮

• General Info
• Technologies Used
• Setup
• Project Status
• Room for Improvement
• Contact

General Information 🖥️

The project has 2 main parts.

First is the Computer Vision part. That part is in charge of understanding the given physical board, live on camera. It identifies the board, the 64 squares of it, and the chess pieces' structure.

Second is the AI agent part. That part is in charge of generating the moves. It gets the current position from the Computer Vision part and generates what it believes to be the best move.

---

The two parts combined -allow the user to play an AI agent, on a physical board. That's a combination that one doesn't see regularly, as most AI agents can be played against only through a digital board.

The project has an option of playing against Stockfish-15 as well.

Technologies Used 💡

• 👁️ Computer Vision part -

  Python, Open CV, Keras, VGG19 model, MediaPipe.

• 🤖 AI agent part -

  Python, python-chess library, AI algorithms

game process

- project structure

Computer Vision part:

• collect Data from physical board.

🔎 -uploading images of boards with chess pieces on it.

🔎 -resizing images.

🔎 -converting images to a gray and blurred version - to get some image noise out for a better later edge detection.

🔎 -edge detection.

🔎 -straight lines detection, while classifying to HORIZONTAL and VERTICAL.

🔎 -detecting lines that couldn't be found, and adding them.

🔎 -finding lines intersection to determine the 64 squares.

🔎 -converting each 4 points that represent a square into an equilateral square.

🔎 -manually determines the piece on the square and classifies it to the relevant class file. 70% train, 15% validation, and another 15% for the test class.

• train VGG19 model.

🧰 -use the data collected in the previous phase to train, an additional last layer of a VGG19 model.

🧰 -validate and test VGG19.

• activate VGG19 model to determine pieces's structure in a given position

AI agent part:

• evaluation functions. -A batch of evaluation functions to determine the 'value' of a given chess position.

The valuation functions represent :

♟️-location of pieces on board in a generic manner.

♟️-piece capture.

♟️-space ( attacked squers ) occupied.

♟️-pawn structure.

♟️-proximity to the enemy's king

♟️-attack squares in relation to defendor squares.

♟️-castle.

♟️-left pieces with respect to strength.

♟️-bishop protector.

♟️-connected rooks.

Its important to mention that these evaluation functions are only a fractionfrom possible evaluation functions.

• AI algorithms- using the evaluation functions and determining the best move. depth 3.

🎲 -MINIMAX wikipedia

🎲 -ALPHA BETA PRUNING wikipedia

🎲 -QUIESCENCE SEARCH wikipedia

🎲 -GENETIC ALGORITHM wikipedia

• Opening Theory Book - pre made decision tree for the opening phase.

📓 -the AI agent uses that database of moves as long as possible.

• Stockfish-15

🐠 -The AI agent trains over Stockfish-15 in the GENETIC ALGORITHM

Setup 🛠️

• computer's camera

• download all files and change 'path's in code accordingly.

• non-trivial libraries needed:

    import cv2
    import numpy as np
    from tensorflow import keras
    from keras import Model
    from keras.applications import VGG19
    from keras.callbacks import EarlyStopping
    from keras.layers import Flatten, Dense
    from keras.preprocessing.image import ImageDataGenerator
    from matplotlib import pyplot as plt
    from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
    
    import chess.engine
    import mediapipe as mp
  

Files 📂

• create_data.py ->

creates the data using computer vision techniques. classifies squares to differentclasse folders.

• VGG-19_train ->

trains the last dense of VGG-19 on 'create_data.py' data

• VGG-19_activate ->

classifies 64 squares into a board representation.

• engin ->

holds evaluation functions and AI algorithms.

There is an option for activate only that part ( without the computer vision part)

• main ->

holds the root code that operates the whole orchestra.

• stockfish_15....zip ->

holds the 'brain' of stockfish

• VGGMODEL.h5 ->

holds the saved VGG-19 model, trained with our dataset. saves resources as we dont need to train the model all over again each time.

• test, train, val ->

each file holds the classes of the pieces with manually classified images( squers)

• computer_data.bin ->

holds the 'Opening theory book' represented by a binary format.

Project Status 🥑

Project is almost finished - small improvements need to be made for a smooth user experience,

As well as there are some changes that need to made for a cleaner code and better usage of raw materials

Room for Improvement 🏋️‍♂️

Its important to mention that the project is a prototype, and still have a lot of room for improvement.

Room for improvement in Computer Vision part:

• train VGG-19 over more images for better accuracy.

• train VGG-19 over more chess boards and pieces sets.

• explore a largerr range of shooting angles.

• increase accuracy of board detection

• add statistical approach to allow a better piece recognition

Room for improvement in AI agent part:

• code more evaluation functions

• run longer and better GENETIC ALGORITHM

• train GENETIC ALGORITHM on human players as well as on various AI agents

• use Neural Networks approach

• make the structure faster by using state of the art approaches.

👤 Add a robotic hand that will enable a more realistic game against the AI agent.

Contact 📧

Created by @Itamar - feel free to contact me!