This repository contains the experiments made to analyze and complete assignment 4 of the CS7641 - Machine Learning course from the Georgia Institute of Technology.
The main purpose of this assignment is to run & analyze 4 reinforcement learning algorithms:
- Value Iteration
- Policy Iteration
- Q-Learning
when ran to solve 2 Markov Decision Processes:
- Cat & Mouse Maze
- Flip Flop
We use the custom classes included in utils package to run these experiments.
The algorithms.py file include the classes that define the algorithms to use.
The mdp.py file defines the MDPs that the algorithms will use.
In the root directory of the project, do one of the following two
$ conda create --name myenv python=3.8
$ conda activate myenv
$ pip install -r requirements.txt$ python -m venv /venv
$ source venv/bin/activate
$ pip install -r requirements.txtOpen up main.py in VS Code (or your preferred editor), select your virtual environment (in this example myenv) run:
$ python3 main.pyYou should see an output like this if you leave everything as default in the codebase:
Running Maze Experiments
3 traps set
M . . .
. T . T
X . . .
. . T C
Running Value Iteration
Running Policy Iteration
Running Q-Learning
100%|████████████████| 5000/5000 [00:00<00:00, 28895.39it/s]
Running Flip Flop Experiments
Running Value Iteration
Running Policy Iteration
100%|████████████████| 10000/10000 [00:01<00:00, 9652.16it/s]
Optimal Policy (VI): {'000000000': 0, '000000001': 0, '000000010': 0, '000000011': 0, '000000100': 0, '000000101': 0, '000000110': 0, '000000111': 0, '000001000': 0, '000001001': 0, '000001010': 0, '000001011': 0, '000001100': 0, '000001101': 0, '000001110': 0, '000001111': 0, '000010000': 0, '000010001': 0, '000010010': 0, '000010011': 0, '000010100': 0, '000010101': 0, '000010110': 0, '000010111': 0, '000011000': 0, '000011001': 0, '000011010': 0, '000011011': 0, '000011100': 0, '000011101': 0, '000011110': 0, '000011111': 0, '000100000': 0, '000100001': 0, '000100010': 0, '000100011': 0, '000100100': 0, '000100101': 0, '000100110': 0, '000100111': 0, '000101000': 0, '000101001': 0, '000101010': 0, '000101011': 0, '000101100': 0, '000101101': 0, '000101110': 0, '000101111': 0, '000110000': 0, '000110001': 0, '000110010': 0, '000110011': 0, '000110100': 0, '000110101': 0, '000110110': 0, '000110111': 0, '000111000': 0, '000111001': 0, '000111010': 0, '000111011': 0, '000111100': 0, '000111101': 0, '000111110': 0, '000111111': 0, '001000000': 0, '001000001': 0, '001000010': 0, '001000011': 0, '001000100': 0, '001000101': 0, '001000110': 0, '001000111': 0, '001001000': 0, '001001001': 0, '001001010': 0, '001001011': 0, '001001100': 0, '001001101': 0, '001001110': 0, '001001111': 0, '001010000': 0, '001010001': 0, '001010010': 0, '001010011': 0, '001010100': 0, '001010101': 0, '001010110': 0, '001010111': 0, '001011000': 0, '001011001': 0, '001011010': 0, '001011011': 0, '001011100': 0, '001011101': 0, '001011110': 0, '001011111': 0, '001100000': 0, '001100001': 0, '001100010': 0, '001100011': 0, '001100100': 0, '001100101': 0, '001100110': 0, '001100111': 0, '001101000': 0, '001101001': 0, '001101010': 0, '001101011': 0, '001101100': 0, '001101101': 0, '001101110': 0, '001101111': 0, '001110000': 0, '001110001': 0, '001110010': 0, '001110011': 0, '001110100': 0, '001110101': 0, '001110110': 0, '001110111': 0, '001111000': 0, '001111001': 0, '001111010': 0, '001111011': 0, '001111100': 0, '001111101': 0, '001111110': 0, '001111111': 0, '010000000': 2, '010000001': 2, '010000010': 2, '010000011': 2, '010000100': 2, '010000101': 2, '010000110': 2, '010000111': 2, '010001000': 2, '010001001': 2, '010001010': 2, '010001011': 2, '010001100': 2, '010001101': 2, '010001110': 2, '010001111': 2, '010010000': 2, '010010001': 2, '010010010': 2, '010010011': 2, '010010100': 2, '010010101': 2, '010010110': 2, '010010111': 2, '010011000': 2, '010011001': 2, '010011010': 2, '010011011': 2, '010011100': 2, '010011101': 2, '010011110': 2, '010011111': 2, '010100000': 4, '010100001': 4, '010100010': 4, '010100011': 4, '010100100': 4, '010100101': 4, '010100110': 4, '010100111': 4, '010101000': 6, '010101001': 6, '010101010': 0, '010101011': 7, '010101100': 5, '010101101': 5, '010101110': 5, '010101111': 5, '010110000': 3, '010110001': 3, '010110010': 3, '010110011': 3, '010110100': 3, '010110101': 3, '010110110': 3, '010110111': 3, '010111000': 3, '010111001': 3, '010111010': 3, '010111011': 3, '010111100': 3, '010111101': 3, '010111110': 3, '010111111': 3, '011000000': 1, '011000001': 1, '011000010': 1, '011000011': 1, '011000100': 1, '011000101': 1, '011000110': 1, '011000111': 1, '011001000': 1, '011001001': 1, '011001010': 1, '011001011': 1, '011001100': 1, '011001101': 1, '011001110': 1, '011001111': 1, '011010000': 1, '011010001': 1, '011010010': 1, '011010011': 1, '011010100': 1, '011010101': 1, '011010110': 1, '011010111': 1, '011011000': 1, '011011001': 1, '011011010': 1, '011011011': 1, '011011100': 1, '011011101': 1, '011011110': 1, '011011111': 1, '011100000': 1, '011100001': 1, '011100010': 1, '011100011': 1, '011100100': 1, '011100101': 1, '011100110': 1, '011100111': 1, '011101000': 1, '011101001': 1, '011101010': 1, '011101011': 1, '011101100': 1, '011101101': 1, '011101110': 1, '011101111': 1, '011110000': 1, '011110001': 1, '011110010': 1, '011110011': 1, '011110100': 1, '011110101': 1, '011110110': 1, '011110111': 1, '011111000': 1, '011111001': 1, '011111010': 1, '011111011': 1, '011111100': 1, '011111101': 1, '011111110': 1, '011111111': 1, '100000000': 1, '100000001': 1, '100000010': 1, '100000011': 1, '100000100': 1, '100000101': 1, '100000110': 1, '100000111': 1, '100001000': 1, '100001001': 1, '100001010': 1, '100001011': 1, '100001100': 1, '100001101': 1, '100001110': 1, '100001111': 1, '100010000': 1, '100010001': 1, '100010010': 1, '100010011': 1, '100010100': 1, '100010101': 1, '100010110': 1, '100010111': 1, '100011000': 1, '100011001': 1, '100011010': 1, '100011011': 1, '100011100': 1, '100011101': 1, '100011110': 1, '100011111': 1, '100100000': 1, '100100001': 1, '100100010': 1, '100100011': 1, '100100100': 1, '100100101': 1, '100100110': 1, '100100111': 1, '100101000': 1, '100101001': 1, '100101010': 1, '100101011': 1, '100101100': 1, '100101101': 1, '100101110': 1, '100101111': 1, '100110000': 1, '100110001': 1, '100110010': 1, '100110011': 1, '100110100': 1, '100110101': 1, '100110110': 1, '100110111': 1, '100111000': 1, '100111001': 1, '100111010': 1, '100111011': 1, '100111100': 1, '100111101': 1, '100111110': 1, '100111111': 1, '101000000': 3, '101000001': 3, '101000010': 3, '101000011': 3, '101000100': 3, '101000101': 3, '101000110': 3, '101000111': 3, '101001000': 3, '101001001': 3, '101001010': 3, '101001011': 3, '101001100': 3, '101001101': 3, '101001110': 3, '101001111': 3, '101010000': 5, '101010001': 5, '101010010': 5, '101010011': 5, '101010100': 7, '101010101': 0, '101010110': 6, '101010111': 6, '101011000': 4, '101011001': 4, '101011010': 4, '101011011': 4, '101011100': 4, '101011101': 4, '101011110': 4, '101011111': 4, '101100000': 2, '101100001': 2, '101100010': 2, '101100011': 2, '101100100': 2, '101100101': 2, '101100110': 2, '101100111': 2, '101101000': 2, '101101001': 2, '101101010': 2, '101101011': 2, '101101100': 2, '101101101': 2, '101101110': 2, '101101111': 2, '101110000': 2, '101110001': 2, '101110010': 2, '101110011': 2, '101110100': 2, '101110101': 2, '101110110': 2, '101110111': 2, '101111000': 2, '101111001': 2, '101111010': 2, '101111011': 2, '101111100': 2, '101111101': 2, '101111110': 2, '101111111': 2, '110000000': 0, '110000001': 0, '110000010': 0, '110000011': 0, '110000100': 0, '110000101': 0, '110000110': 0, '110000111': 0, '110001000': 0, '110001001': 0, '110001010': 0, '110001011': 0, '110001100': 0, '110001101': 0, '110001110': 0, '110001111': 0, '110010000': 0, '110010001': 0, '110010010': 0, '110010011': 0, '110010100': 0, '110010101': 0, '110010110': 0, '110010111': 0, '110011000': 0, '110011001': 0, '110011010': 0, '110011011': 0, '110011100': 0, '110011101': 0, '110011110': 0, '110011111': 0, '110100000': 0, '110100001': 0, '110100010': 0, '110100011': 0, '110100100': 0, '110100101': 0, '110100110': 0, '110100111': 0, '110101000': 0, '110101001': 0, '110101010': 0, '110101011': 0, '110101100': 0, '110101101': 0, '110101110': 0, '110101111': 0, '110110000': 0, '110110001': 0, '110110010': 0, '110110011': 0, '110110100': 0, '110110101': 0, '110110110': 0, '110110111': 0, '110111000': 0, '110111001': 0, '110111010': 0, '110111011': 0, '110111100': 0, '110111101': 0, '110111110': 0, '110111111': 0, '111000000': 0, '111000001': 0, '111000010': 0, '111000011': 0, '111000100': 0, '111000101': 0, '111000110': 0, '111000111': 0, '111001000': 0, '111001001': 0, '111001010': 0, '111001011': 0, '111001100': 0, '111001101': 0, '111001110': 0, '111001111': 0, '111010000': 0, '111010001': 0, '111010010': 0, '111010011': 0, '111010100': 0, '111010101': 0, '111010110': 0, '111010111': 0, '111011000': 0, '111011001': 0, '111011010': 0, '111011011': 0, '111011100': 0, '111011101': 0, '111011110': 0, '111011111': 0, '111100000': 0, '111100001': 0, '111100010': 0, '111100011': 0, '111100100': 0, '111100101': 0, '111100110': 0, '111100111': 0, '111101000': 0, '111101001': 0, '111101010': 0, '111101011': 0, '111101100': 0, '111101101': 0, '111101110': 0, '111101111': 0, '111110000': 0, '111110001': 0, '111110010': 0, '111110011': 0, '111110100': 0, '111110101': 0, '111110110': 0, '111110111': 0, '111111000': 0, '111111001': 0, '111111010': 0, '111111011': 0, '111111100': 0, '111111101': 0, '111111110': 0, '111111111': 0}
Optimal Policy (PI): {'000000000': 0, '000000001': 0, '000000010': 0, '000000011': 0, '000000100': 0, '000000101': 0, '000000110': 0, '000000111': 0, '000001000': 0, '000001001': 0, '000001010': 0, '000001011': 0, '000001100': 0, '000001101': 0, '000001110': 0, '000001111': 0, '000010000': 0, '000010001': 0, '000010010': 0, '000010011': 0, '000010100': 0, '000010101': 0, '000010110': 0, '000010111': 0, '000011000': 0, '000011001': 0, '000011010': 0, '000011011': 0, '000011100': 0, '000011101': 0, '000011110': 0, '000011111': 0, '000100000': 0, '000100001': 0, '000100010': 0, '000100011': 0, '000100100': 0, '000100101': 0, '000100110': 0, '000100111': 0, '000101000': 0, '000101001': 0, '000101010': 0, '000101011': 0, '000101100': 0, '000101101': 0, '000101110': 0, '000101111': 0, '000110000': 0, '000110001': 0, '000110010': 0, '000110011': 0, '000110100': 0, '000110101': 0, '000110110': 0, '000110111': 0, '000111000': 0, '000111001': 0, '000111010': 0, '000111011': 0, '000111100': 0, '000111101': 0, '000111110': 0, '000111111': 0, '001000000': 0, '001000001': 0, '001000010': 0, '001000011': 0, '001000100': 0, '001000101': 0, '001000110': 0, '001000111': 0, '001001000': 0, '001001001': 0, '001001010': 0, '001001011': 0, '001001100': 0, '001001101': 0, '001001110': 0, '001001111': 0, '001010000': 0, '001010001': 0, '001010010': 0, '001010011': 0, '001010100': 0, '001010101': 0, '001010110': 0, '001010111': 0, '001011000': 0, '001011001': 0, '001011010': 0, '001011011': 0, '001011100': 0, '001011101': 0, '001011110': 0, '001011111': 0, '001100000': 0, '001100001': 0, '001100010': 0, '001100011': 0, '001100100': 0, '001100101': 0, '001100110': 0, '001100111': 0, '001101000': 0, '001101001': 0, '001101010': 0, '001101011': 0, '001101100': 0, '001101101': 0, '001101110': 0, '001101111': 0, '001110000': 0, '001110001': 0, '001110010': 0, '001110011': 0, '001110100': 0, '001110101': 0, '001110110': 0, '001110111': 0, '001111000': 0, '001111001': 0, '001111010': 0, '001111011': 0, '001111100': 0, '001111101': 0, '001111110': 0, '001111111': 0, '010000000': 2, '010000001': 2, '010000010': 2, '010000011': 2, '010000100': 2, '010000101': 2, '010000110': 2, '010000111': 2, '010001000': 2, '010001001': 2, '010001010': 2, '010001011': 2, '010001100': 2, '010001101': 2, '010001110': 2, '010001111': 2, '010010000': 2, '010010001': 2, '010010010': 2, '010010011': 2, '010010100': 2, '010010101': 2, '010010110': 2, '010010111': 2, '010011000': 2, '010011001': 2, '010011010': 2, '010011011': 2, '010011100': 2, '010011101': 2, '010011110': 2, '010011111': 2, '010100000': 4, '010100001': 4, '010100010': 4, '010100011': 4, '010100100': 4, '010100101': 4, '010100110': 4, '010100111': 4, '010101000': 6, '010101001': 6, '010101010': 0, '010101011': 7, '010101100': 5, '010101101': 5, '010101110': 5, '010101111': 5, '010110000': 3, '010110001': 3, '010110010': 3, '010110011': 3, '010110100': 3, '010110101': 3, '010110110': 3, '010110111': 3, '010111000': 3, '010111001': 3, '010111010': 3, '010111011': 3, '010111100': 3, '010111101': 3, '010111110': 3, '010111111': 3, '011000000': 1, '011000001': 1, '011000010': 1, '011000011': 1, '011000100': 1, '011000101': 1, '011000110': 1, '011000111': 1, '011001000': 1, '011001001': 1, '011001010': 1, '011001011': 1, '011001100': 1, '011001101': 1, '011001110': 1, '011001111': 1, '011010000': 1, '011010001': 1, '011010010': 1, '011010011': 1, '011010100': 1, '011010101': 1, '011010110': 1, '011010111': 1, '011011000': 1, '011011001': 1, '011011010': 1, '011011011': 1, '011011100': 1, '011011101': 1, '011011110': 1, '011011111': 1, '011100000': 1, '011100001': 1, '011100010': 1, '011100011': 1, '011100100': 1, '011100101': 1, '011100110': 1, '011100111': 1, '011101000': 1, '011101001': 1, '011101010': 1, '011101011': 1, '011101100': 1, '011101101': 1, '011101110': 1, '011101111': 1, '011110000': 1, '011110001': 1, '011110010': 1, '011110011': 1, '011110100': 1, '011110101': 1, '011110110': 1, '011110111': 1, '011111000': 1, '011111001': 1, '011111010': 1, '011111011': 1, '011111100': 1, '011111101': 1, '011111110': 1, '011111111': 1, '100000000': 1, '100000001': 1, '100000010': 1, '100000011': 1, '100000100': 1, '100000101': 1, '100000110': 1, '100000111': 1, '100001000': 1, '100001001': 1, '100001010': 1, '100001011': 1, '100001100': 1, '100001101': 1, '100001110': 1, '100001111': 1, '100010000': 1, '100010001': 1, '100010010': 1, '100010011': 1, '100010100': 1, '100010101': 1, '100010110': 1, '100010111': 1, '100011000': 1, '100011001': 1, '100011010': 1, '100011011': 1, '100011100': 1, '100011101': 1, '100011110': 1, '100011111': 1, '100100000': 1, '100100001': 1, '100100010': 1, '100100011': 1, '100100100': 1, '100100101': 1, '100100110': 1, '100100111': 1, '100101000': 1, '100101001': 1, '100101010': 1, '100101011': 1, '100101100': 1, '100101101': 1, '100101110': 1, '100101111': 1, '100110000': 1, '100110001': 1, '100110010': 1, '100110011': 1, '100110100': 1, '100110101': 1, '100110110': 1, '100110111': 1, '100111000': 1, '100111001': 1, '100111010': 1, '100111011': 1, '100111100': 1, '100111101': 1, '100111110': 1, '100111111': 1, '101000000': 3, '101000001': 3, '101000010': 3, '101000011': 3, '101000100': 3, '101000101': 3, '101000110': 3, '101000111': 3, '101001000': 3, '101001001': 3, '101001010': 3, '101001011': 3, '101001100': 3, '101001101': 3, '101001110': 3, '101001111': 3, '101010000': 5, '101010001': 5, '101010010': 5, '101010011': 5, '101010100': 7, '101010101': 0, '101010110': 6, '101010111': 6, '101011000': 4, '101011001': 4, '101011010': 4, '101011011': 4, '101011100': 4, '101011101': 4, '101011110': 4, '101011111': 4, '101100000': 2, '101100001': 2, '101100010': 2, '101100011': 2, '101100100': 2, '101100101': 2, '101100110': 2, '101100111': 2, '101101000': 2, '101101001': 2, '101101010': 2, '101101011': 2, '101101100': 2, '101101101': 2, '101101110': 2, '101101111': 2, '101110000': 2, '101110001': 2, '101110010': 2, '101110011': 2, '101110100': 2, '101110101': 2, '101110110': 2, '101110111': 2, '101111000': 2, '101111001': 2, '101111010': 2, '101111011': 2, '101111100': 2, '101111101': 2, '101111110': 2, '101111111': 2, '110000000': 0, '110000001': 0, '110000010': 0, '110000011': 0, '110000100': 0, '110000101': 0, '110000110': 0, '110000111': 0, '110001000': 0, '110001001': 0, '110001010': 0, '110001011': 0, '110001100': 0, '110001101': 0, '110001110': 0, '110001111': 0, '110010000': 0, '110010001': 0, '110010010': 0, '110010011': 0, '110010100': 0, '110010101': 0, '110010110': 0, '110010111': 0, '110011000': 0, '110011001': 0, '110011010': 0, '110011011': 0, '110011100': 0, '110011101': 0, '110011110': 0, '110011111': 0, '110100000': 0, '110100001': 0, '110100010': 0, '110100011': 0, '110100100': 0, '110100101': 0, '110100110': 0, '110100111': 0, '110101000': 0, '110101001': 0, '110101010': 0, '110101011': 0, '110101100': 0, '110101101': 0, '110101110': 0, '110101111': 0, '110110000': 0, '110110001': 0, '110110010': 0, '110110011': 0, '110110100': 0, '110110101': 0, '110110110': 0, '110110111': 0, '110111000': 0, '110111001': 0, '110111010': 0, '110111011': 0, '110111100': 0, '110111101': 0, '110111110': 0, '110111111': 0, '111000000': 0, '111000001': 0, '111000010': 0, '111000011': 0, '111000100': 0, '111000101': 0, '111000110': 0, '111000111': 0, '111001000': 0, '111001001': 0, '111001010': 0, '111001011': 0, '111001100': 0, '111001101': 0, '111001110': 0, '111001111': 0, '111010000': 0, '111010001': 0, '111010010': 0, '111010011': 0, '111010100': 0, '111010101': 0, '111010110': 0, '111010111': 0, '111011000': 0, '111011001': 0, '111011010': 0, '111011011': 0, '111011100': 0, '111011101': 0, '111011110': 0, '111011111': 0, '111100000': 0, '111100001': 0, '111100010': 0, '111100011': 0, '111100100': 0, '111100101': 0, '111100110': 0, '111100111': 0, '111101000': 0, '111101001': 0, '111101010': 0, '111101011': 0, '111101100': 0, '111101101': 0, '111101110': 0, '111101111': 0, '111110000': 0, '111110001': 0, '111110010': 0, '111110011': 0, '111110100': 0, '111110101': 0, '111110110': 0, '111110111': 0, '111111000': 0, '111111001': 0, '111111010': 0, '111111011': 0, '111111100': 0, '111111101': 0, '111111110': 0, '111111111': 0}
Optimal Policy (QL): {'000000000': 4, '000000001': 8, '000000010': 4, '000000011': 7, '000000100': 5, '000000101': 8, '000000110': 5, '000000111': 5, '000001000': 6, '000001001': 7, '000001010': 0, '000001011': 8, '000001100': 6, '000001101': 8, '000001110': 6, '000001111': 8, '000010000': 8, '000010001': 2, '000010010': 4, '000010011': 1, '000010100': 6, '000010101': 6, '000010110': 0, '000010111': 7, '000011000': 4, '000011001': 5, '000011010': 4, '000011011': 3, '000011100': 5, '000011101': 0, '000011110': 5, '000011111': 5, '000100000': 1, '000100001': 8, '000100010': 1, '000100011': 1, '000100100': 5, '000100101': 1, '000100110': 1, '000100111': 5, '000101000': 6, '000101001': 0, '000101010': 1, '000101011': 8, '000101100': 1, '000101101': 0, '000101110': 5, '000101111': 5, '000110000': 4, '000110001': 2, '000110010': 4, '000110011': 4, '000110100': 5, '000110101': 5, '000110110': 0, '000110111': 4, '000111000': 4, '000111001': 0, '000111010': 4, '000111011': 8, '000111100': 6, '000111101': 8, '000111110': 6, '000111111': 5, '001000000': 4, '001000001': 3, '001000010': 1, '001000011': 0, '001000100': 3, '001000101': 5, '001000110': 1, '001000111': 5, '001001000': 2, '001001001': 4, '001001010': 2, '001001011': 1, '001001100': 2, '001001101': 6, '001001110': 6, '001001111': 3, '001010000': 8, '001010001': 0, '001010010': 7, '001010011': 3, '001010100': 6, '001010101': 5, '001010110': 4, '001010111': 3, '001011000': 8, '001011001': 5, '001011010': 1, '001011011': 2, '001011100': 0, '001011101': 4, '001011110': 6, '001011111': 7, '001100000': 1, '001100001': 4, '001100010': 4, '001100011': 5, '001100100': 5, '001100101': 5, '001100110': 1, '001100111': 5, '001101000': 6, '001101001': 1, '001101010': 1, '001101011': 8, '001101100': 6, '001101101': 7, '001101110': 6, '001101111': 6, '001110000': 8, '001110001': 2, '001110010': 3, '001110011': 4, '001110100': 5, '001110101': 5, '001110110': 6, '001110111': 1, '001111000': 4, '001111001': 0, '001111010': 1, '001111011': 8, '001111100': 4, '001111101': 5, '001111110': 6, '001111111': 8, '010000000': 7, '010000001': 8, '010000010': 5, '010000011': 7, '010000100': 7, '010000101': 3, '010000110': 5, '010000111': 7, '010001000': 6, '010001001': 8, '010001010': 3, '010001011': 8, '010001100': 7, '010001101': 8, '010001110': 5, '010001111': 8, '010010000': 7, '010010001': 2, '010010010': 5, '010010011': 4, '010010100': 3, '010010101': 0, '010010110': 5, '010010111': 4, '010011000': 6, '010011001': 3, '010011010': 4, '010011011': 8, '010011100': 6, '010011101': 1, '010011110': 5, '010011111': 7, '010100000': 7, '010100001': 6, '010100010': 5, '010100011': 4, '010100100': 6, '010100101': 6, '010100110': 5, '010100111': 6, '010101000': 6, '010101001': 6, '010101010': 0, '010101011': 8, '010101100': 6, '010101101': 6, '010101110': 6, '010101111': 5, '010110000': 3, '010110001': 3, '010110010': 3, '010110011': 3, '010110100': 4, '010110101': 3, '010110110': 4, '010110111': 3, '010111000': 4, '010111001': 6, '010111010': 3, '010111011': 4, '010111100': 4, '010111101': 3, '010111110': 3, '010111111': 6, '011000000': 4, '011000001': 4, '011000010': 4, '011000011': 2, '011000100': 4, '011000101': 8, '011000110': 5, '011000111': 7, '011001000': 7, '011001001': 8, '011001010': 2, '011001011': 8, '011001100': 5, '011001101': 6, '011001110': 5, '011001111': 8, '011010000': 1, '011010001': 1, '011010010': 4, '011010011': 2, '011010100': 1, '011010101': 5, '011010110': 5, '011010111': 8, '011011000': 3, '011011001': 5, '011011010': 3, '011011011': 7, '011011100': 7, '011011101': 6, '011011110': 5, '011011111': 6, '011100000': 5, '011100001': 2, '011100010': 1, '011100011': 2, '011100100': 6, '011100101': 7, '011100110': 5, '011100111': 1, '011101000': 7, '011101001': 8, '011101010': 2, '011101011': 2, '011101100': 2, '011101101': 2, '011101110': 6, '011101111': 1, '011110000': 6, '011110001': 1, '011110010': 5, '011110011': 8, '011110100': 3, '011110101': 5, '011110110': 1, '011110111': 4, '011111000': 4, '011111001': 4, '011111010': 3, '011111011': 2, '011111100': 2, '011111101': 6, '011111110': 3, '011111111': 6, '100000000': 8, '100000001': 4, '100000010': 0, '100000011': 7, '100000100': 6, '100000101': 0, '100000110': 8, '100000111': 4, '100001000': 0, '100001001': 5, '100001010': 3, '100001011': 0, '100001100': 5, '100001101': 3, '100001110': 0, '100001111': 0, '100010000': 5, '100010001': 1, '100010010': 5, '100010011': 2, '100010100': 7, '100010101': 1, '100010110': 6, '100010111': 6, '100011000': 0, '100011001': 1, '100011010': 0, '100011011': 4, '100011100': 5, '100011101': 1, '100011110': 6, '100011111': 4, '100100000': 7, '100100001': 5, '100100010': 0, '100100011': 0, '100100100': 6, '100100101': 0, '100100110': 5, '100100111': 2, '100101000': 0, '100101001': 0, '100101010': 0, '100101011': 0, '100101100': 0, '100101101': 7, '100101110': 5, '100101111': 0, '100110000': 0, '100110001': 1, '100110010': 4, '100110011': 7, '100110100': 6, '100110101': 6, '100110110': 3, '100110111': 3, '100111000': 0, '100111001': 1, '100111010': 0, '100111011': 1, '100111100': 5, '100111101': 0, '100111110': 0, '100111111': 1, '101000000': 4, '101000001': 2, '101000010': 6, '101000011': 3, '101000100': 3, '101000101': 2, '101000110': 3, '101000111': 6, '101001000': 4, '101001001': 4, '101001010': 0, '101001011': 7, '101001100': 5, '101001101': 5, '101001110': 1, '101001111': 2, '101010000': 8, '101010001': 5, '101010010': 7, '101010011': 6, '101010100': 8, '101010101': 0, '101010110': 6, '101010111': 6, '101011000': 5, '101011001': 5, '101011010': 5, '101011011': 6, '101011100': 4, '101011101': 5, '101011110': 7, '101011111': 6, '101100000': 3, '101100001': 3, '101100010': 2, '101100011': 2, '101100100': 2, '101100101': 5, '101100110': 2, '101100111': 4, '101101000': 1, '101101001': 3, '101101010': 1, '101101011': 2, '101101100': 8, '101101101': 6, '101101110': 2, '101101111': 0, '101110000': 8, '101110001': 2, '101110010': 3, '101110011': 5, '101110100': 3, '101110101': 6, '101110110': 5, '101110111': 0, '101111000': 8, '101111001': 3, '101111010': 2, '101111011': 7, '101111100': 5, '101111101': 5, '101111110': 1, '101111111': 6, '110000000': 0, '110000001': 1, '110000010': 2, '110000011': 8, '110000100': 4, '110000101': 3, '110000110': 0, '110000111': 7, '110001000': 7, '110001001': 8, '110001010': 0, '110001011': 2, '110001100': 0, '110001101': 6, '110001110': 5, '110001111': 3, '110010000': 0, '110010001': 0, '110010010': 0, '110010011': 4, '110010100': 1, '110010101': 1, '110010110': 1, '110010111': 7, '110011000': 2, '110011001': 2, '110011010': 0, '110011011': 4, '110011100': 4, '110011101': 6, '110011110': 5, '110011111': 6, '110100000': 6, '110100001': 7, '110100010': 0, '110100011': 8, '110100100': 5, '110100101': 0, '110100110': 0, '110100111': 0, '110101000': 0, '110101001': 7, '110101010': 0, '110101011': 0, '110101100': 0, '110101101': 8, '110101110': 0, '110101111': 8, '110110000': 4, '110110001': 3, '110110010': 3, '110110011': 5, '110110100': 6, '110110101': 3, '110110110': 6, '110110111': 4, '110111000': 6, '110111001': 6, '110111010': 4, '110111011': 3, '110111100': 5, '110111101': 0, '110111110': 3, '110111111': 5, '111000000': 1, '111000001': 0, '111000010': 1, '111000011': 0, '111000100': 5, '111000101': 3, '111000110': 6, '111000111': 3, '111001000': 0, '111001001': 0, '111001010': 3, '111001011': 2, '111001100': 0, '111001101': 2, '111001110': 0, '111001111': 5, '111010000': 8, '111010001': 1, '111010010': 7, '111010011': 6, '111010100': 1, '111010101': 2, '111010110': 6, '111010111': 0, '111011000': 4, '111011001': 4, '111011010': 0, '111011011': 1, '111011100': 4, '111011101': 1, '111011110': 0, '111011111': 6, '111100000': 6, '111100001': 3, '111100010': 1, '111100011': 8, '111100100': 5, '111100101': 2, '111100110': 1, '111100111': 1, '111101000': 2, '111101001': 0, '111101010': 0, '111101011': 1, '111101100': 5, '111101101': 6, '111101110': 7, '111101111': 5, '111110000': 1, '111110001': 3, '111110010': 2, '111110011': 2, '111110100': 0, '111110101': 6, '111110110': 0, '111110111': 2, '111111000': 3, '111111001': 2, '111111010': 2, '111111011': 4, '111111100': 6, '111111101': 3, '111111110': 5, '111111111': 0}