Skip to content
/ Reinforcement-Learning-Markov-Decision-Processes Public

Creating two interesting Markov Decision Processes, solving them using value iteration , policy iteration as well as Q-learning, comparing them and analysing how each method converges, how it performs and how their parameters affects the solution.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Younes43/Reinforcement-Learning-Markov-Decision-Processes

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit

Frozen Lake Policy Map Iteration 0 (Value Iteration) Gamma: 0.01.png

Frozen Lake Policy Map Iteration 0 (Value Iteration) Gamma: 0.01.png

 
 

Frozen Lake Policy Map Iteration 0 (Value Iteration) Gamma: 0.05.png

Frozen Lake Policy Map Iteration 0 (Value Iteration) Gamma: 0.05.png

 
 

Frozen Lake Policy Map Iteration 1 (Value Iteration) Gamma: 0.11888888888888888.png

Frozen Lake Policy Map Iteration 1 (Value Iteration) Gamma: 0.11888888888888888.png

 
 

Frozen Lake Policy Map Iteration 1 (Value Iteration) Gamma: 0.15.png

Frozen Lake Policy Map Iteration 1 (Value Iteration) Gamma: 0.15.png

 
 

Frozen Lake Policy Map Iteration 2 (Value Iteration) Gamma: 0.22777777777777777.png

Frozen Lake Policy Map Iteration 2 (Value Iteration) Gamma: 0.22777777777777777.png

 
 

Frozen Lake Policy Map Iteration 2 (Value Iteration) Gamma: 0.25.png

Frozen Lake Policy Map Iteration 2 (Value Iteration) Gamma: 0.25.png

 
 

Frozen Lake Policy Map Iteration 3 (Value Iteration) Gamma: 0.33666666666666667.png

Frozen Lake Policy Map Iteration 3 (Value Iteration) Gamma: 0.33666666666666667.png

 
 

Frozen Lake Policy Map Iteration 3 (Value Iteration) Gamma: 0.35.png

Frozen Lake Policy Map Iteration 3 (Value Iteration) Gamma: 0.35.png

 
 

Frozen Lake Policy Map Iteration 4 (Value Iteration) Gamma: 0.44555555555555554.png

Frozen Lake Policy Map Iteration 4 (Value Iteration) Gamma: 0.44555555555555554.png

 
 

Frozen Lake Policy Map Iteration 4 (Value Iteration) Gamma: 0.45.png

Frozen Lake Policy Map Iteration 4 (Value Iteration) Gamma: 0.45.png

 
 

Frozen Lake Policy Map Iteration 5 (Value Iteration) Gamma: 0.55.png

Frozen Lake Policy Map Iteration 5 (Value Iteration) Gamma: 0.55.png

 
 

Frozen Lake Policy Map Iteration 5 (Value Iteration) Gamma: 0.5544444444444444.png

Frozen Lake Policy Map Iteration 5 (Value Iteration) Gamma: 0.5544444444444444.png

 
 

Frozen Lake Policy Map Iteration 6 (Value Iteration) Gamma: 0.65.png

Frozen Lake Policy Map Iteration 6 (Value Iteration) Gamma: 0.65.png

 
 

Frozen Lake Policy Map Iteration 6 (Value Iteration) Gamma: 0.6633333333333333.png

Frozen Lake Policy Map Iteration 6 (Value Iteration) Gamma: 0.6633333333333333.png

 
 

Frozen Lake Policy Map Iteration 7 (Value Iteration) Gamma: 0.75.png

Frozen Lake Policy Map Iteration 7 (Value Iteration) Gamma: 0.75.png

 
 

Frozen Lake Policy Map Iteration 7 (Value Iteration) Gamma: 0.7722222222222221.png

Frozen Lake Policy Map Iteration 7 (Value Iteration) Gamma: 0.7722222222222221.png

 
 

Frozen Lake Policy Map Iteration 8 (Value Iteration) Gamma: 0.85.png

Frozen Lake Policy Map Iteration 8 (Value Iteration) Gamma: 0.85.png

 
 

Frozen Lake Policy Map Iteration 8 (Value Iteration) Gamma: 0.8811111111111111.png

Frozen Lake Policy Map Iteration 8 (Value Iteration) Gamma: 0.8811111111111111.png

 
 

Frozen Lake Policy Map Iteration 9 (Value Iteration) Gamma: 0.95.png

Frozen Lake Policy Map Iteration 9 (Value Iteration) Gamma: 0.95.png

 
 

Frozen Lake Policy Map Iteration 9 (Value Iteration) Gamma: 0.99.png

Frozen Lake Policy Map Iteration 9 (Value Iteration) Gamma: 0.99.png

 
 

README.md

README.md

 
 

README.txt

README.txt

 
 

hmw4.html

hmw4.html

 
 

hmw4.ipynb

hmw4.ipynb

 
 

mdp.py

mdp.py

 
 

ybouzekraoui3_analysis.pdf

ybouzekraoui3_analysis.pdf

 
 

Repository files navigation

Markov Decision Processes

Author: Younes EL BOUZEKRAOUI

Author Email: ybouzekraoui3@gatech.edu

Git Repository :

Google Drive : https://drive.google.com/drive/folders/1eBCz1aysS-T2zLGcNMg-syHbuCPO2jJU?usp=sharing

This Google drive https://drive.google.com/drive/folders/1eBCz1aysS-T2zLGcNMg-syHbuCPO2jJU?usp=sharing contains one jupyter notebook (.ipynb) and a folder named 'archive' containing 4 datasets (.csv)

To reproduce the results and the figures, please make sure that the file mdp.py is in the same repository as the notebook hmw4.ipynb and then run all the cells of the notebook.

Note that it might take few minutes to run all the cells and get the results.

Repository Structure:

|--hmw4.ipynb
|--hmw4.html
|--README.md
|--README.txt
|--mdp.py
|--ybouzekraoui3_analysis.pdf

Programming language and Libraries

All the code related to this assignement, the results and the ploted figures are in the jupyter notebook file.

The programming language used is Python (Python 3) and the libraries used are :

  • Numpy
  • Matplotlib
  • Random
  • Seaborn
  • gym
  • pymdptoolbox

Please make sure to have Jupter notebook and all the libraries above installed, the first cell of the notebook installes the required packages if the are not already installed .

About

Creating two interesting Markov Decision Processes, solving them using value iteration , policy iteration as well as Q-learning, comparing them and analysing how each method converges, how it performs and how their parameters affects the solution.

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages