Skip to content

An C++ Version of Q-Learning, to Train Robot Play with Flappybird!!

Notifications You must be signed in to change notification settings

guoyu07/Q-Learning

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

QLearning Using C++ and Python

Well, for now, this repo include an simple instance using Q-Learning Algorithm to teach robot get out from a room: PicName The purpose of robot is get rid out of room and get into No. 5 space which is the outside. And our Q-Learning robot work very well with this!!! After 500 episode, we get an convergence Q matrix, and when you input and initial room of robot, he can using this map get out of room correct!!!!

-- updated Q matrix: 
0   0   0   0   400 0   
0   0   0   320 0   500 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   500 
0   400 0   0   400 500 
[INFO] Episode: 999
-- step 0: initial state: 4
-- step 0: next action: 5
-- updated Q matrix: 
0   0   0   0   400 0   
0   0   0   320 0   500 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   500 
0   400 0   0   400 500 
Q convergence matrix:
0   0   0   0   400 0   
0   0   0   320 0   500 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   500 
0   400 0   0   400 500 
please input robot locate room: 
1
1->5->out
please input robot locate room: 
2
2->3->1->5->out
please input robot locate room: 
4
4->5->out
please input robot locate room: 

Install and Play with it

To test our Q-Learning algorithm, you can:

git clone https://github.com/jinfagang/Q-Learning.git

and mkdir build:

mkdir build
cd build

and then cmake and make source code:

cmake ..
make -j8

last and the exciting moment:

./Q-Learning

You will get a Q-Learning Robot!!!

We Implement using Full-Network-Training-Like

As we all know, Q-Learning is actually not like nerual network, but we did it in an neural network way, our Q-Learning structure is just like training a model in Deep-Learning method!! For instance, we have also something like epoch which we called episode, which means a series of events:

[INFO] Episode: 894
-- step 0: initial state: 1
-- step 0: next action: 5
-- updated Q matrix: 
0   0   0   0   400 0   
0   0   0   320 0   499.999 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   499.999 
0   400 0   0   400 499.999 
[INFO] Episode: 895
-- step 0: initial state: 3
-- step 0: next action: 2
-- step 1: initial state: 2
-- step 1: next action: 3
-- step 2: initial state: 3
-- step 2: next action: 4
-- step 3: initial state: 4
-- step 3: next action: 0
-- step 4: initial state: 0
-- step 4: next action: 4
-- step 5: initial state: 4
-- step 5: next action: 0
-- step 6: initial state: 0
-- step 6: next action: 4
-- step 7: initial state: 4
-- step 7: next action: 0
-- step 8: initial state: 0
-- step 8: next action: 4
-- step 9: initial state: 4
-- step 9: next action: 3
-- step 10: initial state: 3
-- step 10: next action: 4
-- step 11: initial state: 4
-- step 11: next action: 5
-- updated Q matrix: 
0   0   0   0   400 0   
0   0   0   320 0   499.999 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   499.999 
0   400 0   0   400 499.999 
[INFO] Episode: 896
-- step 0: initial state: 0
-- step 0: next action: 4
-- step 1: initial state: 4
-- step 1: next action: 3
-- step 2: initial state: 3
-- step 2: next action: 1
-- step 3: initial state: 1
-- step 3: next action: 5
-- updated Q matrix: 
0   0   0   0   400 0   
0   0   0   320 0   499.999 
0   0   0   320 0   0   
0   400 256 0   400 0   
320 0   0   320 0   499.999 
0   400 0   0   400 499.999 

Future Work

This Q-Learning repo will be maintained, and we will add it more functional and powerful, you can train your own Q-Learning model using this tiny tool, not bother anything. And most important thing is that, Q-Learning is a very very intelligence way then other machine learning method!!

License

this work is implement by Jin Fagang, if you repost it or distribute it, please add this anounce and follow the Apache License. For any question, you can contact me via wechat: jintianiloveu.

About

An C++ Version of Q-Learning, to Train Robot Play with Flappybird!!

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 93.0%
  • CMake 3.8%
  • C 3.2%