Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DQN Atari examples #187

Merged
merged 19 commits into from
Aug 29, 2020
Merged

DQN Atari examples #187

merged 19 commits into from
Aug 29, 2020

Conversation

yingchengyang
Copy link
Contributor

@yingchengyang yingchengyang commented Aug 19, 2020

This PR aims to provide the script of Atari DQN setting:

  1. A speedrun of PongNoFrameskip-v4 (finished, about half an hour in i7-8750 + GTX1060 with 1M environment steps)
  2. A general script for all atari game

Since we use multiple env for simulation, the result is slightly different from the original paper, but consider to be acceptable.

@Trinkle23897 Trinkle23897 changed the title pong-dqn WIP: pong-dqn Aug 19, 2020
@Trinkle23897 Trinkle23897 reopened this Aug 19, 2020
@Trinkle23897 Trinkle23897 changed the title WIP: pong-dqn WIP: DQN Atari examples and optimize training procedure Aug 19, 2020
@Trinkle23897 Trinkle23897 linked an issue Aug 19, 2020 that may be closed by this pull request
8 tasks
@Trinkle23897 Trinkle23897 changed the title WIP: DQN Atari examples and optimize training procedure WIP: DQN Atari examples Aug 22, 2020
@Trinkle23897 Trinkle23897 changed the base branch from dev to master August 25, 2020 09:11
@thu-ml thu-ml deleted a comment from codecov-commenter Aug 27, 2020
@Trinkle23897 Trinkle23897 marked this pull request as draft August 27, 2020 06:53
@Trinkle23897 Trinkle23897 changed the title WIP: DQN Atari examples DQN Atari examples Aug 27, 2020
@Trinkle23897 Trinkle23897 marked this pull request as ready for review August 27, 2020 23:11
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Show resolved Hide resolved
examples/atari/atari_wrapper.py Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Outdated Show resolved Hide resolved
examples/atari/atari_wrapper.py Show resolved Hide resolved
examples/atari/atari_wrapper.py Show resolved Hide resolved
@youkaichao
Copy link
Collaborator

It is amazing these small wrappers are essential for DQN to play atari. Evils are in the details :(

tianshou/data/buffer.py Outdated Show resolved Hide resolved
examples/atari/atari_dqn.py Outdated Show resolved Hide resolved
@thu-ml thu-ml deleted a comment from codecov-commenter Aug 28, 2020
youkaichao
youkaichao previously approved these changes Aug 28, 2020
@Trinkle23897
Copy link
Collaborator

@duburcqa It's ready now and please have a look when you are free.

@codecov-commenter
Copy link

codecov-commenter commented Aug 28, 2020

Codecov Report

Merging #187 into master will decrease coverage by 0.48%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #187      +/-   ##
==========================================
- Coverage   95.75%   95.26%   -0.49%     
==========================================
  Files          39       39              
  Lines        2425     2430       +5     
==========================================
- Hits         2322     2315       -7     
- Misses        103      115      +12     
Flag Coverage Δ
#unittests 95.26% <100.00%> (-0.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
tianshou/data/buffer.py 98.98% <100.00%> (+0.02%) ⬆️
tianshou/trainer/onpolicy.py 77.61% <0.00%> (-17.92%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 94bfb32...69b860c. Read the comment docs.

Trinkle23897
Trinkle23897 previously approved these changes Aug 29, 2020
@Trinkle23897 Trinkle23897 merged commit 5b49192 into thu-ml:master Aug 29, 2020
@duburcqa
Copy link
Collaborator

Sorry, too many notifications, I missed the request for review :/

@Trinkle23897
Copy link
Collaborator

You can still add the comment and the change will appear in my next pr :)

@yingchengyang yingchengyang deleted the atari branch August 30, 2020 01:59
BFAnas pushed a commit to BFAnas/tianshou that referenced this pull request May 5, 2024
This PR aims to provide the script of Atari DQN setting:
- A speedrun of PongNoFrameskip-v4 (finished, about half an hour in i7-8750 + GTX1060 with 1M environment steps)
- A general script for all atari game
Since we use multiple env for simulation, the result is slightly different from the original paper, but consider to be acceptable.

It also adds another parameter save_only_last_obs for replay buffer in order to save the memory.

Co-authored-by: Trinkle23897 <463003665@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Why is dqn playing breakout so bad? I trained 400 epochs and the final result did not exceed 6 points.
5 participants