Updated 1.dqn for compatability with PyTorch 0.4 and 1.0 #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joleeson
commented
Feb 8, 2019
joleeson
commented
- Updated for compatibility with latest PyTorch versions. (more thorough than recommendations in Update to run on torch 0.4 #20)
- no longer uses the deprecated "Variable" class
- use of appropriate dtypes
- cpu/gpu agnostic code
- use of tensor.item() for conversion of 0-dimensional tensors to ordinary python numbers
- Made changes such that the algorithm more closely matches that in Mnih et al. (2015) and other DQN literature:
- linear epsilon decay
- frame stacking
- training frequency is now once every 4 steps in the environment for Atari env
- option of using Huber loss instead of RMS loss in def compute_td_loss()
- Borrowed monitoring wrapper from OpenAI's Baselines to log progress of training.
- Modified the wrappers such that it now accommodates stacked frames frame_stack default to False #9 , and outputs them as a LazyFrames object. Axes of the data is appropriately swapped for PyTorch i.e. (no. of channels)x(breadth)x(height)
Updated for PyTorch 0.4. Made changes such that the algorithm more closely matches that in Mnih et al. (2015) and other DQN literature: - linear epsilon decay - frame stacking - training frequency is now once every 4 steps in the environment for Atari env - option of using Huber loss instead of RMS loss in def compute_td_loss() Also borrowed logging facility from OpenAI's Baselines
-Borrowed monitoring wrapper from OpenAI's Baselines to log progress of training. -Modified the wrappers such that it now accommodates stacked frames, and outputs them as a LazyFrames object. Axes of the data is appropriately swapped for PyTorch i.e. (no. of channels)x(breadth)x(height)
colin-leu
reviewed
May 21, 2019
| "import torch.nn.functional as F\n", | ||
| "\n", | ||
| "import os\n", | ||
| "import logger\n", |
There was a problem hiding this comment.
which specific module is this? (can't find a module named logger)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.