Add exploration noise to rl training collector #1481
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add exploration_noise=True to training collector
github-actions
bot
added
the
waiting for triage
Reformat
|
I think it would be better to make the exploration_noise a configurable variable instead of hardcoding it in the code. |
|
This is in the train function, I don't see any scenario where we want to disable exploration when training a RL agent. In the test function, the flag is default to False. |
|
Thank you so much for taking the time to submit your pull request. Your contribution is greatly appreciated! |
PaleNeutron
pushed a commit
to PaleNeutron/qlib
that referenced
this pull request
Oct 11, 2023
* Update vessel.py Add exploration_noise=True to training collector * Update vessel.py Reformat
SunsetWolf
added
enhancement
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add exploration_noise=True to training collector.
Description
DQN family need to use this parameter to randomly sample action.
Default behavior won't change action, this will only affect policy which implemented exploration_noise function :
https://github.com/thu-ml/tianshou/blob/7f8fa241dd501dbab7dba5c7e95f66702ecf039b/tianshou/policy/base.py#L92
Motivation and Context
Current rl framework don't support DQN family algorithm
How Has This Been Tested?
pytest qlib/tests/test_all_pipeline.pyunder upper directory ofqlib.Screenshots of Test Results (if appropriate):
Types of changes