Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Trainer sub-class DQN/SimpleQ/APEX-DQN/R2D2 (instead of using build_trainer). #20633

Merged
merged 10 commits into from
Nov 30, 2021
Merged

[RLlib] Trainer sub-class DQN/SimpleQ/APEX-DQN/R2D2 (instead of using build_trainer). #20633

merged 10 commits into from
Nov 30, 2021

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Nov 22, 2021 •
edited
Loading

DQN/SimpleQ/APEX-DQN/R2D2 Trainers use sub-classing (instead of using build_trainer).

  • GenericOffPolicyTrainer remains as-is (for algos that build on top of that, like DDPG, CQL, etc..). This will be resolved in a follow up PR.
  • For all DQN-derivatives:
SimpleQ -> DQN -> R2D2
               \-> APEX-DQN

Why are these changes needed?

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into trai…
d43f628 
…ner_sub_class_dqn
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into trai…
d0a9ca0 
…ner_sub_class_dqn

# Conflicts:
#	rllib/agents/dqn/apex.py
#	rllib/agents/trainer.py
@sven1977
fix
46df84e
@sven1977
fix
c482b26
@sven1977 sven1977 added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 24, 2021
@sven1977 sven1977 assigned gjoliver and unassigned avnishn Nov 29, 2021
gjoliver
gjoliver approved these changes Nov 30, 2021
View reviewed changes
Copy link
Member

@gjoliver gjoliver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks very good, if the tests pass, let's merge.

@sven1977 sven1977 merged commit 3d2e274 into ray-project:master Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gjoliver gjoliver gjoliver approved these changes

@avnishn avnishn Awaiting requested review from avnishn

Assignees

@gjoliver gjoliver

Labels
tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sven1977 @gjoliver @avnishn