-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RLlib] Trajectory view API: Simple List Collector (on by default for PPO); LSTM-agnostic #11056
[RLlib] Trajectory view API: Simple List Collector (on by default for PPO); LSTM-agnostic #11056
Conversation
rllib/agents/a3c/a3c.py
Outdated
@@ -59,6 +59,14 @@ def validate_config(config): | |||
logger.warning("`sample_async=True` is not supported for PyTorch! " | |||
"Multithreading can lead to crashes.") | |||
|
|||
# Switch on trajectory view API by default for this algo (if None). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switch on by default for A2/3C.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please set this in the A2C config instead of in the code.
rllib/agents/pg/pg.py
Outdated
@@ -33,6 +36,24 @@ | |||
# yapf: enable | |||
|
|||
|
|||
def validate_config(config: TrainerConfigDict) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switch on by default for PG.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please set this in the algorithm config instead of in the code, here and elsewhere.
LGTM, let's get it merged! |
rllib/agents/ppo/ppo.py
Outdated
@@ -89,6 +89,9 @@ | |||
# Whether to fake GPUs (using CPUs). | |||
# Set this to True for debugging on non-GPU machines (set `num_gpus` > 0). | |||
"_fake_gpus": False, | |||
# Switch on Trajectory View API for A2/3C by default. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PPO
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
rllib/agents/pg/pg.py
Outdated
@@ -27,6 +30,9 @@ | |||
"num_workers": 0, | |||
# Learning rate. | |||
"lr": 0.0004, | |||
# Switch on Trajectory View API for A2/3C by default. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PG
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Fixing last failing tests. ... |
…ectory_view_api_simple_collector
Trajectory view API:
This finishes the following part of the Traj. View milestone:
Issue #10879
Closes Issue #10879
Checks
scripts/format.sh
to lint the changes in this PR.