[RLlib] Minor fixes (torch GPU bugs + some cleanup). #11609
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sven1977
commented
Oct 26, 2020
| @@ -331,7 +332,8 @@ def compute_actions( | |||
| fetched = builder.get(to_fetch) | |||
|
|
|||
| # Update our global timestep by the batch size. | |||
| self.global_timestep += fetched[0].shape[0] | |||
| self.global_timestep += len(obs_batch) if isinstance(obs_batch, list) \ | |||
There was a problem hiding this comment.
This may be a tensor (eager tracing) or a list.
sven1977
commented
Oct 26, 2020
| @@ -28,7 +28,7 @@ def test_multi_agent_pendulum(self): | |||
| "env": "multi_agent_pendulum", | |||
| "stop": { | |||
| "timesteps_total": 500000, | |||
| "episode_reward_mean": -300.0, | |||
| "episode_reward_mean": -400.0, | |||
There was a problem hiding this comment.
Make some tests run a little faster.
sven1977
commented
Oct 26, 2020
| @@ -187,39 +187,35 @@ def test_batch_ids(self): | |||
| def test_global_vars_update(self): | |||
| # Allow for Unittest run. | |||
| ray.init(num_cpus=5, ignore_reinit_error=True) | |||
| for fw in framework_iterator(frameworks=()): | |||
| for fw in framework_iterator(frameworks=("tf2", "tf")): | |||
There was a problem hiding this comment.
This was completely deactivated! frameworks=empty
sven1977
commented
Oct 26, 2020
| agent.stop() | ||
|
|
||
| def test_no_step_on_init(self): | ||
| register_env("fail", lambda _: FailOnStepEnv()) | ||
| for fw in framework_iterator(frameworks=()): | ||
| for fw in framework_iterator(): |
sven1977
commented
Oct 26, 2020
| @@ -80,7 +80,8 @@ def __init__(self, | |||
| behaviour_policy_logits=behaviour_logits, | |||
| target_policy_logits=target_logits, | |||
| actions=tf.unstack(actions, axis=2), | |||
| discounts=tf.cast(~dones, tf.float32) * discount, | |||
| discounts=tf.cast(~tf.cast(dones, tf.bool), tf.float32) * | |||
There was a problem hiding this comment.
This would fail if dones are floats (0.0 or 1.0).
sven1977
commented
Oct 26, 2020
| if sess: | ||
| expected_logp = sess.run(expected_logp) | ||
| elif fw == "torch": | ||
| expected_logp = expected_logp.detach().cpu().numpy() | ||
| adv = adv.detach().cpu().numpy() |
ericl
reviewed
Oct 26, 2020
rllib/offline/json_reader.py
Outdated
| @@ -150,6 +150,7 @@ def _from_json(batch: str) -> SampleBatchType: | |||
|
|
|||
| if data_type == "SampleBatch": | |||
| for k, v in data.items(): | |||
| print("Trying to unpack {}: {}".format(k, v))#TODO | |||
There was a problem hiding this comment.
Please remove this prior to merging.
There was a problem hiding this comment.
Of course, trying to catch the failing test_marwil/bc.py problem, which is related to a different compression format on the observation and will go into a different PR.
ericl
approved these changes
Oct 26, 2020
ericl
added
the
@author-action-required
cathrinS
pushed a commit
to mariodoebler/ray
that referenced
this pull request
Nov 8, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Minor fixes:
Why are these changes needed?
Related issue number
Checks
scripts/format.shto lint the changes in this PR.