Various MTSAC bug fixes #1975
Merged
Various MTSAC bug fixes #1975
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mergify
bot
requested review from
a team,
AiRuiChen and
nicolengsy
and removed request for
a team
|
@avnishn Thanks Avnishn! I think I forget to read contribution guidelines when submit the PR, sorry for making so much troubles to you. |
Codecov Report
@@ Coverage Diff @@
## master #1975 +/- ##
==========================================
- Coverage 93.50% 93.40% -0.11%
==========================================
Files 192 192
Lines 10182 10184 +2
Branches 1268 1269 +1
==========================================
- Hits 9521 9512 -9
- Misses 438 446 +8
- Partials 223 226 +3
Continue to review full report at Codecov.
|
ryanjulian
reviewed
Aug 27, 2020
| env_spec=ml1_train_envs.spec, | ||
| num_tasks=50, | ||
| steps_per_epoch=epoch_cycles, | ||
| replay_buffer=replay_buffer, |
There was a problem hiding this comment.
please only call args as args and kwargs as kwargs.
ryanjulian
reviewed
Aug 27, 2020
| 'The correct number of tasks?') | ||
| obs = torch.Tensor([env.reset()[0]] * buffer_batch_size) | ||
| with pytest.raises(ValueError, match=error_string): | ||
| mtsac._get_log_alpha(dict(observation=obs)) |
There was a problem hiding this comment.
no need to test a private method?
There was a problem hiding this comment.
This is true, however, the tests are in place to verify the correctness of this implementation. I feel more comfortable having the tests than not. At the same time, it makes no sense to have this as a publicly exposed field because it has no use outside of the algorithm.
|
Please reference the issues you're fixing |
ryanjulian
approved these changes
Aug 27, 2020
maliesa96
reviewed
Aug 28, 2020
| runner.setup(algo=mtsac, | ||
| env=mt10_train_envs, | ||
| sampler_cls=LocalSampler, | ||
| n_workers=1) |
There was a problem hiding this comment.
Why limit the number of workers here, and can't we use ray?
|
@maliesa96 we can't. Tldr; using the ray sampler with the old metaworld envs uses more memory than we have on lab machines. |
|
Damn alright, LGTM then. |
maliesa96
approved these changes
Aug 28, 2020
fixes to Examples to use the correct num_tasks fixes to max_episode_length_eval being used by the algorithm Co-authored-by: Tianhong Dai <tianhongdai914@gmail.com>
ahtsan
force-pushed
the
Avnish-mtsac-bug-fixes
branch
from
e191183
to
c83d34e
Compare
maliesa96
approved these changes
Aug 28, 2020
avnishn
pushed a commit
that referenced
this pull request
Sep 9, 2020
Backport #1905, #1975, #1908 to fix problems with max_eval_path_length being not used by mtsac and sac, and add checking for incorrect num_tasks being set in mtsac. Timelimit.truncated modified only when necessary This issue occurs when there are multiple garage envs that are nested or timelimit truncated = False is included in the environment keys. Previously, our timelimit truncated logic was written with the idea in mind that the key was only added when a time limit truncation occured. If an environment already has timelimit truncated = False in its keys then the previous behavior was to set Done = True which is the incorrect behavior. That was causing performance degradation in MTSAC and MTPPO/TRPO. Now Done is only true in the normal/trivial case, never if timelimit truncated is False.
avnishn
pushed a commit
that referenced
this pull request
Sep 11, 2020
Backport #1905, #1975, #1908 to fix problems with max_eval_path_length being not used by mtsac and sac, and add checking for incorrect num_tasks being set in mtsac. Timelimit.truncated modified only when necessary This issue occurs when there are multiple garage envs that are nested or timelimit truncated = False is included in the environment keys. Previously, our timelimit truncated logic was written with the idea in mind that the key was only added when a time limit truncation occured. If an environment already has timelimit truncated = False in its keys then the previous behavior was to set Done = True which is the incorrect behavior. That was causing performance degradation in MTSAC and MTPPO/TRPO. Now Done is only true in the normal/trivial case, never if timelimit truncated is False.
avnishn
pushed a commit
that referenced
this pull request
Sep 11, 2020
Backport #1905, #1975, #1908 to fix problems with max_eval_path_length being not used by mtsac and sac, and add checking for incorrect num_tasks being set in mtsac. Timelimit.truncated modified only when necessary This issue occurs when there are multiple garage envs that are nested or timelimit truncated = False is included in the environment keys. Previously, our timelimit truncated logic was written with the idea in mind that the key was only added when a time limit truncation occured. If an environment already has timelimit truncated = False in its keys then the previous behavior was to set Done = True which is the incorrect behavior. That was causing performance degradation in MTSAC and MTPPO/TRPO. Now Done is only true in the normal/trivial case, never if timelimit truncated is False.
mergify bot
pushed a commit
that referenced
this pull request
Sep 11, 2020
Backport #1905, #1975, #1908 to fix problems with max_eval_path_length being not used by mtsac and sac, and add checking for incorrect num_tasks being set in mtsac. Timelimit.truncated modified only when necessary This issue occurs when there are multiple garage envs that are nested or timelimit truncated = False is included in the environment keys. Previously, our timelimit truncated logic was written with the idea in mind that the key was only added when a time limit truncation occured. If an environment already has timelimit truncated = False in its keys then the previous behavior was to set Done = True which is the incorrect behavior. That was causing performance degradation in MTSAC and MTPPO/TRPO. Now Done is only true in the normal/trivial case, never if timelimit truncated is False.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes to Examples to use the correct num_tasks
fixes to max_episode_length_eval being used by the algorithm
Co-authored-by: Tianhong Dai tianhongdai914@gmail.com