Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mtsac_metaworld_mt50.py sets num_tasks=10 #1948

Closed
eholly1 opened this issue Aug 20, 2020 · 2 comments · Fixed by #1975
Closed

mtsac_metaworld_mt50.py sets num_tasks=10 #1948

eholly1 opened this issue Aug 20, 2020 · 2 comments · Fixed by #1975
Assignees
Labels
backport-to-2020.06 Backport this PR to release-2020.06 bug Something isn't working
Milestone

Comments

@eholly1
Copy link

eholly1 commented Aug 20, 2020

at commit: edf2615

the MTSAC object is created with num_tasks=10, even though being run with MT-50

mtsac = MTSAC(policy=policy,
                  qf1=qf1,
                  qf2=qf2,
                  gradient_steps_per_itr=150,
                  max_episode_length=150,
                  eval_env=mt50_test_envs,
                  env_spec=mt50_train_envs.spec,
                  num_tasks=10,   <---------------------------------------- HERE
                  steps_per_epoch=epoch_cycles,
                  replay_buffer=replay_buffer,
                  min_buffer_size=7500,
                  target_update_tau=5e-3,
                  discount=0.99,
                  buffer_batch_size=6400)

If the MTSAC class is intended to be used with MultiEnvWrapper, it should explicitly check that num_tasks is set properly. Or even just get the value directly from the wrapper.

@eholly1
Copy link
Author

eholly1 commented Aug 20, 2020

The result of this bug was that log_alpha was always evaluated as 0 for 40 out of the 50 tasks. Undetected in code, it just looked like poor task performance.

@ryanjulian ryanjulian added the bug Something isn't working label Aug 20, 2020
@ryanjulian
Copy link
Member

@avnishn

@ryanjulian ryanjulian added this to the v2020.09rc4 milestone Aug 20, 2020
@ryanjulian ryanjulian added the backport-to-2020.06 Backport this PR to release-2020.06 label Aug 20, 2020
@avnishn avnishn linked a pull request Aug 27, 2020 that will close this issue
@mergify mergify bot closed this as completed in #1975 Aug 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-to-2020.06 Backport this PR to release-2020.06 bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants