mtsac_metaworld_mt50.py sets num_tasks=10

[eholly1]

at commit: edf2615

the MTSAC object is created with num_tasks=10, even though being run with MT-50

mtsac = MTSAC(policy=policy,
                  qf1=qf1,
                  qf2=qf2,
                  gradient_steps_per_itr=150,
                  max_episode_length=150,
                  eval_env=mt50_test_envs,
                  env_spec=mt50_train_envs.spec,
                  num_tasks=10,   <---------------------------------------- HERE
                  steps_per_epoch=epoch_cycles,
                  replay_buffer=replay_buffer,
                  min_buffer_size=7500,
                  target_update_tau=5e-3,
                  discount=0.99,
                  buffer_batch_size=6400)

If the MTSAC class is intended to be used with MultiEnvWrapper, it should explicitly check that num_tasks is set properly. Or even just get the value directly from the wrapper.

[eholly1]

The result of this bug was that log_alpha was always evaluated as 0 for 40 out of the 50 tasks. Undetected in code, it just looked like poor task performance.

[ryanjulian]

@avnishn