Fails to run SAC from terminal.

[rock-it-with-asher]

Hi Everyone!
I work in a PyCharm environment over Ubuntu 18.04
when I try to execute a variation on this command:

python -m spinup.run sac --hid "[32,32]" --env LunarLander-v2 --exp_name installtest --gamma 0.999

variation means I've tried different combinations of --env, with/without --gamma with/without --hid and its always the next Error Callstack

Here is the Error Callstack:

Saving config:

{
    "ac_kwargs":        {
        "hidden_sizes": [
            32,
            32
        ]
    },
    "actor_critic":     "mlp_actor_critic",
    "alpha":    0.2,
    "batch_size":       100,
    "env_fn":   "<function call_experiment.<locals>.thunk_plus.<locals>.<lambda> at 0x7fc51adfe7b8>",
    "epochs":   100,
    "exp_name": "installtest",
    "gamma":    0.999,
    "logger":   {
        "<spinup.utils.logx.EpochLogger object at 0x7fc51ae0d8d0>":     {
            "epoch_dict":       {},
            "exp_name": "installtest",
            "first_row":        true,
            "log_current_row":  {},
            "log_headers":      [],
            "output_dir":       "/home/asheryartsev/Dev/spinningup/data/installtest/installtest_s0",
            "output_file":      {
                "<_io.TextIOWrapper name='/home/asheryartsev/Dev/spinningup/data/installtest/installtest_s0/progress.txt' mode='w' encoding='UTF-8'>":  {
                    "mode":     "w"
                }
            }
        }
    },
    "logger_kwargs":    {
        "exp_name":     "installtest",
        "output_dir":   "/home/asheryartsev/Dev/spinningup/data/installtest/installtest_s0"
    },
    "lr":       0.001,
    "max_ep_len":       1000,
    "polyak":   0.995,
    "replay_size":      1000000,
    "save_freq":        1,
    "seed":     0,
    "start_steps":      10000,
    "steps_per_epoch":  5000
}
WARNING:tensorflow:From /home/asheryartsev/Dev/spinningup/spinup/algos/sac/sac.py:135: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

Traceback (most recent call last):
  File "/home/asheryartsev/Dev/spinningup/spinup/utils/run_entrypoint.py", line 11, in <module>
    thunk()
  File "/home/asheryartsev/Dev/spinningup/spinup/utils/run_utils.py", line 162, in thunk_plus
    thunk(**kwargs)
  File "/home/asheryartsev/Dev/spinningup/spinup/algos/sac/sac.py", line 140, in sac
    act_dim = env.action_space.shape[0]
IndexError: tuple index out of range



================================================================================


There appears to have been an error in your experiment.

Check the traceback above to see what actually went wrong. The 
traceback below, included for completeness (but probably not useful
for diagnosing the error), shows the stack leading up to the 
experiment launch.

================================================================================



Traceback (most recent call last):
  File "/home/asheryartsev/anaconda3/envs/spinup/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/asheryartsev/anaconda3/envs/spinup/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/asheryartsev/Dev/spinningup/spinup/run.py", line 230, in <module>
    parse_and_execute_grid_search(cmd, args)
  File "/home/asheryartsev/Dev/spinningup/spinup/run.py", line 162, in parse_and_execute_grid_search
    eg.run(algo, **run_kwargs)
  File "/home/asheryartsev/Dev/spinningup/spinup/utils/run_utils.py", line 546, in run
    data_dir=data_dir, datestamp=datestamp, **var)
  File "/home/asheryartsev/Dev/spinningup/spinup/utils/run_utils.py", line 171, in call_experiment
    subprocess.check_call(cmd, env=os.environ)
  File "/home/asheryartsev/anaconda3/envs/spinup/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/asheryartsev/anaconda3/envs/spinup/bin/python', '/home/asheryartsev/Dev/spinningup/spinup/utils/run_entrypoint.py', 'eJyNUs9v0zAUdpquyzYYiKEhdprEpROinZA4MYZQERwy7TC4IsuL3cTUcUJsF4o0CWls7SSLyx4Iif+JO38FR668tGMbJ4jl5L3v5Xu/PzQ/fw3J9PG3ElU4XspkoETnkgx+mfalUrTvdGJloeEU2hnx12jOBoKagZiZEEb0Kt13UlmpqR2VAjEf9QouXtbKEexBOybTE8YrvWc8mDQGrWqTNzhR5AQ1HtwmJ2RMxkG/wUPe/BghNrdFassjYoNJeBgEZNLsI/IJJU5eEGjv+kjoIdUsFxCTbOFPkAYGWZwEr8mEHAb44+4R+GadNaay4R92syIXXWYyUY1YZY0Ydp/iNaXUWurUlVMRP85KZbqV03QqdcoRlrWlWL7P2TbE33oEsiUfpqMcjmHDYpn+QcKwZeJdKSqZC207W6pAyGx3bOb0gJbKmQvs3Jdv1YX0NVjIVnyUl5L2i2pQu82u+8ULLsRfesukEQX1uRG0wihE8uAtq1IDfl67nCalAz83pcAYE8xWjqHOLQ4PUN+DA2j7+VQV+5gCKtmaX53V25lVWQe3RYHGbO0UjI+46DOnrIFd3+QysUjyrbzgrl6Sv7nnvcLtwV0yrhJ0yJQTBl5Bu448l7I8Z/D88c9f33+sju9iA5d3nGbVDtNcVPeG98EvsISelYShrmSSc6Gpke9nbuL1eF0Y3zRCcJy7v6qKNBXVBaXtFwtnS2cplxX4J/8eOGeWdaU2FqdnhbGXZWo2cew40tmi+aVLNnAuDvzNsx4wlRamY1hSX/Bh/T4F4aM3jqkZ+c5/7Ac4u9f5DVTuVy0=']' returned non-zero exit status 1.

[ac-93]

The spinning up version of SAC only works with continuous action environments by default. LunarLander has discrete actions however LunarLanderContinuous-v2 uses continuous actions, just replacing LunarLander-v2 with LunarLanderContinuous-v2 should work!

[rock-it-with-asher]

@ac-93 I actually read it in the documentation but I assumed it is continuous already - is there a simple way to predict whether a gym environment is discrete or continuous?

[ac-93]

@AsherYartsevTech Here is a table of OpenAI environments https://github.com/openai/gym/wiki/Table-of-environments, in the 'Action Space' column 'Box' effectively means continuous actions. There may be some other compatibility issues dependent on the environment, e.g. CarRacing-v0 uses image observations which are not compatible with the default Spinning up algorithms.

[rock-it-with-asher]

@ac-93 You brought up an important issue for me: I actually planned to use your SAC implementation for a racing car model we built (with Unreal Engine 4) - can you think of a way of feeding the algorithm with features extracted from racing car's cameras?

[ac-93]

@AsherYartsevTech The most general way would be to pass the images directly into a neural net (probably a convolutional NN) and use that to map directly to actions. You could also create an observation space compiled of features that you extract from the images and then feed these as input to the NN, hopefully the features will contain enough information to make good action predictions.

If you're using continuous actions then there's a slightly modified version of the spinning up SAC algo that works with image observations here https://github.com/ac-93/soft-actor-critic/tree/master/sac_cont_image.
If you're using discrete actions then there is a version that maps to discrete actions in that repo, however I'm not 100% convinced that my implementation is correct.

I think your original question is answered so should probably close this issue. It sounds like a fun project though so if you've got any more questions feel free to drop me an email! (alexchurch1993@gmail.com).

[rock-it-with-asher]

Thanks a lot!! I'll probably consult with you in the near future!