IndexError: invalid index to scalar variable.

[ccarterc]

Issue summary

Trying to setup a simple agent from baselines. For some reason it crashes on first pass when it reaches this line: ap = a[self.num_buttons * p:self.num_buttons * (p + 1)] in retro_env.py where self.num_buttons = 12 and p = 0

System information

• PopOS (Ubuntu 18.04 variant)
• Python 3.6.6
• 0.6.1 version of retro

Here is the output:

name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
totalMemory: 5.94GiB freeMemory: 5.16GiB
2018-10-19 11:38:33.600157: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1490] Adding visible gpu devices: 0
2018-10-19 11:38:33.784812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-10-19 11:38:33.784840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977]      0
2018-10-19 11:38:33.784844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0:   N
2018-10-19 11:38:33.784954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4921 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060, pci bus id: 0000:01:00.0, compute capability: 6.1)
Traceback (most recent call last):
  File "sonicLearner.py", line 20, in <module>
    callback=callback
  File "/home/ccallahan/code/baselines/baselines/deepq/deepq.py", line 284, in learn
    new_obs, rew, done, _ = env.step(env_action)
  File "/home/ccallahan/code/gym-retro/retro/retro_env.py", line 185, in step
    for p, ap in enumerate(self.action_to_array(a)):
  File "/home/ccallahan/code/gym-retro/retro/retro_env.py", line 170, in action_to_array
    ap = a[self.num_buttons * p:self.num_buttons * (p + 1)]
IndexError: invalid index to scalar variable.

Here is the code:

import gym
import retro
from baselines import deepq

def callback(lcl, _glb):
    # stop training if reward exceeds 199
    is_solved = lcl['t'] > 100 and sum(lcl['episode_rewards'][-101:-1]) / 100 >= 199
    return is_solved

env = retro.make(game='SonicTheHedgehog-Genesis', state='GreenHillZone.Act1')
act = deepq.learn(
    env,
    network='mlp',
    lr=1e-3,
    total_timesteps=1000,
    buffer_size=50000,
    exploration_fraction=0.1,
    exploration_final_eps=0.02,
    print_freq=10,
    callback=callback
)
print("Saving model to sonicLearnerModel.pkl")
act.save("sonicLearnerModel.pkl")

[endrift]

I'm pretty sure Deep-Q requires a discrete action space. Pass use_restricted_actions=retro.Actions.DISCRETE to retro.make.

[ccarterc]

That did the trick. :)