Possible to define an action spec of dictionary?

[IanQS]

Hey there!

How might I go about defining an action spec that is part of a dictionary? Say an action in a parameterized action space? In OpenAI gym something like this has the following signature:

class gym.spaces.Dict(spaces: Optional[dict[str, gym.spaces.space.Space]] = None, seed: Optional[Union[dict, int, gym.utils.seeding.RandomNumberGenerator]] = None, **spaces_kwargs: gym.spaces.space.Space)

and an example of

self.nested_observation_space = spaces.Dict({
    'sensors':  spaces.Dict({
        'position': spaces.Box(low=-100, high=100, shape=(3,)),
        'velocity': spaces.Box(low=-1, high=1, shape=(3,)),
        'front_cam': spaces.Tuple((
            spaces.Box(low=0, high=1, shape=(10, 10, 3)),
            spaces.Box(low=0, high=1, shape=(10, 10, 3))
        )),
        'rear_cam': spaces.Box(low=0, high=1, shape=(10, 10, 3)),
    }),
    'ext_controller': spaces.MultiDiscrete((5, 2, 2)),
    'inner_state':spaces.Dict({
        'charge': spaces.Discrete(100),
        'system_checks': spaces.MultiBinary(10),
        'job_status': spaces.Dict({
            'task': spaces.Discrete(5),
            'progress': spaces.Box(low=0, high=100, shape=()),
        })
    })
})

Would this amount to simply having a standard python dictionary of specs?

[IanQS]

@alimuldal

Sorry for pinging you directly but I wanted to follow up on this

[alimuldal]

dm_env is extremely permissive about what actions are allowed to be, per the docstring:

An Array spec, or a nested dict, list or tuple of Array specs.

The main requirement is that the "leaf nodes" in the structure are subclasses of specs.Array (typically BoundedArrays), e.g.

def action_spec(self):
  return {
    'single_array': specs.BoundedArray(...),
    'nested_sequence_of_arrays': [specs.BoundedArray(...), specs.BoundedArray(...), ...],
    'nested_dict': {
        'inner_array': specs.BoundedArray(...),
        ...
    }
  }

The action passed to step must have the same structure as the thing returned by action_spec, and the arrays it contains must conform to the corresponding specs.Array objects.