[Feature Request] MultiDiscreteTensorSpec

[matteobettini]

Motivation

Currently there are the following avaialble specs for discrete spaces:

OneHotDiscreteTensorSpec # One hot for one discrete action
MultOneHotDiscreteTensorSpec # One hot for n actions
DiscreteTensorSpec # Categorical integer for one action

It would be nice to introduce the remaining missing spec MultiDiscreteTensorSpec for n categorical actions.

This would allow the _gym_to_torchrl_spec_transform to use this new spec with the parameter categorical_action_encoding to decide which space to assign to gym.spaces.MultIDiscrete doing somethig like this:

    elif isinstance(spec, gym.spaces.multi_discrete.MultiDiscrete):
        return    MultiDiscreteTensorSpec(spec.nvec, device=device)  if categorical_action_encoding
            else MultOneHotDiscreteTensorSpec(spec.nvec, device=device)

as it is already done for gym.spaces.Discrete.

Furthermore it would be nice for these discrete classes to possess the methods to_onehot() and to_categorical() respectively.

[riiswa]

@matteobettini will you work on it? If not I would like to work on it 😄

[matteobettini]

I have a few thing in my hands these days, so I do not know when I could start on this. if you want to do it, go for it :) it would be super helpful