Fully support from/to numpy/pytorch for Batch

[duburcqa]

The current implementation of PPO and other policy algorithm do not support action dict because of this line.

It could be solved by adding to new method to Batch class to convert back the relevant fields to torch.Tensor.

I'm opening a PR to fix that.

[duburcqa]

Also, the use of torch.tensor must be prohibited to convert numpy array to torch tensor since it is less efficient and break memory sharing on cpu.

[Trinkle23897]

Also, the use of torch.tensor must be prohibited to convert numpy array to torch tensor since it is less efficient and break memory sharing on cpu.

For most of the scenarios, the agent's action contains only a few elements that could be considered as negligible.
Currently, the replay buffer is stored as np.ndarray. If you want to prohibit the conversion, the underlying data structure should change to torch.tensor. But I think it is not a good approach since the memory in GPU is far less than RAM, and some basic operations (e.g. compute the returns) are more efficient in the CPU side.

[duburcqa]

I'm not saying to use only torch.tensor, but convert them using torch.from_numpy. Sorry for lack of clarity :/

[duburcqa]

@Trinkle23897 The PR should be ready right now. I try to do the minimal modifications to fully support Batch from/to numpy/pytorch.