[RLlib] Fix torch None conversion in `torch_utils.py::convert_to_torc…

…h_tensor`. (ray-project#26863)

Signed-off-by: Stefan van der Kleij <s.vanderkleij@viroteq.com>

rllib/utils/torch_utils.py

@@ -146,6 +146,9 @@ def convert_to_torch_tensor(x: TensorStructType, device: Optional[str] = None):
     """
 
     def mapping(item):
+        if item is None:
+            # returns None with dtype=np.obj
+            return np.asarray(item)
         # Already torch tensor -> make sure it's on right device.
         if torch.is_tensor(item):
             return item if device is None else item.to(device)