[BUG] Error when not closing an environment

[ErwanFagnou]

Describe the bug

(I am not sure whether this fits in the "bug" or "feature request" category)

I was trying to use the code from the documentation for environment creation, and I faced an quite strange error. The reason was that my program finished without me having called env.close().

Although the fix was very simple, it took me a lot of time to figure out the cause. I would simply suggest having a more explicit error message, and/or if possible automatically closing the environments when exiting the program.

To Reproduce

This code raises the error:

import gym
from torchrl.envs.libs.gym import GymWrapper

gym_env = gym.make("CartPole-v1")
env = GymWrapper(gym_env)
# env.close()

Exception ignored in: <function EnvBase.__del__ at 0x000001C666103AF0>
Traceback (most recent call last):
  File "C:\Users\r1ben\AppData\Local\Programs\Python\Python39\lib\site-packages\torchrl\envs\common.py", line 619, in __del__
  File "C:\Users\r1ben\AppData\Local\Programs\Python\Python39\lib\site-packages\torchrl\envs\common.py", line 759, in close
  File "C:\Users\r1ben\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1282, in __setattr__
  File "C:\Users\r1ben\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\parameter.py", line 10, in __instancecheck__
TypeError: isinstance() arg 2 must be a type or tuple of types

And the error disappears if the env is closed.

System info

• Windows 10
• TorchRL: I installed the version 0.0.3+c2b9f9c using pip install . from the source directory
• Python version: 3.9.9
• Versions of any other relevant libraries:
  • PyTorch: torch-1.13.1+cu116
  • TensorDict: (nightly) 2022.12.22

print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)

None 1.21.0 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] win32

Reason and Possible fixes

I would simply suggest having a more explicit error message, and/or if possible automatically closing the environments when exiting the program.

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[vmoens]

Thanks for reporting this!
The easy fix would definitely be to capture the errors and give a more explicit error message.
I think that what's happening is that when the program ends the order in which objects disappear from the workspace doesn't really make sense, and deleting the env calls env.close() which runs into an error because some classes or objects are out of scope.
TLDR: Besides capturing the error and ignoring it in the __del__ method and/or making it more explicit in the close method I don't think there's an easy fix.
Wdyt?

[ErwanFagnou]

I think the easy fix would definetly be enough. As long as the user understands that the environment has not been closed, it should be fine.
Given the actual state of TorchRL, it would be better to not loose too much time for this. But you may want to keep the issue in mind for when the project becomes more stable!