record_video.py not working on PyBullet envs [bug]

[mcres]

Describe the bug

After training a PyBullet environment, I try to record a video of it:

python -m utils.record_video --algo tqc --env ReacherBulletEnv-v0 -n 1000

And I get the following output:

pybullet build time: Mar  8 2021 17:24:12
Loading latest experiment, id=1
pybullet build time: Mar  8 2021 17:24:12
Loading latest experiment, id=1
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/forkserver.py", line 196, in main
    _serve_one(s, listener, alive_r, old_handlers)
  File "/usr/lib/python3.6/multiprocessing/forkserver.py", line 231, in _serve_one
    code = spawn._main(child_r)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 114, in _main
    prepare(preparation_data)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 223, in prepare
    _fixup_main_from_name(data['init_main_from_name'])
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 249, in _fixup_main_from_name
    alter_sys=True)
  File "/usr/lib/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/mcres/git/rl-baselines3-zoo/utils/record_video.py", line 74, in <module>
    hyperparams=hyperparams,
  File "/home/mcres/git/rl-baselines3-zoo/utils/utils.py", line 219, in create_test_env
    vec_env_kwargs=vec_env_kwargs,
  File "/home/mcres/git/stable-baselines3/stable_baselines3/common/env_util.py", line 102, in make_vec_env
    return vec_env_cls([make_env(i + start_index) for i in range(n_envs)], **vec_env_kwargs)
  File "/home/mcres/git/stable-baselines3/stable_baselines3/common/vec_env/subproc_vec_env.py", line 106, in __init__
    process.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 291, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_forkserver.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 143, in get_preparation_data
    _check_not_importing_main()
  File "/usr/lib/python3.6/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/mcres/git/rl-baselines3-zoo/utils/record_video.py", line 74, in <module>
    hyperparams=hyperparams,
  File "/home/mcres/git/rl-baselines3-zoo/utils/utils.py", line 219, in create_test_env
    vec_env_kwargs=vec_env_kwargs,
  File "/home/mcres/git/stable-baselines3/stable_baselines3/common/env_util.py", line 102, in make_vec_env
    return vec_env_cls([make_env(i + start_index) for i in range(n_envs)], **vec_env_kwargs)
  File "/home/mcres/git/stable-baselines3/stable_baselines3/common/vec_env/subproc_vec_env.py", line 111, in __init__
    observation_space, action_space = self.remotes[0].recv()
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

The same thing happens if I change algorithm and environment, e.g. td3 and HalfCheetahBulletEnv-v0, respectively.

System Info
Describe the characteristic of your environment:

• Describe how Stable Baselines3 was installed: pip install -e
• Python version: 3.6.9
• PyTorch version: 1.8.0
• Versions of any other relevant libraries:

gym==0.18.0
pybullet==3.1.0

Additional context
Recording video on normal gym environments is working.
Executing the PyBullet envs with enjoy.py is also working.

[araffin]

you have two solutions : try with MKL_THREADING_LAYER=GNU (see https://github.com/DLR-RM/stable-baselines3/blob/master/.gitlab-ci.yml) or change the default start method to fork (cf doc).

[mcres]

I don't follow, could you please be more specific? Where should I make those specific changes?

Also, would be fixing this doable? If not, handling the error and display a message to how to manually fix it sounds like a good option to me.

[araffin]

I don't follow, could you please be more specific? Where should I make those specific changes?

MKL_THREADING_LAYER=GNU is an environment variable, so it works like that:
MKL_THREADING_LAYER=GNU python train.py ...

the start method can be specified here:

https://github.com/DLR-RM/rl-baselines3-zoo/blob/master/utils/exp_manager.py#L91

Also, would be fixing this doable? If not, handling the error and display a message to how to manually fix it sounds like a good option to me.

unfortunately, it may come from python and on some linux computers only, so I'm not sure how easy it is to catch...

[mcres]

Thanks! I tried both methods but unfortunately, I keep getting the same error :/
What I don't understand is why it's trying to create several pybullet envs to create a single video. Isn't one env enough for that?

unfortunately, it may come from python and on some linux computers only, so I'm not sure how easy it is to catch...

I guessed it wasn't easy, at least this issue can be useful for the moment

[araffin]

So, the fix is here: #77

Different things:

• as mentioned in the doc (https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html) the thread safe method need to be protected with the __main__ (I wrote the code when we were still using fork as default)
• the start method was not at the right place, for testing, it is defined here: https://github.com/DLR-RM/rl-baselines3-zoo/blob/master/utils/utils.py#L203
• we are not using multiple envs, but just one subproc env for rendering PyBullet envs (this is a hack), you can also pass --no-render if you just need the video without direct visual feedback (faster).