Make compilation for GPU rendering more transparent

[SudeepDasari]

TL-DR If you can't figure out why your mujoco-py refuses to render headless for GPU this may help.

Describe the bug
mujoco-py relies on an extremely brittle test to determine if a GPU is present during compilation. Note this test is neither necessary - the recommended apt-get method for installing Nvidia drivers doesn't create this directory - nor is it sufficient, since the directory could exist even after the driver is long gone.

Even worse, this isn't documented anywhere and it's entirely unclear during compilation if the mujoco-py stack has successfully found my gpu driver. Often times when the driver is found this error occurs.

To Reproduce
Import mujoco-py on an Ubuntu 18 machine with drivers installed through apt-get. The CPU only version will be compiled, thus causing extremely slow headless rendering.

Expected behavior
GPU compatible headless rendering automatically works on machines with Nvidia drivers, or at the very least some messages warns the user that CPU only version is being built.

Quick hack to resolve
Change this line to LinuxGPUExtensionBuilder and refer to #383 if that issue comes up.

[tkelestemur]

I forced LinuxGPUExtensionBuilder to be the default builder as you suggested. Now I can use GPU when I use windowed rendering, however, windowless rendering (mj.MjRenderContext(self.sim, offscreen=True, opengl_backend='glfw', quiet=False)) still doesn't use GPU. Did you manage to get that working too?

[SudeepDasari]

I have no idea why offscreen=True doesn't work. Maybe this is another thing that needs fixing.

Instead try something like:

from mujoco_py import MjSim, MjRenderContextOffscreen

sim = ....
if sim._render_context_offscreen is None:
                render_context = MjRenderContextOffscreen(sim)
                sim.add_render_context(render_context)

Sourced from here

[Amir-Arsalan]

@tkelestemur How do you check whether the code uses CPU or GPU when you run it?

[tkelestemur]

I basically look at the output of nvidia-smi which shows the programs using the GPU.

[Kelly510]

I changed the line to Builder = LinuxCPUExtensionBuilder and removed all GL headers from mujoco_py/gl/eglshim.c. But the compilation still fails with these messages. Does anyone have the same problem?

Traceback (most recent call last):
  File "train.py", line 236, in <module>
    main()
  File "train.py", line 110, in main
    base_kwargs={'recurrent': args.recurrent_policy})
  File "../dex-3d/ppo/model.py", line 28, in __init__
    self.base = base(num_inputs, cnn_args=cnn_args, **base_kwargs)
  File "../ppo/model.py", line 235, in __init__
    self.cnn[cam] = cnn_model(cnn_args['arch'], num_inputs[cam])
  File "../ppo/model.py", line 220, in cnn_model
    return model.cuda()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 311, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/torch/nn/modules/module.py", line 208, in _apply
  File "../miniconda3/envs/dex/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/ccompiler.py", line 910, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command '../miniconda3/envs/dex/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train.py", line 11, in <module>
    from mujoco_py import load_model_from_xml, MjSim
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "..li/miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 105, in load_cython_ext
    cext_so_path = builder.build()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 221, in build
    built_so_file_path = self._build_impl()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 291, in _build_impl
    so_file_path = super()._build_impl()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 244, in _build_impl
    dist.run_commands()
  File "../miniconda3/envs/dex/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/mujoco_py/builder.py", line 144, in build_extensions
    build_ext.build_extensions(self)
  File "../miniconda3/envs/dex/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "../miniconda3/envs/dex/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "../miniconda3/envs/dex/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command '../miniconda3/envs/dex/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1