[BUG] Cannot speedup DMControl suite.

[sweetice]

Describe the bug

I test the dmcontrol suite of envpool using the official code.

(base) mzy813@slurm-bm-23:/mnt/qb/work/maghsudi/mzy813/code/test_script$ python test_envpool.py
/mnt/qb/work/maghsudi/mzy813/anaconda3/lib/python3.9/site-packages/glfw/__init__.py:912: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
  warnings.warn(message, GLFWError)
Namespace(domain='cheetah', task='run', total_step=200000, seed=0)
100%|█████████████████████████████████████████████████████████████████████████████████████| 200000/200000 [00:14<00:00, 14148.00it/s]
FPS(dmc) = 14132.27
100%|██████████████████████████████████████████████████████████████████████████████████████| 200000/200000 [00:29<00:00, 6761.19it/s]
FPS(envpool) = 6761.02
EnvPool Speedup: 0.48x
(base) mzy813@slurm-bm-23:/mnt/qb/work/maghsudi/mzy813/code/test_script$ MUJOCO_GL=egl python test_envpool.py
Namespace(domain='cheetah', task='run', total_step=200000, seed=0)
100%|█████████████████████████████████████████████████████████████████████████████████████| 200000/200000 [00:14<00:00, 14249.50it/s]
FPS(dmc) = 14226.82
100%|██████████████████████████████████████████████████████████████████████████████████████| 200000/200000 [00:29<00:00, 6707.58it/s]
FPS(envpool) = 6707.47
EnvPool Speedup: 0.47x

It seems that the envpool cannot speed up dmcontrol.

To Reproduce

The link of code

Expected behavior

Envpool can have an accelerating effect.

Screenshots

If applicable, add screenshots to help explain your problem.

System info

Describe the characteristic of your environment:

• Describe how the library was installed (pip, source, ...)

pip install envpool

• Python version
  Python 3.9.12 (main, Apr 5 2022, 06:56:58)

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                72
On-line CPU(s) list:   0-71
Thread(s) per core:    2
Core(s) per socket:    18
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz
Stepping:              7
CPU MHz:               3299.829
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000
BogoMIPS:              5200.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              25344K
NUMA node0 CPU(s):     0-17,36-53
NUMA node1 CPU(s):     18-35,54-71

• Versions of any other relevant libraries

(base) mzy813@slurm-bm-23:/mnt/qb/work/maghsudi/mzy813/code/test_script$ cat /proc/version
Linux version 3.10.0-1160.76.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Wed Aug 10 16:21:17 UTC 2022

Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import envpool, numpy, sys
l.__version__, numpy.__version__, sys.version, sys.platform)

>>> print(envpool.__version__, numpy.__version__, sys.version, sys.platform)
0.6.4 1.21.5 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] linux

Additional context

nvidia-smi

(base) mzy813@slurm-bm-23:/mnt/qb/work/maghsudi/mzy813/code/test_script$ nvidia-smi
Mon Oct 10 10:06:09 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:DA:00.0 Off |                  N/A |
|  0%   25C    P8    15W / 250W |      1MiB / 11264MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Reason and Possible fixes

pass

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[Trinkle23897]

It's because we ran benchmark result before Aug 5 -- where dm people applied this optimization to dmc google-deepmind/dm_control@ac6d2cd

Will figure out the bottleneck soon, thanks for reporting this issue!