PyBullet Environments not works

[haochihlin]

Describe the bug
Hi,
First, thanks for such a good repo.

After installation, I could run the trained agent for BreakoutNoFrameskip-v4 with video showing up.
However, when I tried to test the trained agent for PyBullet env, such as HopperBulletEnv-v0, the window pops up but stop with whole black image.

There is no apparent error msg on the terminal, how could I solve this bug?

Cheers

Code example
Succeed:
$ python enjoy.py --algo ppo2 --env BreakoutNoFrameskip-v4 --folder trained_agents/ -n 5000

Failed (window pop up, but with whole black image):
$ python enjoy.py --algo ppo2 --env HopperBulletEnv-v0 --folder trained_agents/ -n 1000
$ python enjoy.py --algo ppo2 --env HumanoidBulletEnv-v0 --folder trained_agents/ -n 1000
...

System Info
Describe the characteristic of your environment:

• pip install all dependencies for stable-baselines (tested with openai-gym 0.12.5, 0.14.0 and 0.15.0)
• GPU models and configuration: CUDA-10.0, Driver-418.87
• Python version: 3.5
• Tensorflow version: 1.14.0
• opencv-python: tested with 3.4.5.20 and 3.4.7.28 --> same issue, however, when changing to 4.X version, the window will crash with error msg "error: (-215:Assertion failed) src_depth != CV_16F && src_depth != CV_32S in function 'convertToShow'".

[araffin]

Hello,
It seems this is an issue with open cv...
Did you succeed creating a window with opencv ?

Like:

import cv2

# Replace with a path to an existing image
image = cv2.imread("image.jpeg")
cv2.imshow("test", image)
cv2.waitKey(0)

[haochihlin]

Hi,

I have tested opencv based on what you suggested, it works without any issue (can see the image on the pop-up window). However, the "HopperBulletEnv-v0" environments still not works ...

The following is the terminal output when executing the command:
$ python enjoy.py --algo ppo2 --env HopperBulletEnv-v0 --folder trained_agents/ -n 1000

---

pybullet build time: Nov 15 2019 11:19:58

pybullet build time: Nov 15 2019 11:19:58
Loading running average
with params: {'norm_reward': False, 'norm_obs': True}

WARNING:tensorflow:From /home/hlin/.virtualenvs/stable-baselines/lib/python3.5/site-packages/stable_baselines/common/policies.py:562: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:Entity <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x7f9b42213390>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x7f9b42213390>>: AttributeError: module 'gast' has no attribute 'Num'
WARNING:tensorflow:Entity <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x7f9b42167b38>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x7f9b42167b38>>: AttributeError: module 'gast' has no attribute 'Num'
WARNING:tensorflow:From /home/hlin/.virtualenvs/stable-baselines/lib/python3.5/site-packages/stable_baselines/ppo2/ppo2.py:193: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

WARNING:tensorflow:From /home/hlin/.virtualenvs/stable-baselines/lib/python3.5/site-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /home/hlin/.virtualenvs/stable-baselines/lib/python3.5/site-packages/stable_baselines/ppo2/ppo2.py:209: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From /home/hlin/.virtualenvs/stable-baselines/lib/python3.5/site-packages/stable_baselines/ppo2/ppo2.py:245: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

---

[erwincoumans]

This should be fixed when using PyBullet 2.6.2. Can you try it?
Note that by default PyBullet uses CPU rendering, which is much slower than hardware OpenGL.
You can accelerate rendering (and enjoy.py) by first starting a GUI server before running the enjoy.py script:

python3 -m pybullet_envs.examples.runServer&
python3 enjoy.py --algo ppo2 --env HopperBulletEnv-v0

Also, now PyBullet can be used in a Google Colab, including GPU rendering using EGL. See this example:
https://colab.research.google.com/drive/1u6j7JOqM05vUUjpVp5VNk0pd8q-vqGlx?authuser=1

[araffin]

@erwincoumans thanks for taking care of that issue =)

[jdagdelen]

@erwincoumans I seem to be getting some sort of credential error when I try to open your Colab notebook liked here. Is that any different than the one here? https://colab.research.google.com/drive/1u6j7JOqM05vUUjpVp5VNk0pd8q-vqGlx?usp=sharing

[erwincoumans]

This should be the same:
https://colab.sandbox.google.com/drive/1u6j7JOqM05vUUjpVp5VNk0pd8q-vqGlx
or this one using the training:
https://colab.sandbox.google.com/drive/15JSROMJbeiqxcUwifPR2NYeeFBKmyIlX#scrollTo=E2eWDjPZsQc5

[jdagdelen]

Thank you very much for the lightning fast response!