Procgen Integration

[YukunJ]

Description

Continued Procgen integration progress with @LeoGuo98 together.

A few things:

1. globalGameRegistry somehow doesn't successfully register each game while bazel build & compile. Temporary solution is just write a patch for each game to return a shared_ptr in this commit.

2. The batch size for the StateBufferQueue seems to be 0, even if we set it explicitly, which leads to division by zero error. We haven't figured out what's the cause behind it. The env._recv() and env._reset() is the "crash entry point". See details comments in this commit. Trinkle points out that it's probably a memory allocation issue regard to array indexing.

3. There is also some problem with loading the game background picture. Currently by setting game_->options.use_generated_assets = true; to avoid such loading. But not sure it it would have adverse effect later on.

4. Currently when stepping, we see different RGB 64*64 observations, meaning the game character is indeed moving.

5. We fix the problem with IsDone always being True. The game could works. And cv2.imwrite() we export the rgb 64x64x3 pictures for verification. It does look correct.

6. We finished deterministic and alignment test on procgen raw environment and Envpool's environment with gym and dmc

7. We supported build environment from scratch on Ubuntu 22.04 LTS (solved QT path dependency problem) (a raw new Ubuntu needs sudo apt update && sudo apt install qt5-default && sudo apt-get install qtdeclarative5-dev to install necessary QT dependency for Procgen compilation

PROBLEM AT HAND:

• Cannot pip install procgen in the Bazel sandbox, because original procgen folder has empty space in between folder name.

Solution now:

• Out sourcing the procgen repo to my own github fork and make modification there. see link

Progress:

• Able to build the me-modified procgen on new AWS Ubuntu 20.04 LTS instance.
• Able to pip install procgen@git+https://github.com/YukunJ/procgen.git and enter Python to import this procgen library and make environments (both on AWS instance and my local MacOS)

But, on the github testing machine, when building the project, it gives the error of links

...
File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/0afe9dcfb3afec104db6df7ee3f95f3c/external/pypi__setuptools/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/tmp/pip-wheel-3u_a5q_c/procgen_c19c3bc9b11045eb8ba9f6db8928ae4c/setup.py", line 65, in run
          import builder
        File "/tmp/pip-wheel-3u_a5q_c/procgen_c19c3bc9b11045eb8ba9f6db8928ae4c/procgen/builder.py", line 11, in <module>
          import gym3
      ModuleNotFoundError: No module named 'gym3'

even if I add the requirements.txt

...
gym3
procgen@git+https://github.com/YukunJ/procgen.git

But I don't have this problem building Envpool project from stratch on AWS Instance. I am able to execute the following command:

// Start a new AWS Instance Ubuntu 22.04 LTS

// Ubuntu Basic Build Tool Support
sudo apt update &&
sudo add-apt-repository ppa:ubuntu-toolchain-r/test &&
sudo apt install -y gcc-9 g++-9 build-essential &&
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-9 &&
sudo apt install -y python3-dev python3-pip &&
sudo ln -sf /usr/bin/python3 /usr/bin/python &&
sudo apt update && sudo apt install qt5-default && 
sudo apt-get install qtdeclarative5-dev
sudo apt install -y python3-dev python3-pip
sudo ln -sf /usr/bin/python3 /usr/bin/python

// Install my 3rd Procgen
pip uninstall procgen
pip install procgen@git+https://github.com/YukunJ/procgen.git

// Run Procgen in Python
$ python3
>>> import gym
>>> import procgen
>>> import numpy as np
>>> procgen_gym = gym.make("procgen:procgen-bigfish-v0", rand_seed=0, use_generated_assets=True)
>>> procgen_gym.reset()
>>> num_env, act_space = 1, procgen_gym.action_space
>>> step_count = 0
>>> raw_done = False
>>> while (not raw_done):
	action = np.array([act_space.sample() for _ in range(num_env)])
	_, raw_reward, raw_done, _ = procgen_gym.step(action[0])
	step_count += 1
	print("Step=", step_count)