✨ build vectorized env from single env

[Yonv1943]

The subprocess Vectorized environment of stable baselines 3 is practically useful.

So I added a simple subprocess Vectorized environment into ElegantRL.

Demo code: Add the num_envs=int and if_build_vec_env=True to the env_args for a single env. Then **function build_env() **will build a vectorized env automatically.

ElegantRL/examples/demo_A2C_PPO.py

Lines 62 to 72 in 003572d

	env_class = PendulumEnv # run a custom env: PendulumEnv, which based on OpenAI pendulum
	env_args = {
	'env_name': 'Pendulum', # Apply torque on the free end to swing a pendulum into an upright position
	'max_step': 200, # the max step number in an episode for evaluation
	'state_dim': 3, # the x-y coordinates of the pendulum's free end and its angular velocity.
	'action_dim': 1, # the torque applied to free end of the pendulum
	'if_discrete': False, # continuous action space, symbols → direction, value → force
	'num_envs': 4, # the number of sub envs in vectorized env
	'if_build_vec_env': True,
	}

Function build_env() build a vectorized env from a single env using class VecEnv:
Build a vectorized env from a single env:

ElegantRL/elegantrl/train/config.py

Lines 112 to 124 in 003572d

	def build_env(env_class=None, env_args: dict = None, gpu_id: int = -1):
	env_args['gpu_id'] = gpu_id # set gpu_id for vectorized env before build it
	if env_args.get('if_build_vec_env'):
	num_envs = env_args['num_envs']
	env = VecEnv(env_class=env_class, env_args=env_args, num_envs=num_envs, gpu_id=gpu_id)
	elif env_class.__module__ == 'gym.envs.registration':
	import gym
	assert '0.18.0' <= gym.__version__ <= '0.25.2' # pip3 install gym==0.24.0
	gym.logger.set_level(40) # Block warning
	env = env_class(id=env_args['env_name'])
	else:
	env = env_class(**kwargs_filter(env_class.__init__, env_args.copy()))

The vectorized env class VecEnv is a simplify version of subprocess Vectorized environment for ElegantRL.
VecEnv build a vectorized env on GPU. The sub-env number is num_envs.

ElegantRL/elegantrl/train/config.py

Lines 245 to 248 in 003572d

	class VecEnv:
	def __init__(self, env_class: object, env_args: dict, num_envs: int, gpu_id: int = -1):
	self.device = torch.device(f"cuda:{gpu_id}" if (torch.cuda.is_available() and (gpu_id >= 0)) else "cpu")
	self.num_envs = num_envs # the number of sub env in vectorized env.

VecEnv use multiprocessing.Pipe for communication between VecEnv and SubEnv:

ElegantRL/elegantrl/train/config.py

Lines 267 to 271 in 003572d

	self.sub_envs = [
	SubEnv(sub_pipe0=sub_pipe0, vec_pipe1=vec_pipe1,
	env_class=env_class, env_args=env_args, env_id=env_id)
	for env_id, sub_pipe0 in enumerate(sub_pipe0s)
	]