Roadmap 0.0.2

[zuoxingdong]

Here we list some todos for next release of 0.0.2

• Global make_env and return EnvSpec. Similar to OpenAI baselines, handle all kinds of environments and make use of functiontools.partial to return argument-free functions

• Support most of function in Network to Policy/Agent class:

  1. to(device)
  2. num_params
  3. train()/eval()
  4. Might be more networks in one policy.
  5. How to group all networks being trackable with internal methods. ModuleList ? such as num_params with all networks together.

• New logger: avoid hierarchical structure of mixture of list, dictionary and ndarray. Pickling it will be extremely slow. Keep only top level as dictionary. Add function with similar to add_tabular.

• Where to handle dtype conversion from numpy to Tensor. Suggested in Agent.choose_action

• Supports VecEnv

  1. StackObservation
  2. VecWrapper
  3. VecNormalize

• Adapts all standard Agent to both single Env and VecEnv

• Write a function to automatically split config IDs with a key

• Write __repr__ for string representation, e.g. Transition/Segment, EnvSpec...

• Add GAE to Trajectory and Segment

• Add non-rolling VecEnv, returning zero for terminated sub-environments, and update TrajectoryRunner to make it more efficient, remove argument N, only with T.