A C++/Python simulator package for reinforcement learning
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cmake check in xworld3d_models as part of xworld May 22, 2018
confs release XWorld3D (#97) May 22, 2018
doc release XWorld3D (#97) May 22, 2018
examples release XWorld3D (#97) May 22, 2018
games update XWorld3D reference Sep 20, 2018
python x3_dry_run -> x3_opengl_init for better understanding Sep 20, 2018
tests Xworld3d (#40) Nov 21, 2017
.clang-format pre-commit check all files Aug 24, 2017
.gitignore xworld3d with simple navigation (#21) Oct 31, 2017
.pre-commit-config.yaml pre-commit check all files Aug 24, 2017
.travis.yml
CMakeLists.txt fix opencv in cmake Mar 10, 2018
LICENSE Check in all files Aug 24, 2017
README.md update XWorld3D reference Sep 20, 2018
data_packet.cpp fix the issue of invalid string memory for xworld3d Sep 11, 2018
data_packet.h multi-process simulation interface (#33) Nov 13, 2017
memory_util.h unify simulator interfaces (#36) Nov 14, 2017
simulator.cpp Added more nav tasks May 22, 2018
simulator.h fix a bug that makes the perceived image mis-aligned with sentence. (#80 Mar 21, 2018
simulator_communication.cpp automatically find ports for simulator servers and clients May 22, 2018
simulator_communication.h automatically find ports for simulator servers and clients May 22, 2018
simulator_entity.h real egocentric for xworld2d; add navgation tasks for xworld3d; simpl… Feb 28, 2018
simulator_interface.cpp xworld silent by default Sep 20, 2018
simulator_interface.h automatically find ports for simulator servers and clients May 22, 2018
simulator_util.cpp xworld3d with simple navigation (#21) Oct 31, 2017
simulator_util.h xworld3d with simple navigation (#21) Oct 31, 2017
teacher.cpp automatically find ports for simulator servers and clients May 22, 2018
teacher.h real egocentric for xworld2d; add navgation tasks for xworld3d; simpl… Feb 28, 2018
teaching_task.cpp automatically find ports for simulator servers and clients May 22, 2018
teaching_task.h automatically find ports for simulator servers and clients May 22, 2018

README.md

XWorld

This repository contains a collection of simulators for Reinforcement Learning research.

Difficulty Name Description Thread-compatible?* Optional? Policy Teacher?
Easy SimpleGame A simple 1D array-walking game. Yes No Discrete No
Easy, Medium SimpleRace A simple synthetic car racing game. Yes No Discrete No
Medium, Hard Atari Wrappers for the Arcade Learning Environment (ALE) environment. For stability, we use a fork version. Yes Yes Discrete No
Medium, Hard XWorld2D A 2D world for an agent to learn vision and language abilities. No No Discrete
Continuous
Yes
Hard XWorld3D A 3D world for an agent to learn vision and language abilities. No Yes Discrete
Continuous
Yes

(*If yes, then multithreading can be used; otherwise multiprocessing is needed.)

Architecture

XWorld features a teacher infrastructure implemented as a scheduler of multiple Finite State Machines (FSMs). The idea is that given the environment, the teacher can propose a task sampled (by some heuristics) from a task set. Each task - formulated as an FSM - has several stages, and the teacher does different things in different stages. The transition from one stage to another is determined by the envionment state, e.g., whehter the agent is idle or whether it has achieved the goal. Each stage returns several things including the next stage and the teacher's action. Currently, we define language (strings) as the teacher's sole action. However, the teacher is able to change the environment (e.g., adding/deleting objects, changing the map size, etc.) within each stage.

The above figure illustrates the architecture. The motivation is to let the users flexibly write simple Python scripts to configure the environment maps and tasks.

Currently, the teacher is only incorporated into XWorld2D and XWorld3D.

Requirements

  • Compiler: GCC 4.8 or above
  • CMake: CMake 3.0 or above
  • Python: Python 2.7

Dependencies

The following softwares must be installed before building XWorld.

Boost, Glog, GFlags, GTest, and Python

In Ubuntu 14.04 and 16.04, you can do

sudo apt-get install libboost-all-dev libgflags-dev libgoogle-glog-dev libgtest-dev python-dev

Build

First get this git repository

git clone https://github.com/PaddlePaddle/XWorld

Suppose the directory is xworld_path, then do

cd <xworld_path>
mkdir -p build
cd build
cmake [<optional parameters>] ..

For example,

cd ~/XWorld; mkdir build; cd build
cmake ..

Finally, in the build directory do

make
make test

By default, XWorld only builds the first three games: SimpleGame, SimpleRace, and XWorld2D.

Optionally, you can install Atari by:

cmake -DWITH_ATARI=ON ..

which will automatically download and build Atari.

You can also install XWorld3D by:

cmake -DWITH_XWORLD3D=ON ..

Usage

Python interface

We provide a set of simple Python APIs for interacting with the simulators. After building XWorld, you need to export the path of the python module:

export PYTHONPATH=<xworld_path>/python:$PYTHONPATH

You can add the above line to ~/.bashrc to avoid doing the export in the future.

To get started, several examples of the simulator Python APIs can be found in

<xworld_path>/python/examples

C++ interface

Alternatively, several C++ examples (run the .sh scripts inside) can be found in

<xworld_path>/examples

These examples use the individual class constructors to create games. However, we also provide a unified simulator interface for creating games in a more convenient way (like in Python). A demo of the unified C++ simulator interface for multi-process simulation can be found in

<xworld_path>/examples/demo_interface.cpp

Generally, C++ APIs are more flexible but expose more details compared to the Python APIs.

Flags of a game

Option flags are passed into a game via different ways for the two interfaces:

  • Python: you need to provide the flags when creating the game as the arguments.
  • C++: you need to set the flags via GFlags in the command line.

For descriptions of the flags of a game, please take a look at the README file under the game directory.

Citations

If you use XWorld2D for research, consider citing

If you use XWorld3D for research, consider citing

If you use our wrappers of the third-party simulators, please follow their original guide for citation.

License

This repository has the Apache2.0 license, except that the third-party simulator ALE has its own license.