No description, website, or topics provided.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation

arXiv paper link

Click below to view the paper video:



Clone the repository and go into the folder:

$ git clone
$ cd CAPs

From now on, we will assume you are in the CAPs folder.

Create the data folder:

$ mkdir data

All experiments will be saved into the data folder. If you have an external hard drive that you wish to use for saving the data, you can create a symlink:

$ ln -s <path/to/external/harddrive/folder/> data


For portability to different machine configurations, we will use docker to run the code. Make sure you have docker (with GPU support) installed. However, you do not need to know anything about how docker works.

Change into the docker directory, and build the dockerfile:

$ cd src/sandbox/caps/docker
$ ./ build

The build will take a while the first time, but only needs to be done once.

After building, start docker and ssh into the session:

$ ./ start
$ ./ ssh

You will now notice you are inside the docker container. The repository is located at ~/caps (while other installed code is located in ~/source). The ~/caps folder in the docker container is the exact same as on your local machine, and changes can be made to the caps files either from inside the docker container or from your local machine.

To exit the docker container, ctrl-d. You can stop the container by:

$ ./ stop

If you stop the container, you will have to start it again before you can ssh in.

From now on, we will assume you are ssh'd into the docker container and are in the ~/caps directory.

Run pre-trained policy

We provide a pre-trained CAPs policy for the task of collision avoidance, driving at 7m/s, staying in the right lane, and driving towards a desired goal in the CARLA simulator.

Download the pre-trained CAPs policy here and unzip it:

$ mkdir -p data/caps/carla/coll_speed_learnedroad_heading/offline
$ cd data/caps/carla/coll_speed_learnedroad_heading/offline
$ mv <path/to/> .
$ unzip

You can then run the CAPs policy:

$ cd scripts
$ python caps/carla/coll_speed_learnedroad_heading/offline/caps -itr 35 -numrollouts 15

Note: sometimes CARLA doesn't close when python finishes, in which case you should run

pkill -9 -f CarlaUE4

If you want to run goal-conditioned Q-learning (GC-DQL) or goal-conditioned Q-learning with separate value functions (GC-DQL-sep), similarly download this file and this file and proceed similarly:

$ cd data/caps/carla/coll_speed_learnedroad_heading/offline
$ mv <path/to/> .
$ unzip
$ mv <path/to/> .
$ unzip

$ cd scripts
$ python caps/carla/coll_speed_learnedroad_heading/offline/gc_dql -itr 500 -numrollouts 15
$ python caps/carla/coll_speed_learnedroad_heading/offline/gc_dql_sep -itr 500 -numrollouts 15

Run train policy

You can also train your own CAPs policy using the same data we gathered. Download this file and extract it:

$ cd data/caps/carla/coll_speed_learnedroad_heading
$ mv <path/to/>
$ unzip

Next, delete the pre-trained policy folder from the previous section. Then train the CAPs policy:

$ cd scripts
$ python caps/carla/coll_speed_learnedroad_heading/offline/caps

You can similarly train GC-DQL and GC-DQL-sep:

$ cd scripts
$ python caps/carla/coll_speed_learnedroad_heading/offline/gc_dql
$ python caps/carla/coll_speed_learnedroad_heading/offline/gc_dql_sep

Run label data

You can also train and run the autonomous CAPs labeling system, which in this experiment is for the right lane event cue.

First, download the CARLA rollouts here and extract it:

$ mkdir -p data/caps/carla/coll_speed
$ cd data/caps/carla/coll_speed
$ mv <path/to/>
$ unzip

Warning: this file is very large (~300GB).

Then, train the right lane road segmenter:

$ cd src/sandbox/caps/scripts/carla
$ python train

After training (or if you stop it early), you can visualize the performance of the model:

$ python eval

Finally, you can then use this learned right lane labeller to label the training data (and convert it into the .tfrecord training data format):

$ cd scripts
$ python caps/carla/coll_speed_learnedroad_heading/label_event_cues -pkl_folders caps/carla/coll_speed/dql -output_folder caps/carla/coll_speed_learnedroad_heading/label_event_cues

Run gather data

You can also gather the CARLA data yourself. In our experiments, we used DQL to gather data for the task of collision avoidance:

$ cd scripts
$ python caps/carla/coll_speed/dql

Warning: this will take multiple days to run.


If you use CAPs, please cite our CoRL 2018 paper.

  title = {Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation},
  author = {Gregory Kahn and Adam Villaflor and Pieter Abbeel and Sergey Levine},
  booktitle = {Proceedings of the 2nd Annual Conference on Robot Learning},
  year = {2018}