Skip to content
Code to accompany "Does computer vision matter for action?"
Branch: master
Clone or download
Latest commit 114ce24 Jun 14, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
DirectFuturePrediction @ e1a0be9 initial commit Jun 3, 2019
ViZDoom @ 1d793c7 initial commit Jun 3, 2019
gamehook @ 6e4c34f initial commit Jun 3, 2019
gamehook_gtav @ 3247717 initial commit Jun 3, 2019
gta_v remove unused Jun 3, 2019
image_utils initial commit Jun 3, 2019
pyhookv @ fc39cf1 fix submodule, citation Jun 3, 2019
training remove unused Jun 3, 2019
vizdoom_utils initial commit Jun 3, 2019
.gitignore initial commit Jun 3, 2019
.gitmodules initial commit Jun 3, 2019
README.md paper link Jun 14, 2019
logo.jpg initial commit Jun 3, 2019

README.md

Does computer vision matter for action?

Brady Zhou, Philipp Krähenbühl, and Vladlen Koltun
Science Robotics, 4(30), 2019
Paper
Project

Code to accompany the paper.

Installation

git clone --recursive https://github.com/intel-isl/vision-for-action.git

ViZDoom

We require building from source using the included submodule,
which includes hacks for getting labels for the walls and floors.

cd ViZDoom
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_PYTHON3=ON
make

Link ViZDoom and helpers to PYTHONPATH.

export PYTHONPATH=$PYTHONPATH:$PWD/ViZDoom/bin/python3.6/pip_package
export PYTHONPATH=$PYTHONPATH:$PWD

GTA V

This is supported on Windows machines only, and will take a little bit of effort.

You will need to install the following -

After successfully building all of these projects, you should have

dxgi.dll (renamed from gamehook.dll)
python.hk
server.hk
gta5.hk
pyhookv.pyd

Take all of these files and put them in the directory where GTAV.exe lives.
Additionally, move the following files from the gta_v directory to the same directory.

agents_privileged.py
constants.py
controller.py
controls.py
message_packer.py
pyhookv_utils.py
presets.py
scenarios.py

Citation

@article{Zhou2019DoesCV,
  title={Does computer vision matter for action?},
  author={Brady Zhou and Philipp Kr{\"a}henb{\"u}hl and Vladlen Koltun},
  journal={Science Robotics},
  volume={4},
  number={30},
  year={2019}
}
You can’t perform that action at this time.