Skip to content
project for paper "Learning 3D Human Dynamics from Video"
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
datasets
doc demo commit! May 15, 2019
resources demo commit! May 15, 2019
src updated num_fill to be more conservative May 24, 2019
.gitignore demo commit! May 15, 2019
README.md demo commit! May 15, 2019
__init__.py demo commit! May 15, 2019
demo_video.py demo commit! May 15, 2019
extract_tracks.py demo commit! May 15, 2019
requirements.txt

README.md

Learning 3D Human Dynamics from Video

Angjoo Kanazawa*, Jason Zhang*, Panna Felsen*, Jitendra Malik

University of California, Berkeley (* Equal contribution)

Project Page

Teaser Image

Requirements

There is currently no CPU-only support.

Installation

Setup virtualenv

virtualenv venv_hmmr -p python3
source venv_hmmr/bin/activate
pip install -U pip
pip install numpy  # Some of the required packages need numpy to already be installed.
deactivate
source venv_hmmr/bin/activate
pip install -r requirements.txt

Install External Dependencies.

Neural Mesh Renderer and AlphaPose for rendering results:

cd src/external
sh install_external.sh

The above script also clones my fork of AlphaPose/PoseFlow, which is necessary to run the demo to extract tracks of people in videos. Please follow the directions in the installation, in particular running pip install -r requirements.txt from src/external/AlphaPose and downloading the trained models.

If you have a pre-installed version of AlphaPose, symlink the directory in src/external. The only change that my fork has is a very minor modification in AlphaPose/pytorch branch's demo.py: see this commit, copy over the changes in demo.py.

Demo

  1. Download the pre-trained models. Place the models folder as a top-level directory.
wget http://angjookanazawa.com/cachedir/hmmr/hmmr_models.tar.gz && tar -xf hmmr_models.tar.gz
  1. Download the demo_data videos. Place the demo_data folder as a top-level directory.
wget http://angjookanazawa.com/cachedir/hmmr/hmmr_demo_data.tar.gz && tar -xf hmmr_demo_data.tar.gz
  1. Run the demo. This code runs AlphaPose/PoseFlow for you. Please make sure AlphaPose can be run on a directory of images if you are having any issues.

Sample usage:

# Run on a single video:
python -m demo_video --vid_path demo_data/penn_action-2278.mp4

# If there are multiple people in the video, you can also pass a track index:
python -m demo_video --track_id 1 --vid_path demo_data/insta_variety-tabletennis_43078913_895055920883203_6720141320083472384_n_short.mp4

# Run on an entire directory of videos:
python -m demo_video --vid_dir demo_data/

This will make a directory demo_output/<video_name>, where intermediate tracking results and our results are saved as video, as well as a pkl file. Alternatively you can specify the output directory as well. See demo_video.py

Training code

Coming soon.

Data

InstaVariety

Insta-Variety Teaser

We provided the raw list of videos used for InstaVariety, as well as the pre-processed files in tfrecords. Please see doc/insta_variety.md for more details..

Citation

If you use this code for your research, please consider citing:

@InProceedings{humanMotionKZFM19,
  title={Learning 3D Human Dynamics from Video},
  author = {Angjoo Kanazawa and Jason Y. Zhang and Panna Felsen and Jitendra Malik},
  booktitle={Computer Vision and Pattern Recognition (CVPR)},
  year={2019}
You can’t perform that action at this time.