Skip to content
PyTorch implementation of our ICCV 2019 paper: Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis
Python Jupyter Notebook Shell
Branch: master
Clone or download
Latest commit 9ee738c Oct 10, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assets
data
doc ImPer-1.2.1, optimize the mininal GPU memory requirements Oct 4, 2019
models
networks update Oct 1, 2019
options ImPer-1.2.1 Sep 27, 2019
outputs ImPer-1.2.1 Sep 27, 2019
scripts
thirdparty/neural_renderer update v1.1 Sep 25, 2019
utils add utils/video.py Sep 27, 2019
.gitignore update Oct 1, 2019
LICENSE Create LICENSE Oct 10, 2019
README.md
demo_imitator.py Update demo_imitator.py Sep 29, 2019
demo_swap.py ImPer-1.2.1 Sep 27, 2019
demo_view.py
environment.yml
impersonator.ipynb
requirements.txt
run_imitator.py ImPer-1.2.1 Sep 27, 2019
run_swap.py ImPer-1.2.1, optimize the mininal GPU memory requirements Oct 4, 2019
run_view.py
train.py dev is under testing Sep 20, 2019

README.md

Impersonator

PyTorch implementation of our ICCV 2019 paper:

Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Please clone the newest codes.

[paper] [website]

Update News

  • 10/05/2019, optimize the minimal requirements of GPU memory (at least 3.8GB available).

Getting Started

Python 3.6+, Pytorch 1.2, torchvision 0.4, cuda10.0, at least 3.8GB GPU memory and other requirements. All codes are tested on Linux Distributions (Ubutun 16.04 is recommended), and other platforms have not been tested yet.

Requirements

pip install -r requirements.txt

Installation

cd thirdparty/neural_renderer
python setup.py install

Download resources.

  1. Download pretrains.zip from OneDrive or BaiduPan and then move the pretrains.zip to the assets directory and unzip this file.
wget -O assets/pretrains.zip https://1drv.ws/u/s!AjjUqiJZsj8whLNw4QyntCMsDKQjSg?e=L77Elv
  1. Download checkpoints.zip from OneDrive or BaiduPan and then unzip the checkpoints.zip and move them to outputs directory.
wget -O outputs/checkpoints.zip https://1drv.ws/u/s!AjjUqiJZsj8whLNyoEh67Uu0LlxquA?e=dkOnhQ
  1. Download samples.zip from OneDrive or BaiduPan, and then unzip the samples.zip and move them to assets directory.
wget -O assets/samples.zip https://1drv.ws/u/s!AjjUqiJZsj8whLNz4BqnSgqrVwAXoQ?e=bC86db

Running Demo

If you want to get the results of the demo shown on the webpage, you can run the following scripts. The results are saved in ./outputs/results/demos

  1. Demo of Motion Imitation

    python demo_imitator.py --gpu_ids 1
  2. Demo of Appearance Transfer

    python demo_swap.py --gpu_ids 1
  3. Demo of Novel View Synthesis

    python demo_view.py --gpu_ids 1

If you get the errors like RuntimeError: CUDA out of memory, please add the flag --batch_size 1, the minimal GPU memory is 3.8 GB.

Running custom examples (Details)

If you want to test other inputs (source image and reference images from yourself), here are some examples. Please replace the --ip YOUR_IP and --port YOUR_PORT for Visdom visualization.

  1. Motion Imitation

    • source image from iPER dataset
    python run_imitator.py --gpu_ids 0 --model imitator --output_dir ./outputs/results/  \
        --src_path      ./assets/src_imgs/imper_A_Pose/009_5_1_000.jpg    \
        --tgt_path      ./assets/samples/refs/iPER/024_8_2    \
        --bg_ks 13  --ft_ks 3 \
        --has_detector  --post_tune  \
        --save_res --ip YOUR_IP --port YOUR_PORT
    • source image from DeepFashion dataset
    python run_imitator.py --gpu_ids 0 --model imitator --output_dir ./outputs/results/  \
    --src_path      ./assets/src_imgs/fashion_woman/Sweaters-id_0000088807_4_full.jpg    \
    --tgt_path      ./assets/samples/refs/iPER/024_8_2    \
    --bg_ks 25  --ft_ks 3 \
    --has_detector  --post_tune  \
    --save_res --ip YOUR_IP --port YOUR_PORT
    • source image from Internet
    python run_imitator.py --gpu_ids 0 --model imitator --output_dir ./outputs/results/  \
        --src_path      ./assets/src_imgs/internet/men1_256.jpg    \
        --tgt_path      ./assets/samples/refs/iPER/024_8_2    \
        --bg_ks 7   --ft_ks 3 \
        --has_detector  --post_tune --front_warp \
        --save_res --ip YOUR_IP --port YOUR_PORT
  2. Appearance Transfer

    An example that source image from iPER and reference image from DeepFashion dataset.

    python run_swap.py --gpu_ids 0 --model imitator --output_dir ./outputs/results/  \
        --src_path      ./assets/src_imgs/imper_A_Pose/024_8_2_0000.jpg    \
        --tgt_path      ./assets/src_imgs/fashion_man/Sweatshirts_Hoodies-id_0000680701_4_full.jpg    \
        --bg_ks 13  --ft_ks 3 \
        --has_detector  --post_tune  --front_warp --swap_part body  \
        --save_res --ip http://10.10.10.100 --port 31102
  3. Novel View Synthesis

    python run_view.py --gpu_ids 0 --model viewer --output_dir ./outputs/results/  \
    --src_path      ./assets/src_imgs/internet/men1_256.jpg    \
    --bg_ks 13  --ft_ks 3 \
    --has_detector  --post_tune --front_warp --bg_replace \
    --save_res --ip http://10.10.10.100 --port 31102

If you get the errors like RuntimeError: CUDA out of memory, please add the flag --batch_size 1, the minimal GPU memory is 3.8 GB.

The details of each running scripts are shown in runDetails.md.

Training from Scratch

The details are shown in train.md [TODO].

Announcement

In our paper, the results of LPIPS reported in Table 1, are calculated by 1 – distance score; thereby, the larger is more similar between two images. The beginning intention of using 1 – distance score is that it is more accurate to meet the definition of Similarity in LPIPS.

However, most other papers use the original definition that LPIPS = distance score; therefore, to eliminate the ambiguity and make it consistent with others, we update the results in Table 1 with the original definition in the latest paper.

Citation

thunmbnail

@InProceedings{lwb2019,
    title={Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis},
    author={Wen Liu and Zhixin Piao, Min Jie, Wenhan Luo, Lin Ma and Shenghua Gao},
    booktitle={The IEEE International Conference on Computer Vision (ICCV)},
    year={2019}
}
You can’t perform that action at this time.