Skip to content
Code repo for 'Learning Selfie-Friendly Abstraction from Artistic Style Images' (ACML 2018).
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
include
model/vgg_16layers
pics
src add initial files May 22, 2018
test
train
utils
LICENSE
README.md

README.md

Learning Selfie-Friendly Abstraction from Artistic Style Images

Yicun Liu | Jimmy Ren | Jianbo Liu | Jiawei Zhang | Xiaohao Chen

ACML 2018


This repository contains code for the paper: Learning Selfie-Friendly Abstraction from Artistic Style Images.

Contact: Yicun Liu (stanleylau@link.cuhk.edu.hk)

Prerequisites

The code is tested on 64 bit Linux (Ubuntu 14.04 LTS). You should also install Matlab (We have tested on R2015a). We have tested our code on GTX TitanX (but it can also run on other GPUs with vRAM >= 2G) with CUDA8.0+cuDNNv5. Please install all these prerequisites before running our code.

Installation

  1. Clone the code.

    git clone https://github.com/DandilionLau/Selfie-Friendly-Abstraction.git 
    cd Selfie-Friendly-Abstraction  
  2. Build standard caffe follow the instruction.

    cd caffe/
    # Modify Makefile.config according to your Caffe installation. 
    # Remember to allow CUDA and CUDNN.
    make -j12
    make matcaffe
  3. Cutomize caffe.

    Add the following message to caffe.proto to configure the new NNUp Upsample Layer:

    Message Parameter{
    // Insert to existing class. Try not to conflict with any existing message numbers.
      optional NNUpsampleParameter nn_upsample_param = 163;
    }
    message NNUpsampleParameter {
    // Append to the last of caffe.proto file
      optional uint32 resize = 1 [default = 2];
    }

    Copy include/nn_upsample_layer.hpp to caffe/include/caffe/layers/. Copy src/nn_upsample_layer.cpp and src/nn_upsample_layer.cu to caffe/src/caffe/layers/. Then recompile both caffe and matcaffe.

    make -j12
    make matcaffe
  4. Download training models.
    To prepare for the testing step, you may simply download the trained caffemodels from [DropBox][BaiduYun] and put them to the model/style/ directory .

    Additionally, if you want to train your own style abstraction model, you need to download the VGG-16 model from [VGG Website][DropBox][BaiduYun] to compute the perceptual loss. It should be put to the model/vgg_16layers/ directory.

Training

  1. Generate image patches
    Run data/GenPatches_train_6chs.m and data/GenPatches_val_6chs.m at MATLAB to extract image patches for training and validation. We provide 40 selfie images and their corresponding output images generated from Prisma. The selfie image and stylistic reference image directory is at data/training/. You may replace it with your own dataset to train with different styles.
  2. Training the model
    Run train/train_6chs_reshape.m at MATLAB to train the model. Remember to include matcaffe before training. In our experiment, the balance factor between loss_pixel and loss_feat is set as 1000.

Testing

  1. Run test/test_6chs_reshape.m at MATLAB to test the model. Remember to include matcaffe before runing the test. We provide 99 images from Flickr for testing, including portraits, landscapes, wild lifes and other scenes. The image directory is at data/testing/. You may replace it with your own dataset for testing.
  2. For inter-frame consitency test, please visit our online demo to check the results.

Results

Citation

Please cite our paper if you find it helpful for your work:

@article{liu2018learning,
  title={Learning Selfie-Friendly Abstraction from Artistic Style Images},
  author={Liu, Yicun and Ren, Jimmy and Liu, Jianbo and Zhang, Jiawei and Chen, Xiaohao},
  journal={arXiv preprint arXiv:1805.02085},
  year={2018}
}
@InProceedings{pmlr-v95-liu18a,
title = {Learning Selfie-Friendly Abstraction from Artistic Style Images},
author = {Liu, Yicun and Ren, Jimmy and Liu, Jianbo and Chen, Xiaohao},
booktitle = {Proceedings of The 10th Asian Conference on Machine Learning},
year = {2018}
}
You can’t perform that action at this time.