Skip to content
The source code of "single-reference color transfer" in "Progressive Color Transfer with Dense Semantic Correspondences".
C++ Jupyter Notebook Python Cuda CMake C Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
code Minor fix for visualization. Jul 19, 2019
demo
README.md Added the figure of intro. Jul 23, 2019
intro.jpg Added intro.jpg. Jul 23, 2019

README.md

Neural Color Transfer

This is the implementation of single-reference color transfer proposed in the paper Progressive Color Transfer with Dense Semantic Correspondences by Mingming He, Jing Liao, Dongdong Chen, Lu Yuan and Pedro V. Sander in ACM Transactions on Graphics (2019).

Introduction

Neural Color Transfer is a progressive color transfer framework, which jointly optimizes dense semantic correspondencesin the deep feature domain and the local color transfer inthe image domain.

image

Given two input images (one color source image S and one color reference image) which share semantically-related content, but may vary dramatically in appearance or structure, the proposed framework first estimates dense correspondence between them using deep features (extracted from VGG19 at level L) and then applies local color transfer to the source image S based on the correspondence. The process repeats from high level (L=5) to low level (L=1).

For more results, please refer to our Supplementary.

Disclaimer

This is a C++ combined with CUDA implementation. It is worth noticing that:

  • The codes are built based on Caffe and Deep Image Analogy.
  • The codes only have been tested on Windows 10 and Windows Server 2012 R2 with CUDA 8 or 7.5.
  • The codes only have been tested on several Nvidia GPU: Titan X, Titan Z, K40, GTX1070, GTX770.
  • The size of input image is limited, mostly the longer side of input images are around 700 and should not be large than 1000.

Getting Started

Prerequisites

Build

The codes requires compiling in Visual Studio as follows:

  • Build Caffe at first. Just follow the tutorial here.
  • Open solution Caffe under code\windows\ and add neural_color_transfer.vcxproj under code\Windows\neural_color_transfer\.
  • Set <MKL_DIR> to the installation directory of Intel® Parallel Studio XE for Windows, e.g., <MKL_DIR>..\..\..\NugetPackages\mkl\compilers_and_libraries_2018.1.156\windows</MKL_DIR>.
  • Set <OPENCV_DIR> to the directory of OpenCV library, e.g., <OPENCV_DIR>..\..\..\NugetPackages\OpenCV.2.4.10\build\native</OPENCV_DIR>.
  • Build project neural_color_transfer.vcxproj.

Download Models

You need to download models before running a demo.

  • Go to demo\model\vgg19\ folder and download model.

Demo

We prepare an example under the folder demo\ with:

(1) Input data folder example\ including two parts:

  • A folder input\ with the input images (color source images and color reference images) inside.
  • A file pairs.txt to specify a source, a reference and a BDS weight (2.0 as default) as an example in each line, e.g.,
    in/in0.png in/tar0.png 2.0
    in/in1.png in/tar1.png 2.0
    ...
    

(2) Executable script run.bat including one command:

neural_color_transfer.exe -m [MODEL_DIR] -i [INPUT_ROOT_DIR] -o [OUTPUT_DIR] -g [GPU_ID]
e.g., exe\neural_color_transfer.exe -m model\ -i example\ -o example\res\ -g 0

Run

We provide pre-built executable files exe. Please download and unzip in folder demo\, and try them.

Tips

  • The proposed algorithm is more applicable to transfer color between images with semantically-related content.
  • Ajusting the parameter of BDS weight (defined in pairs.txt, regularly in [0, 4]) according to similarity of input images leads to better results if their content are not highly related, smaller value for less similar pair.

Citation

If you find Neural Color Transfer helpful for your research, please consider citing:

@article{he2019progressive,
  title={Progressive color transfer with dense semantic correspondences},
  author={He, Mingming and Liao, Jing and Chen, Dongdong and Yuan, Lu and Sander, Pedro V},
  journal={ACM Transactions on Graphics (TOG)},
  volume={38},
  number={2},
  pages={13},
  year={2019},
  publisher={ACM}
}
You can’t perform that action at this time.