High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
Lua
Switch branches/tags
Nothing to show
Clone or download
leehomyc Merge pull request #34 from merofeev/master
Switch content network to evaluation mode at test stage.
Latest commit e910854 May 29, 2018
Permalink
Failed to load latest commit information.
examples remove DS_Store Feb 21, 2017
gt update Feb 21, 2017
images update Feb 10, 2017
models update models Feb 21, 2017
mylib small fix Sep 17, 2017
.gitignore remove DS_Store Feb 21, 2017
LICENSE Initial commit Jan 26, 2017
README.md Update README.md Oct 16, 2017
blend.lua Update blend.lua Sep 26, 2017
mask.png update Feb 21, 2017
run_content_network.lua Switch content network to evaluation mode at test stage. May 29, 2018
run_texture_optimization.lua add fast inpainting Sep 17, 2017
transfer_CNNMRF_wrapper.lua remove redundent Sep 17, 2017
util.lua update Feb 20, 2017

README.md

High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis

[update 10/10/2017] Example of photo editing using inpainting at the project website.

[update 9/30/2017] We shared the inpainting result of 200 ImageNet images and 100 Paris StreetView Images at the project website.

[update 9/16/2017] We increase the speed of original version by 6x (30s/image on GPU).

[update 9/16/2017] All raw images of the paper and supplementary material (including input, output and intermediate results) are available at the project website.

teaser

[Faster Inpainting Updates]:

  1. We remove layer 12 of vgg in texture optimization.
  2. In texture optimization, we have three scales and 100 iterations at each scale. Now it only computes the nearest patch at the first iteration of each scale, and re-use the nearest index in later iterations.

This greatly increases the speed at the cost of very subtle inpainting quality.

This is the code for High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis. Given an image, we use the content and texture network to jointly infer the missing region. This repository contains the pre-trained model for the content network and the joint optimization code, including the demo to run example images. The code is adapted from the Context Encoders and CNNMRF. Please contact Harry Yang for questions regarding the paper or the code. Note that the code is for research purpose only.

Demo

(The code may give better results when supplying 32 different images (a batch) as input to the content network rather than using one single image as input. This may be due to the mismatch of the training and testing coefficients in batch normalization.)

  git clone https://github.com/leehomyc/High-Res-Neural-Inpainting.git
  • Download the pre-trained models for the content and texture networks and put them under the folder models/.

  • Run the Demo

  cd High-Res-Neural-Inpainting
  # This will use the trained model to generate the output of the content network
  th run_content_network.lua
  # This will use the trained model to run texture optimization
  th run_texture_optimization.lua
  # This will generate the final result
  th blend.lua

Citation

If you find this code useful for your research, please cite:

@InProceedings{Yang_2017_CVPR,
author = {Yang, Chao and Lu, Xin and Lin, Zhe and Shechtman, Eli and Wang, Oliver and Li, Hao},
title = {High-Resolution Image Inpainting Using Multi-Scale Neural Patch Synthesis},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}