multi_scale_deep_texture

This is a Tensorflow implementation of multi-scale texture synthesis algorithm based on Gatys' deep texture algorithm in paper Texture Synthesis Using Convolutional Neural, and the ideas of synthesizing texture in a coarse-to-fine hierarchical pipeline by building image pyramid has been presented in Gatys' paper Controlling Perceptual Factors in Neural Style Transfer.

The texture synthesis using CNNs proposed by Gatys can produce impressive results given an stochastic textures. However, when the texture image size is larger (e.g 2K, 4K and even higher resolution), the results will lack large scale structure and present as so-called Texture soup due to the restriction of receptive field size of VGG network.

As will be shown below, the synthesis results using multi-scale pipeline will produce better results in maintain the large scale structure of texture. First, we down sample the texture sample image in size $[w,h]$ by a ratio 2 over $L-1$ times, thus build a $L$ layer image pyramid, in which the original texture image will be on $l_{L-1}$ layer, and the $L_0$ layer with minimum resolution $[\frac{w}{L-1}, frac{h}{L-1}]$. Second, we initial a noise image in size $[\frac{w}{L-1}, frac{h}{L-1}]$, and synthesize it by optimizing the Gram Loss computed with texture sample in $l_{L-1}$ layer. Third, we up sample the synthesized texture by ratio 2, and continue to optimize the Gram Loss, until the current texture size reach target size.

Multi-scale synthesis pipeline is condusive to maintain the large scale structure

Texture image	Synthesized image
	L = 1
	L = 3
	L = 6

Results

Texture image	Synthesized image

Dependency

• python3.5
• tensoflow (tested on r1.4)
• VGG19 model weights download from the TF-slim models
• The code is tested on:
  • Ubuntu 16.04 LTS with CPU architecture x86_64 + Nvidia GeForce GTX 1080

Recommended

• Ubuntu 16.04 with TensorFlow GPU edition

Getting started

Denote the directory of this repository as ./multi_scale_deep_texture/

• Download the VGG19 pre-trained model

# clone the repository from github
git clone https://github.com/LiaoQian1996/multi_scale_deep_texture.git
cd $multi_scale_deep_texture/

# download the vgg19 model weights from 
# http://download.tensorflow.org/models/vgg_19_2016_08_28.tar.gz
# to multi_scale_deep_texture/vgg19/

• Synthesis your texture image

modify the parameters in run_syn.sh

CUDA_VISIBLE_DEVICES=0 python main.py \
    --output_dir results \
    --target_dir ./imgs/fire.png \  # texture sample image path
    --texture_shape -1 -1 \         # texture shape will be as same as the sample image if [-1,- 1]
    --top_style_layer VGG54 \
    --max_iter 50 \
    --pyrm_layers 6 \
    --W_tv 0.001 \
    --vgg_ckpt ./vgg19/ \
    #--print_loss 

then run the shell script

sh run_syn.sh