- Official PyTorch implementation of the paper Co-occurrence Based Texture Synthesis.
As image generation techniques mature, there is a growing interest in explainable representations that are easy to understand and intuitive to manipulate. In this work, we turn to co-occurrence statistics, which have long been used for texture analysis, to learn a controllable texture synthesis model. We propose a fully convolutional generative adversarial network, conditioned locally on co-occurrence statistics, to generate arbitrarily large images while having local, interpretable control over the texture appearance. To encourage fidelity to the input condition, we introduce a novel differentiable co-occurrence loss that is integrated seamlessly into our framework in an end-to-end fashion. We demonstrate that our solution offers a stable, intuitive and interpretable latent representation for texture synthesis, which can be used to generate a smooth texture morph between different textures. We further show an interactive texture tool that allows a user to adjust local characteristics of the synthesized texture image using the co-occurrence values directly.
Naive interpolation in RGB space
Interpolation in the co-occurrence space
- Python 3.7
- PyTorch (conda version recommended, pytorch-gpu = 1.3.1)
- torchvision, numpy, scipy, matplotlib, tqdm, pillow, os, scikit-learn
In order to install all the requirements via conda, just run the following command:
conda create -n cooc_texture python=3.7 pytorch-gpu=1.3.1 torchvision numpy scipy matplotlib tqdm pillow=6.1 scikit-learn
and then activate the environment:
conda activate cooc_texture
Run the following command for training:
python train_model.py --texturePath=samples/marbled_0095.jpg --imageSize=128 --kVal=4
Takes around 3 hours on a NVIDIA 1080Ti GPU, for an image crop size of 128x128 and kVal=4.
- Here, kVal is the number of cluster centers for co-occurrence calculation.
- You can use any image you want, and give the path in the format shown above.
python evaluate_model.py --texturePath=samples/marbled_0095.jpg --modelPath=results/marbled_0095_2020-03-02_23-02-09/ --checkpointNumber=120 --kVal=4 --outputFolder=eval_results/marbled_0095/ --evalFunc=f_d
Replace the modelPath with the actual path of the directory. The one included is just for representation purposes
modelPath
is the folder containing the checkpoint models.checkpointNumber
is the number of the epoch of which model we want to use.outputFolder
is the location where we want to save the evaluated results.evalFunc
is the selector between:f_d
= Fidelity and Diversity - saves an image showing the fidelity with respect to the input crop, and diversity with respect to random noise vector seeds.interp
= Interpolation - saves an image of the interpolation between two crops, by interpolating in the co-occurrence space and then generating the result.write_tex
= Large Image Generation with Control - saves a high resolution image generated from the input co-occurrence and writes the text '2020' on it.
Check config.py for all options
🎉 Thank you for Zalando SE, for providing for the code which served as a base for this project.
👍 The Describable Textures Dataset (DTD) for providing with a highly diverse dataset of images.
This project is licensed under the terms of the MIT license (see LICENSE
for details).