PAINT: Photo-realistic Fashion Design Synthesis

Arxiv Paper | BibTex

Introduction

In this paper, we investigate a new problem of generating a variety of multi-view fashion designs conditioned on a human pose and texture examples of arbitrary sizes, which can replace the repetitive and low-level design work for fashion designers. To solve this challenging multi-modal image translation problem, we propose a novel Photo-reAlistic fashIon desigN synThesis (PAINT) framework, which decomposes the framework into three manageable stages.

Our proposals combine these three stage by,

1. Layout Generative Network (LGN). we employ a Layout Generative Network (LGN) to transform an input human pose into a series of person semantic layouts.
2. Texture Synthesis Network (TSN). we propose a Texture Synthesis Network (TSN) to synthesize textures on all transformed semantic layouts. Specifically, we design a novel attentive texture transfer mechanism for precisely expanding texture patches to the irregular clothing regions of the target fashion designs.
3. Appearance Flow Network (AFN). we leverage an Appearance Flow Network (AFN) to generate the fashion design images of other viewpoints from a single-view observation by learning 2D multi-scale appearance flow fields.

Requirement

• pytorch(1.1.0)
• torchvision
• numpy
• scipy
• scikit-image
• pillow
• pandas
• tqdm
• dominate

Getting Started

Data Preperation

We provide our dataset files , extracted keypoints files and extracted parsing files for convience.

Fashion-Gen

• Download the Fashion-Gen dataset from here.
• Download train/test splits from here, including all_train.txt, all_test.txt.
• We use OpenPose to generate keypoints.Download the keypoints files from here.
• We use Human Parser to generate human parsing.Download the human parsing files from here.

Train the model

• Run LGN python train.py --dataroot dataset_root --name LGN --model stage1_gan --direction AtoB .
• Run TSN python train.py .
• Run AFN python train.py .

Test the model

• Run LGN python test.py --dataroot dataset_root --name LGN --model stage1_gan --direction AtoB .
• Run TSN python test.py.
• Run AFN python test.py.

Pretrained models

Download the models below and put it under release_model/

LGN | TSN | AFN

TensorBoard

Visualization on TensorBoard for training is supported.

Run TSN tensorboard --logdir release_model --port 6006 to view training progress.

Run AFN tensorboard --logdir release_model --port 6006 to view training progress.

Example Results

Citation

If any part of our paper and code is helpful to your work, please generously cite with: