Skip to content

johnPertoft/smile

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Smile

f: 😒 → 😄

This repository contains Tensorflow implementations of some models dealing with image translation. Here they are applied to the problem of facial attribute editing (e.g. smile to non-smile and vice versa). Some models can only handle one attribute at a time and some can handle multiple.

Download and Prepare Dataset

For dataset split on a given feature. (Expected by CycleGAN, etc).

$ python -m smile.data.prepare.create_dataset --dataset-dir datasets/celeb --split-attribute Smiling

For dataset with attributes included in Tfrecords. (Expected by AttGAN, etc).

$ python -m smile.data.prepare.create_dataset --dataset-dir datasets/celeb --include-attributes

Results

CycleGAN

$ python -m smile.models.cyclegan.train \
    --x-train datasets/celeb/tfrecords/smiling/train/* \
    --x-test datasets/celeb/tfrecords/smiling/test/* \
    --y-train datasets/celeb/tfrecords/not_smiling/train/* \
    --y-test datasets/celeb/tfrecords/not_smiling/test/*

cyclegan

See more results and commands to recreate

AttGAN

$ python -m smile.models.attgan.train \
    --train-tfrecords datasets/celeb/tfrecords/all_attributes/train/* \
    --test-tfrecords datasets/celeb/tfrecords/all_attributes/test/* \
    --considered-attributes Smiling Male Mustache Blond_Hair

alt text

See more results and commands to recreate

StarGAN

$ python -m smile.models.stargan.train \
    --train-tfrecords datasets/celeb/tfrecords/all_attributes/train/* \
    --test-tfrecords datasets/celeb/tfrecords/all_attributes/test/* \
    --considered-attributes Smiling Male Mustache Blond_Hair

stargan

See more results and commands to recreate

UNIT

$ python -m smile.models.unit.train \
    --x-train datasets/celeb/tfrecords/smiling/train/* \
    --x-test datasets/celeb/tfrecords/smiling/test/* \
    --y-train datasets/celeb/tfrecords/not_smiling/train/* \
    --y-test datasets/celeb/tfrecords/not_smiling/test/* \
    --adversarial_loss lsgan

unit

See more results and commands to recreate

Celeb-A attribute ratios

On Off
5_o_Clock_Shadow 0.111 0.889
Arched_Eyebrows 0.267 0.733
Attractive 0.513 0.487
Bags_Under_Eyes 0.205 0.795
Bald 0.022 0.978
Bangs 0.152 0.848
Big_Lips 0.241 0.759
Big_Nose 0.235 0.765
Black_Hair 0.239 0.761
Blond_Hair 0.148 0.852
Blurry 0.051 0.949
Brown_Hair 0.205 0.795
Bushy_Eyebrows 0.142 0.858
Chubby 0.058 0.942
Double_Chin 0.047 0.953
Eyeglasses 0.065 0.935
Goatee 0.063 0.937
Gray_Hair 0.042 0.958
Heavy_Makeup 0.387 0.613
High_Cheekbones 0.455 0.545
Male 0.417 0.583
Mouth_Slightly_Open 0.483 0.517
Mustache 0.042 0.958
Narrow_Eyes 0.115 0.885
No_Beard 0.835 0.165
Oval_Face 0.284 0.716
Pale_Skin 0.043 0.957
Pointy_Nose 0.277 0.723
Receding_Hairline 0.08 0.92
Rosy_Cheeks 0.066 0.934
Sideburns 0.057 0.943
Smiling 0.482 0.518
Straight_Hair 0.208 0.792
Wavy_Hair 0.32 0.68
Wearing_Earrings 0.189 0.811
Wearing_Hat 0.048 0.952
Wearing_Lipstick 0.472 0.528
Wearing_Necklace 0.123 0.877
Wearing_Necktie 0.073 0.927
Young 0.774 0.226

Celeb-A attribute correlations

celeb-a-correlation

TODO

  • Add gif of progress samples.
  • Add evaluation method based on classifier of identities? sort of like inception score

Models / Papers

  • CycleGAN
  • AttGAN
  • UNIT (bad results, needs work)
  • StarGAN
  • Augmented CycleGAN
  • Sparsely Grouped GAN
  • Fusion GAN
  • DiscoGAN
  • MUNIT
  • XGAN
  • DTN

General

  • Spectral normalization
  • Progressive growing utility
  • Attention mechanism, see self-attention GAN
  • Facial landmarks as supervision
  • https://github.com/yingcong/Facelet_Bank
  • Simultaneous vs alternating gradient descent.
  • TTUR
  • Standardize architecture for comparisons. Densenet, resnet, unet.
  • Try other upsampling methods, see some checkerboarding sometimes. Or tune kernel sizes / strides.

Engineering

  • Docker for reproducing.
  • Add terraform/cloudformation scripts for cloud resource management.
  • Option to download dataset (and read from) cloud storage.