Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

pix2latent: framework for inverting images into generative models

Framework for inverting images. Codebase used in:

Transforming and Projecting Images into Class-conditional Generative Networks
project page | paper
Minyoung Huh   Richard Zhang   Jun-Yan Zhu   Sylvain Paris   Aaron Hertzmann
MIT CSAIL   Adobe Research
ECCV 2020 (oral)

    title = {Transforming and Projecting Images to Class-conditional Generative Networks},
    author = {Minyoung Huh and Richard Zhang, Jun-Yan Zhu and Sylvain Paris and Aaron Hertzmann},
    booktitle = {ECCV},
    year = {2020}

NOTE [8/25/20] The codebase has been renamed from GAN-Transform-and-Project to pix2latent, and also refactored to make it easier to use and extend to any generative model beyond BigGAN. To access the original codebase refer to the legacy branch.

Example results

All results below are without fine-tuning.

BigGAN (z-space) - ImageNet (256x256)

StyleGAN2 (z-space) - LSUN Cars (384x512)

StyleGAN2 (z-space) - FFHQ (1024x1024)


The code was developed on

  • Ubuntu 18.04
  • Python 3.7
  • PyTorch 1.4.0

Getting Started

  • Install PyTorch
    Install the correct PyTorch version for your machine

  • Install the python dependencies
    Install the remaining dependencies via

    pip install -r requirements.txt
  • Install pix2latent

    git clone
    cd pix2latent
    pip install .


We provide several demo codes in ./examples/ for both BigGAN and StyleGAN2. Note that the codebase has been tuned and developed on BigGAN.

> cd examples
> python --num_samples 4

Using the make_video flag will save the optimization trajectory as a video.

> python --make_video --num_samples 4

(slow) To optimize with CMA-ES or BasinCMA, we use PyCMA. Note that the PyCMA version of CMA-ES has a predefined number of samples to jointly evaluate (18 for BigGAN) and (22 for StyleGAN2).

> python 
> python 

(fast) Alternatively CMA-ES in Nevergrad provides sample parallelization so you can set your own number of samples. Although this runs faster, we have observed the performance to be slightly worse. (warning: performance depends on num_samples).

> python --ng_method CMA --num_samples 4
> python --ng_method CMA --num_samples 4

Same applies to StyleGAN2. See ./examples/ for extensive list of examples.

Template pseudocode

import torch, torch.nn as nn
import pix2latent.VariableManger
from pix2latent.optimizer import GradientOptimizer

# load your favorite model
class Generator(nn.Module):
    def forward(self, z):
        return im

model = Generator() 

# define your loss objective .. or use the predefined loss functions in pix2latent.loss_functions
loss_fn = lambda out, target: (target - out).abs().mean()

# tell the optimizer what the input-output relationship is
vm = VariableManager()
vm.register(variable_name='z', shape=(128,), var_type='input')
vm.register(variable_name='target', shape(3, 256, 256), var_type='output')

# setup optimizer
opt = GradientOptimizer(model, vm, loss_fn)

# optimize
vars, out, loss = opt.optimize(num_samples=1, grad_steps=500)

detailed usage


Command Description
pix2latent.loss_function predefined loss functions
pix2latent.distribution distribution functions used to initialize variables


class variable for managing variables. variable manager instance is initialized by

var_man = VariableManager()
Method Description
var_man.register(...) registers variable. this variable is created when initialize is called
var_man.unregister(...) removes a variable that is already registered
var_man.edit_variable(...) edits existing variable
var_man.initialize(...) initializes variable from defined specification


Command Description
pix2latent.optimizer.GradientOptimizer gradient-based optimizer. defaults to optimizer defined in pix2latent.VariableManager
pix2latent.optimizer.CMAOptimizer uses CMA optimization to search over latent variables z
pix2latent.optimizer.BasinCMAOptimizer uses BasinCMA optimization. a combination of CMA and gradient-based optimization
pix2latent.optimizer.NevergradOptimizer uses Nevergrad library for optimization. supports most gradient-free optimization method implemented in Nevergrad
pix2latent.optimizer.HybridNevergradOptimizer uses hybrid optimization by alternating gradient and gradient-free optimization provided by Nevergrad


Command Description
pix2latent.SpatialTransform spatial transformation function, used to optimize for image scale and position
pix2latent.TransformBasinCMAOptimizer BasinCMA-like optimization method used to search for image transformation


Command Description
pix2latent.util.image utility for image pre and post processing utility for video (e.g. saving videos)
pix2latent.util.misc miscellaneous functions
pix2latent.util.function_hooks function hooks that can be attached to variables in the optimization loop. (e.g. Clamp, Perturb)


Command Description
pix2latent.model.BigGAN BigGAN model wrapper. Uses implementation by huggingface using the official weights
pix2latent.model.StyleGAN2 StyleGAN2 model wrapper. Uses PyTorch implementation by rosinality using the official weights


Command Description
pix2latent.edit.BigGANLatentEditor BigGAN editor. Simple interface to edit class and latent variables using oversimplified version of GANSpace


Code for: Transforming and Projecting Images into Class-conditional Generative Networks








No releases published


No packages published