StyleGAN2 is capable of generating natural looking images out of nothing - even images that haven't been discovered yet. HR_Encoder extends Alberto Rosas Garcia's unofficial StyleGAN2-Tensorflow-2.x implementation to find your own images in StyleGAN2's W space.
Legal information
HR_Encoder.ipynb, HR_Encoder.py and w_utilities.py are Copyright © 2020 HANS ROETTGER and distributed under the terms of AGPLv3.
Severability clause: If a provision of the AGPLv3 is or becomes illegal, invalid or unenforceable in any jurisdiction, that shall not affect:
- the validity or enforceability in that jurisdiction of any other provision of the AGPLv3; or
- the validity or enforceability in other jurisdictions of that or any other provision of the AGPLv3.
Images used to showcase HR_Encoder are copyrighted by their respective owners and were licensed by them CC0/PD/pixabay: 1 2 3 4 5 6 7
.jpg){kind=link}
Encoding into StyleGAN2's W space is not lossless. The figure shows the differences between an encoded and the original image.
  |
 |
  |
|
Examples shown above are generated in the jupyter notebook. First column visualizes encoding of a portrait and ease of manipulation in W space (sunglasses added or image style changed by one line of code). Middle shows the result of morphing between two encoded images in W space. Finally encoding results for further StyleGAN2 domain models.
(High Resolution) HR_Encoder calculates latent codes w⃗∈W for your images
What for? HR_Encoder transfers your images into StyleGAN2's W space and enables you to apply state of the art image manipulation to your own material. Simple computations on latent codes w⃗∈W create fantastic effects in the StyleGAN2-generated images.
What is new? In contrast to Dmitry Nikitko's original encoder, HR_Encoder uses the StyleGAN2 discriminator model itself instead of the VGG16 model to preserve high level image features during encoding - resulting in latent codes w⃗ for very sharp and detailed images.
BTW: this approach leads to a more consice algorithm without the need of complicated image similarity metrics.
Is the encoding lossless? No, not at all. But StyleGAN2 generates naturally looking images very similar to your input images, although details might be missing or could be artificially invented.
Requirements? HR_Encoder requires a tensorflow2/Jupyter environment and uses up to 8 GB of RAM or GPU memory. It is possible to run the algorithms on a CPU without Nvidia GPU/ Cuda support - although it takes plenty of time (see run time estimations in the model descriptions).
# get Alberto Rosas Garcia's StyleGAN2 implementation
git clone https://github.com/rosasalberto/StyleGAN2-TensorFlow-2.x.git
# from https://drive.google.com/drive/folders/1rhuvN90EGsRhvjQq5gio8VYw7f0LojaK?usp=sharing
# download the StyleGAN2 models (all files) to ./StyleGAN2-TensorFlow-2.x/weights/
# get HR_encoder
git clone https://github.com/oss-roettger/HR_Encoder.git
# copy HR_Encoder files to the StyleGAN2 directory
# !!! stylegan2_discriminator.py was modified and will be replaced
\cp -rf ./HR_Encoder/* ./StyleGAN2-TensorFlow-2.x/
# Recommendation: run the project in a docker container preconfigured for tensorflow2 -e.g. w/o GPU:
/usr/bin/docker run -it --rm -v $(realpath ./StyleGAN2-TensorFlow-2.x):/tf/HR_Encoder -p 8888:8888 tensorflow/tensorflow:2.1.1-jupyter
# Finally: start HR_Encoder.ipynb [Kernal/Restart & Run All]
If you run HR_Encoder with GPU/CUDA support, the tensorflow library might raise a "ressource exhausted" error from time to time. This seems to be the case when the GPU memory is to fragmented to initialize and optimize the huge StyleGAN2 model. Work around:
- try to restart the python/ jupyter kernel and run again
- restart your tensorflow environment (e.g. the docker container) and run again
- restart your system