How to use W+ space to generate samples from MobileStyleGAN

[RahulBhalley]

Hi!

I want to sample images from W+ space using PyTorch checkpoints. But there doesn't seem to exist any argument to generate.py script for that. Could you please guide me regarding this?

The images I sampled from CoreML's W+ space models (using both Mapping and Synthesis) were weirdly in bluish color. These models were exported using --export-w-plus argument. I've attached few of them here.

When I use W space in CoreML models then the samples are colored correctly.

Any help is highly appreciated!

Regards
Rahul Bhalley

[bes-dev]

@RahulBhalley how do you make samples from W+ space? W+ is good for optimization, but it is much worse for random sampling than W-space.

[RahulBhalley]

@RahulBhalley how do you make samples from W+ space?

@bes-dev With the --export-w-plus, I get MappingNetwork with (23, 512) I/O tensors shapes. I pass z sampled from Gaussian distribution with this shape to MappingNetwork. The output is then fed to SynthesisNetwork with un-squeezed zeroth-dimension. Finally, I produce the image using tensor_to_img() function from your code.

W+ is good for optimization, but it is much worse for random sampling than W-space.

Interesting, I didn't know about that.

My actual goal is to perform face manipulation using W+ space. I have few queries before I proceed with that code just to save my time:

Q1. So, if I use encoder from ReStyle paper then will MobileStyleGAN be able to reconstruct the image correctly? I have this query because their encoder was trained using StyleGAN2's generator as loss function. (I'm aware that I've to add latent average tensor to W+ latent code before feeding it to the generator.)

Q2. Regarding Q1, should I use W space MappingNetwork and then tile the generated W latent code to match the input for SynthesisNetwork's W+ space input or should I use W+ space MappingNetwork directly?

Thanks for quick response! I love ❤️ your work. It's really amazing to be able to run compressed StyleGAN2 on-device!

Regards
Rahul Bhalley

[bes-dev]

@RahulBhalley Unfortunately, our W+ space is not exactly equal to W+ space of the original StyleGAN2. So, I'm not sure that models like ReStyle would works good with MobileStyleGAN without finetuning. For second question I think it will better to check the both ways.

[RahulBhalley]

Okay, I'll try fine-tuning ReStyle's encoder using MobileStyleGAN.
I'm closing this issue. If I need help, I'll open an issue related to fine-tuning ReStyle's encoder.

Thanks for conversation! 🙂