compute_mean_style method generates gray image

[RahulBhalley]

Hi @bes-dev,

In issue #36 you told me that compute_mean_style method can be used to compute latent average vector for MobileStyleGAN. But the image produced using this vector via student network is all gray.

I ran the following code to produce image using latent average vector.

# Import libraries.
import cv2
from core.utils import load_cfg, load_weights, tensor_to_img
from core.distiller import Distiller

# Load configuration.
cfg = load_cfg("configs/mobile_stylegan_ffhq.json")

distiller = Distiller(cfg)
style_mean = distiller.compute_mean_style(style_dim=512, wsize=23)
out = distiller.student(style_mean)['img']
cv2.imwrite('style_mean.png', tensor_to_img(out[0]))

I get the following average image as output.

Furthermore, every call to compute_mean_style returns a different vector. It's very weird. Shouldn't it be the same? Also larger batch_size doesn't make any difference. :(

style_mean1 = distiller.compute_mean_style(style_dim=512, wsize=23)
style_mean2 = distiller.compute_mean_style(style_dim=512, wsize=23)
style_mean3 = distiller.compute_mean_style(style_dim=512, wsize=23)

# Comparing different means. 😅 Shouldn't they all be the same 'cause it's just an average. right?
style_mean1 == style_mean2
style_mean1 == style_mean3
style_mean2 == style_mean3

Output

tensor([[[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]]])
tensor([[[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]]])
tensor([[[False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         ...,
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False],
         [False, False, False,  ..., False, False, False]]])

I could be missing something. Please let me know what could be the issue on my side.

Regards
Rahul Bhalley

[bes-dev]

@RahulBhalley did you load pre-trained checkpoint?

[RahulBhalley]

@bes-dev after running the following lines of code

cfg = load_cfg("configs/mobile_stylegan_ffhq.json")
distiller = Distiller(cfg)

I get the following output.

load mapping network...
load pretrained model: stylegan2_ffhq_config_f_mapping_network.ckpt...
Computing MD5: /tmp/stylegan2_ffhq_config_f_mapping_network.ckpt
MD5 matches: /tmp/stylegan2_ffhq_config_f_mapping_network.ckpt
load synthesis network...
load pretrained model: stylegan2_ffhq_config_f_synthesis_network.ckpt...
Computing MD5: /tmp/stylegan2_ffhq_config_f_synthesis_network.ckpt
MD5 matches: /tmp/stylegan2_ffhq_config_f_synthesis_network.ckpt
/Users/codeboy/Freelancing/SocialKit/face-app/restyle-encoder-light/venv/lib/python3.9/site-packages/torchvision/models/inception.py:80: FutureWarning: The default weight initialization of inception_v3 will be changed in future releases of torchvision. If you wish to keep the old behavior (which leads to long initialization times due to scipy/scipy#11299), please set init_weights=True.
  warnings.warn('The default weight initialization of inception_v3 will be changed in future releases of '

It looks like the checkpoints are loaded (for mapping_net also).

[RahulBhalley]

I tried using distiller.style_mean but still I get gray image.

out = distiller.student(distiller.style_mean)['img']

[RahulBhalley]

Even truncation trick doesn't produce face image (but just gray).

var = torch.randn(1, 512)
style = distiller.mapping_net(var)
style = distiller.style_mean + 0.5 * (style - distiller.style_mean)
out = distiller.student(style)['img']

[bes-dev]

@RahulBhalley our code doesn't load pre-trained student automatically, you should load it on your side. Please, look at this sample of code:

MobileStyleGAN.pytorch/demo.py

Line 14 in a9776ff

if args.ckpt != "":

[RahulBhalley]

Brilliant! Dumb me. 😅 I forgot looking at demo code. Thank you! 🙏