Can this be used inside GFP - GAN?

[glennois]

Hi Sergei, thank you for your amazing research and this repo. By following the readme, I was able to generate the CoreML model and I noticed that the generated model fully runs on ANE in my testing which is amazing.

So far so great. My goal is to use MobileStyleGAN inside GFP-GAN, which looks like this:

From the diagram, the green "Pretrained GAN as prior" is StyleGAN2 with some additions. Their StyleGAN2 has some additions called CS-SFT that transforms the output at each layer. This StyleGAN2 takes 2 inputs style & conditions:

style.shape = [
    (16,512)
]

conditions.shape = [
    (1,256,8,8),
    (1,256,8,8),
    (1,256,16,16),
    (1,256,16,16),
    (1,256,32,32),
    (1,256,32,32),
    (1,256,64,64),
    (1,256,64,64),
    (1,128,128,128),
    (1,128,128,128),
    (1,64,256,256),
    (1,64,256,256),
    (1,32,512,512),
    (1,32,512,512)
]

So, from what you can tell, is it possible to use MobileStyleGAN in GFP-GAN? I've been playing with your repo for the past few days. If you can advise on if this will work I would really appreciate it.

If you think it's not possible then I will stop here.

Thank you.

[bes-dev]

@glennios hey, thank you for your feedback. I think it possible to adopt MobileStyleGAN to work with GFP-GAN pipeline, but we didn't try it before.

[glennois]

Cool, I'll keep pushing forward. :) Thank you for your research.

[Harrypotterrrr]

Any progress on the thread ? @glennois
Or have you ever tried to build a gan inversion (encoder) part for the mobile-stylegan?

[NingNanXin]

@glennois
Hi, have you solved the problem? I try to combine the GFP_GAN with mobile StyleGAN, but I found the noises' shapes are different in this model. And this caused a problem in that GFP_GAN and mobile StyleGAN could not share the noise.