Poor results at 1024x1024

[DavidRees87]

I'm having issues training the same images that worked well at 512x512 at the higher resolution of 1024x1024. I've seen some other comments from people experiencing something similar. I've tried using 'fastgan_lite' but in colab I get this error message:

---

UnboundLocalError Traceback (most recent call last)
in ()
21 seed=0,
22 workers=0,
---> 23 restart_every=999999,
24 )

in train(**kwargs)
72 c.D_kwargs.backbone_kwargs.proj_type = 2
73 c.D_kwargs.backbone_kwargs.num_discs = 4
---> 74 c.D_kwargs.backbone_kwargs.separable = use_separable_discs
75 c.D_kwargs.backbone_kwargs.cond = opts.cond
76

UnboundLocalError: local variable 'use_separable_discs' referenced before assignment

Any ideas how to resolve this and get better training results at 1024?

Thank you

[DavidRees87]

Any thoughts on this? Thank you

[woctezuma]

The error happens here, because the variable use_separable_discs is not initialized.

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L248

However, this variable should exist if you use one of the 3 available configs (['fastgan', 'fastgan_lite', 'stylegan2']).

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L204-L207
https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L209-L213

Do you use another config than these?

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L130

[DavidRees87]

Thanks for the reply. No, I'm not using anything different. It works just fine when I use --cfg = 'fastgan' but as soon as I try and use --cfg = 'fastgan_lite' I get that error message.

Everything else is the same and up to date, I can see in 'train.py' the fastgan_lite config is there but still get the issue.

Tried again just now and still happening.

[woctezuma]

Actually, there could be a bug due to _ and -.

For instance, the argument --batch-gpu:

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L141

is interpreted as batch_gpu:

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L181

See also the mention of --batch-gpu and batch_gpu here:

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L197-L198

[woctezuma]

Try replacing:

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L209

with:

elif opts.cfg in ['fastgan', 'fastgan_lite', 'fastgan-lite']:

---

And replacing:

https://github.com/autonomousvision/projected_gan/blob/e1c246b8bdce4fac3c2bfcb69df309fc27df9b86/train.py#L211

with:

c.G_kwargs.synthesis_kwargs.lite = (opts.cfg == 'fastgan_lite') or (opts.cfg == 'fastgan-lite')

---

Otherwise, check with a debugger the value of opts.cfg.

[DavidRees87]

Ah, I've figured out what was wrong! It wasn't an issue of confusing the _ with -.

This part of the code is in the 'train.py' but it isn't in the notebook cell you run before training, the "fastgan_lite" part is missing as well as another line. Once I copied across the update in 'train.py' it has started training properly using fastgan_lite.

Thanks for your help, hopefully this will help improve the results I get at 1024x1024!

Cheers