uw initialization

[antct]

Hi, I found that the value -0.5 was used when initializing the parameter in line 19 of uw.py.
My doubt is why this value is not 0, since the variable loss_scale is equivalent to log \sigma in the original paper.

self.loss_scale = nn.Parameter(torch.tensor([-0.5]*self.task_num, device=self.device))

[Baijiong-Lin]

The initialization here refers to mtan. I think you can also use zero initialization.

[antct]

Thanks for your reply~