Question about pFedMeOptimizer.

[siddharthdivi]

pFedMe/FLAlgorithms/optimizers/fedoptimizer.py

Line 64 in 96863e0

p.data = p.data - group['lr'] * (p.grad.data + group['lamda'] * (p.data - localweight.data) + group['mu']*p.data)

Can you please explain the update in this line? I'm not sure what 'mu' signifies here, as it is not present in the algorithm given on page 5 of the corresponding paper?

[CharlieDinh]

Hi siddharthdivi,
Mu in here is a kind of regularization inside f_i(\theta).
For example: The local optimization problem for each client is defined as $theta_i = min( f_i(\theta_i) + \frac{\lamda}{2}(theta_i -w))$. I can consider the f_i(\theta_i) is cross entropy loss function with regularization $\frac{mu}{2}(theta_i)^2$ inside. So here $\mu$ is regularization term to make f_i function becomes strongly-convex

[siddharthdivi]

Thanks for the explanation!