New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Does the backend handle Federated learning asynchronously? #47
Comments
Hi, we have the example config to run asyncFL here and the code for async is here |
Hello!
Thank you for the reply. I have one more question. I have been trying to
understand the algorithm in the paper. So please correct me if I am wrong
and I kindly request to break it down in much simpler words and terms.
Here's what I understood (ignoring the differential privacy and SecAgg):
1. Every client will train for Q epochs. After that the difference between
the weights of the previously averaged model (yo=w) and this new locally
trained model (yo-yq) is submitted to the cloud. This is now called as
delta or client update. (∆i)
2. Once the server gets K=10 such client updates it then simply adds (∆ t ←
∆ t + ∆i) and divides it by K (∆ t ← ∆ t /K) meaning that it has taken the
average of all the client updates
3. After this point I am a bit lost. This expression does not make sense
to me : (w t+1 ← w t − ηg∆ t) so you multiply the averaged client update
with the server learning rate which is fine but then take the difference of
the global model weights and ηg∆ t ??? And now these are the new weights
of the global model?? Shouldn't it be just addition instead of
subtraction?? Cause learning rate is a positive value unless the averaged
delta or averaged client update is negative , then in that case
negative*negative makes it positive ?? Can you please clarify and please do
correct me if I am going wrong in understanding the overall algorithm. I
really want to understand this in depth.
Thank you
Regards
Kaushal Rathi
…On Tue, Aug 2, 2022 at 9:15 AM John Nguyen ***@***.***> wrote:
Closed #47 <#47> as
completed.
—
Reply to this email directly, view it on GitHub
<#47 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AVWGRJ6ZZXP4LYEIHFFPEG3VXFCS5ANCNFSM55ITAZFQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I found this repo from this blog: - https://ai.facebook.com/blog/asynchronous-federated-learning/
However I do not find any mentioning on this repo and also I cannot decipher from the code examples whether this is synchronous version or asynchronous version of Federated learning?
Can you please clarify this for me? And also if this is the asynchronous version how can I dive deeper in to the libraries and look at the code of implementation for the asynch handling mechanism?
Thank you
The text was updated successfully, but these errors were encountered: