The Reason of Changing Total Number of Clients

[hiyuchang]

Thank you for your nice codes.

I have a question about main_fedavg.py#L115:

args.client_num_in_total = client_num

While loading data, why should we change args.client_num_in_total = client_numinto 1000? What is the intuition behind this?

[chaoyanghe]

for cross-device FL, the client number is normally much larger than the worker number (the processes that execute the local SGD), so as the original FedAvg paper suggested, we need to do a client sampling to tackle the scalability issue. So here we set the client number equal to the number in the dataset and then do client sampling:

FedML/fedml_api/distributed/fedavg/FedAvgServerManager.py

Line 66 in 79825b4

client_indexes = self.preprocessed_client_lists[self.round_idx]

Let's think about a concrete example in practice. We may have 1 million users in our cloud service, but you cannot ask all of them to join a round of federated training because of straggler, device failure, time zone gap, out of battery, user rejection, etc. So we sample those users who have a reliable connection and high intention to help the training. As for research, we can use random sampling to mimic this practice.

[hiyuchang]

Thank you for your clear explanation.