In the Per-FedAvg experiments, there is always an unignorable gap between the accuracy of the actual experimental results and the expected accuracy provided under the same conditions.

[Drizzlingg]

For example:
\begin{table}[]
\begin{tabular}{ccc}
ACC(Per-FedAvg) & MNIST & Synthetic \
MLR & 92.96% & 81.04% \
DNN & 93.01% & 76.79%
\end{tabular}
\end{table}

Does this algorithm require special settings in actual experiments?
Finally, sincerely thank you for your work.

[CharlieDinh]

Hi Drizzling, there are no special settings for Per-FedAvg, maybe there is an update on implementation for per-fedavg and I haven't checked the running result. Please check this issue:
#8

[Drizzlingg]

Hi CharlieDinh, thank you for your prompt response. I have reviewed this issue #8 and verified that the previous experimental results are no longer correct after preliminary experiments. Here I would like to sincerely thank you @CharlieDinh, @chuanting, and @sshpark for your contributions.

[CharlieDinh]

HI @Drizzlingg,
I will try to test the new implementation of per-fedavg and update the results soon.

[mengcz13]

What about the train_one_step method of UserPerAvg? Seems it still uses the two-step-update mentioned in #8 .

pFedMe/FLAlgorithms/users/userperavg.py

Line 73 in caf24b1

def train_one_step(self):

In addition, why do we need 2 batches of test data in train_one_step, which is used for evaluating personalized models in per-FedAvg? The per-FedAvg paper mentions that

The second and third algorithms that we consider are two different efficient approximations of Per-FedAvg.
Similarly, we evaluate the performance of these methods for the case that one step of local stochastic
gradient descent is performed during test time. (Page 8, 2nd paragraph of Sec 5)

I think it should simply be one step of SGD with 1 batch of test data?

[CharlieDinh]

HI @mengcz13
It follows the way per-FedAvg trains the model but on the test data (perFedAvg requires to adapt one step on the test data before evaluate.). One step means one local epoch not one batch. If just uses one batch, it will not follow the training step.