digits experiment (precision)

[wnma3mz]

Hi, I came across your work. It was an interesting idea, but it ran into some problems during the experiment.

When I clone your code, run the command below. The accuracy in the paper cannot be obtained.

# benchmark experiment
python fed_digits.py --mode fedbn --iters 300

This is my best experiment.

============ Train epoch 208 ============
 MNIST      | Train Loss: 0.0003 | Train Acc: 1.0000
 SVHN       | Train Loss: 0.0010 | Train Acc: 1.0000
 USPS       | Train Loss: 0.0004 | Train Acc: 1.0000
 SynthDigits| Train Loss: 0.0007 | Train Acc: 1.0000
 MNIST-M    | Train Loss: 0.0007 | Train Acc: 1.0000
 MNIST      | Test  Loss: 0.0985 | Test  Acc: 0.9689
 SVHN       | Test  Loss: 0.9124 | Test  Acc: 0.7232
 USPS       | Test  Loss: 0.1033 | Test  Acc: 0.9683
 SynthDigits| Test  Loss: 0.5342 | Test  Acc: 0.8359
 MNIST-M    | Test  Loss: 0.6474 | Test  Acc: 0.7904

Did I miss any details?

My platform

Ubuntu 20.04
GeForce RTX 2080 Ti
Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz

Thanks for taking the time to help!

[MeiruiJiang]

Hi, we have updated our published code and the absolute values for fedbn have some changes, please refer to the latest version. https://arxiv.org/pdf/2102.07623.pdf

[wnma3mz]

Hi, we have updated our published code and the absolute values for fedbn have some changes, please refer to the latest version. https://arxiv.org/pdf/2102.07623.pdf

Thank you very much for your prompt reply!

That helps me a lot!