reproducing your result

[NoaGarnett]

Thanks for the simple and elegant implementation!
I tried running your code as is, on Multi-MNIST data, and failed to reproduce results.
I ran main_multi_mnist.py without changing any hyper parameter (learning rate (0.0005) , batch size (256), number of epochs (100)). For comparison, I created a version with no pcgrad:
1. comment out line 57: ## optimizer = PCGrad(optimizer)
2. replace line 72: optimizer.pc_backward(losses) -> torch.sum(torch.stack(losses)).backward()

I run each version 7 times. my results (averaging left-digit and right-digit accuracy) are:

Without PCGrad: average accuracy 89.5%, max accuracy 89.9%, standard deviation 0.38
With PCGrad: average accuracy 89.5%, max accuracy 89.8%, standard deviation 0.20

Can you come up with an explanation?
Many thanks,
Noa Garnett

[Lizhaoqing123]

Thanks for the simple and elegant implementation! I tried running your code as is, on Multi-MNIST data, and failed to reproduce results. I ran main_multi_mnist.py without changing any hyper parameter (learning rate (0.0005) , batch size (256), number of epochs (100)). For comparison, I created a version with no pcgrad: 1. comment out line 57: ## optimizer = PCGrad(optimizer) 2. replace line 72: optimizer.pc_backward(losses) -> torch.sum(torch.stack(losses)).backward()

I run each version 7 times. my results (averaging left-digit and right-digit accuracy) are:

Without PCGrad: average accuracy 89.5%, max accuracy 89.9%, standard deviation 0.38 With PCGrad: average accuracy 89.5%, max accuracy 89.8%, standard deviation 0.20

Can you come up with an explanation? Many thanks, Noa Garnett

My result is same as yours, did you finally solve this problem?

[NoaGarnettDia]

No, never did. Just moved on...

[yangmin666]

Have you solved it? I have the same issue.

[NoaGarnett]

No, gave it up long ago.

…

On Wed, Aug 23, 2023, 06:15 yangmin666 ***@***.***> wrote: Have you solved it? I have the same issue. — Reply to this email directly, view it on GitHub <#13 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNDAFIIUG7UBFFQJPJBDOLXWVYV3ANCNFSM5LV6WUVQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>