Simplify EMA to use Pytorch's update_parameters

[prabhat00155]

Pytorch's AveragedModel handles buffer properly after pytorch/pytorch#71763. This can be used now instead of having a custom update_parameters() here.

cc @datumbox

[vadimkantorov]

A related issue to propose a EMA optimizer / functional optimizer into core: pytorch/pytorch#71683

[xiaohu2015]

I'm interested in this work.

[datumbox]

@xiaohu2015 It's yours. :)

It should be really straight forward as we basically want to reuse what we upstream to Core. We will need to do a couple of dummy training runs with very little data to confirm that the EMA utility works as expected but nothing as large-scale as training models.

[prabhat00155]

Should we wait for Pytorch's release with the related changes?

[xiaohu2015]

Should we wait for Pytorch's release with the related changes?

Pytorch 1.11 version?

[datumbox]

We can use the nightly of PyTorch that contains the change. Our release branch for the upcoming release is already cut so your changes won't affect it.