Add persistent flag to Metric.add_state

[teddykoker]

What does this PR do?

Adds persistent flag to Metrics, to allow the user to disable state being
added to the state_dict.

[pep8speaks]

Hello @teddykoker! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-10-16 16:16:47 UTC

[SkafteNicki]

Can we add test for this? One metric class with and one without the persistent flag on their state, and then check their state dict.

[teddykoker]

Can we add test for this? One metric class with and one without the persistent flag on their state, and then check their state dict.

Done :)

[Borda]

lgtm

[codecov]

Codecov Report

Merging #4195 into master will increase coverage by 0%.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master   #4195   +/-   ##
======================================
  Coverage      93%     93%           
======================================
  Files         103     103           
  Lines        7844    7847    +3     
======================================
+ Hits         7277    7280    +3     
  Misses        567     567           

[Vichoko]

Why would someone want to save a metric to the state_dict?
When you add metrics to the state_dict by default, you cannot load checkpoints any longer after you make changes to any of the metrics.

I feel pretty natural to add or update metrics of an already trained module. For example, adding the new metrics from the ptl.metrics module in the test loop and running the trainer in test mode fails catastrophically. Just like in my issue #4666 .