adds tracking and logging best metrics to wandb cb

[arampacha]

As per my understanding currently one can use only the last logged metric in W&B in some scenarios (e.g. comparing runs in sweep, reporting scores with barplot etc.) which might be inconvenient when best score is different from final score. This PR adds logging best_metrics dict, which contain the best observed scores during the run. best_metrics contains an entry for each logged metric named best_{metric_name}. The compare argument is added to __init__ to allow user to provide custom list of comparison functions as it's not trivial to automate determining appropriate functions for all possible scenarios, I aimed made it consistent with TrackerCallback in this regard.

cc @morganmcg1 @borisdayma

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[borisdayma]

Awesome!
I try to always reuse existing parameters from other callbacks. Don't we already have access to best metrics? Maybe here

[arampacha]

@borisdayma If I understand it correctly recorder.log contains the metrics corresponding to the most recent epoch not the best one. And there is separate TrackerCallback here which monitors best value for early stopping and saving model. But it only monitors particular metric selected for monitoring, while all metrics might be interesting for analysis using W&B.

[borisdayma]

Is there a way to add TrackerCallback and make it monitor every metric?
Just trying to find a way to make it easily maintainable.

[jph00]

@arampacha @borisdayma I'll leave this open until you two have resolved the above conversation