Make MlflowMetricsDataSet capable of saving a Dict[str, float] object, as mlflow.log_metrics() does

[yury-fedotov]

Description

At the moment MlflowMetricsDataSet can only save an object which is a nested Dict. For example:

{
    "metric1": {"value": 1.1, "step": 1},
    "metric2": [{"value": 1.1, "step": 1}, {"value": 1.2, "step": 2}],
}

However, it cannot save an object like this:

{
    "metric1": 1.1,
    "metric2": 1.2,
}

Which is frequently used and e.g. mlflow's native log_metrics() API deals fine with it.

Context

Having this support would be great to:

• Make MlflowMetricsDataSet more compatible with mlflow.log_metrics()
• Allow users to save simple non-nested Dict, without the need to convert it to nested format.

Possible Implementation

Either add a new Dataset or handle this within existing MlflowMetricsDataSet.

[yury-fedotov]

P.S. I read about flatten and recursive settings in mlflow.yml, but at least to my understanding those serve other purposes than the one I described above.

[Galileo-Galilei]

Hi, this is indeed really something needed and it's been on my to do liste for a long time. As for MlflowMetricDataSet we should likely modify MlflowMetricsDataSet to have suggested behaviour and move the existing behaviour to a new MlflowMetricsHistoryDataSet.

The reason why it is done like this currently is to maintain consistency between save and load method but I acknowledge that the user experience is a pain.

Feel free to open a PR to help this getting real ;)