Optimize AUC for binary classification

[pplonski]

Right now, all metrics as minimized. AUC should be maximized.

Please add a check in AutoML constructor for correct metric name.

[shahules786]

@pplonski Can you elaborate this a little more?

[pplonski]

We minimize logloss for binary classification. There is no option to optimize different metrics in AutoML. This issue is to add an option to optimize AUC as eval_metric. During the implementation of this issue, it will be worth to check what other metrics can be optimized for classification (F1, Acc, MCC) and for regression.

It is worth to check which metrics can be optimized directly by algorithms. I would avoid callbacks in algorithms because they slow down the training 20x (yes, 20x, even empty list of callbacks).

[shahules786]

Cool @pplonski , will look at it.