early stopping criteria not working with eval_metric "aucpr"

[ankurkhurana03]

The metrics aucpr should increase for the convergence, whereas the metrics required for early stopping criteria should be minimized;

[hcho3]

Try setting maximize=True when calling xgboost.train().

[ankurkhurana03]

thanks, that worked!

[imranahmed96]

I'm actually not sure maximize works when using the Scikit-Learn API for XGBoost

If you set maximize = True in the XGBClassifuer, and then set eval_metric = 'aucpr' in the .fit, this will not work. However, it will work if you use auc instead of aucpr

[hcho3]

@imranahmed96 Indeed, it is not possible to set maximize parameter in scikit-learn interface; it will be silently ignored. I will submit a fix so that aucpr gets automatically maximized.

[imranahmed96]

@hcho3 Thank-you for the lightning-fast reply! Perfect, that sounds great! 😄

[batmaxx]

@hcho3 is the change already available?

[hcho3]

@batmax2015 The change is not part of the stable 0.81 release. It will be part of the upcoming 0.82 release. You should use the latest source in the mean time.

[johntiger1]

Still not working for me (for auc-pr)

As an aside, why will it work for regular auc? @imranahmed96

If you set maximize = True in the XGBClassifuer, and then set eval_metric = 'aucpr' in the .fit, this will not work. However, it will work if you use auc instead of aucpr