New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Early stopping is implemented incorrectly in XGBoost.java:203 (jvm-packages) #3140

Closed
yvirin-apixio opened this Issue Feb 28, 2018 · 3 comments

Comments

Projects
None yet
4 participants
@yvirin-apixio

yvirin-apixio commented Feb 28, 2018

Environment info

xgboost version used: 0.8

The commit hash (git rev-parse HEAD):
11bfa85

The description of the problem

In the file: https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j/src/main/java/ml/dmlc/xgboost4j/java/XGBoost.java line 203:

seems like the "&=" is not the correct operator to use, and instead it should be "|=" with decreasing initialized to false, in order to make sure consecutive "increases" in the loss function during evaluation rounds have been observed. With the current implementation the code stops after first increase, which is wrong. Probably it is a hard bug to catch in practice, I was just inspecting the implementation of early stopping and found this issue.
The intended usage of earlyStoppingRound parameter is described in line 121 (comment).

@superbobry

This comment has been minimized.

Contributor

superbobry commented Feb 28, 2018

Feel free to submit a PR fixing this. There is also #2982.

@yvirin-apixio

This comment has been minimized.

yvirin-apixio commented Feb 28, 2018

@superbobry Thanks for connecting to the other issue. Somewhat related question: currently there is no way to know which is the optimal treeLimit to use in predictions, correct? Presumably it would be the tree which performed the best on evaluation during training.

yanvirin pushed a commit to yanvirin/xgboost that referenced this issue Mar 1, 2018

@hcho3

This comment has been minimized.

Collaborator

hcho3 commented Jul 4, 2018

Closing this for now. Feel free to re-open for any updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment