New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

customizable early stopping #3822

Open
CodingCat opened this Issue Oct 23, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@CodingCat
Member

CodingCat commented Oct 23, 2018

proposed by @deepxg in #3808 (comment)

"Can I suggest that we make early stopping a policy-based API with some sort of callback? So instead of just a count and a direction, we supply an object which receives the metrics up to the current round, and whatever other information might be useful, and we have a default implementation that is just the normal early stopping after X rounds without increase/decrease. I ask because I suspect there might be circumstances where you care about more than just an absolute number of rounds, or you might for efficiency reasons want to stop training even while still decreasing, just because the returns appear to be diminishing. Externalising the decision and making it configurable would future-proof the API somewhat.

So, just something like IStopping with boolean shouldStopAfter(float[][] metrics) in the simplest case, but arguments could be made for more context. Making these objects stateful across training runs also simplifies the logic in the early stopping case, as you can just maintain the min/max as you go, and a count of how long ago it was."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment