Join GitHub today
customizable early stopping #3822
"Can I suggest that we make early stopping a policy-based API with some sort of callback? So instead of just a count and a direction, we supply an object which receives the metrics up to the current round, and whatever other information might be useful, and we have a default implementation that is just the normal early stopping after X rounds without increase/decrease. I ask because I suspect there might be circumstances where you care about more than just an absolute number of rounds, or you might for efficiency reasons want to stop training even while still decreasing, just because the returns appear to be diminishing. Externalising the decision and making it configurable would future-proof the API somewhat.
So, just something like IStopping with boolean shouldStopAfter(float metrics) in the simplest case, but arguments could be made for more context. Making these objects stateful across training runs also simplifies the logic in the early stopping case, as you can just maintain the min/max as you go, and a count of how long ago it was."