[RFC][python] deprecate advanced args of train() and cv() functions and sklearn wrapper

[StrikerRUS]

Proposing this PR to include in #4310

I also think this release could be a good opportunity for maintainers to think carefully about what breaking changes might be made (in addition to #3234) in a 4.0.0 release, and to add deprecation warnings for them in the 3.3.0 release.

Encourage users to use callbacks. With removed in 4.0.0 release advanced arguments codebase will become a little bit clearer, confusing cases when both callback and corresponding advanced argument are used will be eliminated, there will be no need to expose new callbacks or new arguments of existing callbacks (#2526) via new arguments of high-level functions/classes making signature of functions incredibly difficult to understand especially for newcomers.

[jameslamb]

I'm supportive of this idea, but could we preserve the current behavior where .best_iteration_ and .best_score_ are None if early stopping is not used?

Having those now be 0 is a breaking change and I think we should only make that breaking change in 4.0.0. There could be user code out there that is using _best_iteration_ is None as a way to detect whether not early stopping was used.

[jmoralez]

maybe we could rename this to log_evaluation as well? Seems like it is only used in engine right now so it could be a good time before it is used explicitly as a callback.

[StrikerRUS]

@jmoralez

Good idea, I like it! Unfortunately, print_evaluation() callback is already in public API, so we cannot simply rename it.
https://lightgbm.readthedocs.io/en/latest/Python-API.html#callbacks

I'll prepare a separate PR for that if you don't mind.

[StrikerRUS]

@jameslamb

could we preserve the current behavior

Sure, will revert some lines of code.

[StrikerRUS]

@jameslamb

could we preserve the current behavior

Addressed in 07046ec.

or :obj:None

in best_score_'s docstring was just a mistake. It's always a dict, possibly an empty one.

Also, fixed a bug when best_iteration_ was not set if early stopping was specified via keyword arguments or a callback, which is now a recommended way of specifying early stop.
In addition, fixed a bug when best_iteration_ and evals_result_ were preserved (not updated) after a previous call to fit() method. This happens when the first call was with early_stopping_rounds=xx and the seconnd one without early_stopping_rounds=None.

I can extract bug fixes into separate PRs to make this one easier to review if needed.

[jameslamb]

I can extract bug fixes into separate PRs to make this one easier to review if needed.

I can see where the bug fixes are and understand them, think it's ok to leave them in this PR. Thanks very much for those!

This looks great to me, I have no other comments. Sorry it took a few days to come review this.

Really looking forward to the point in the future where we come back and remove all these deprecation warnings and backwards-compatibility things in the Python and R packages 😀

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.