Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Use first_metric_only flag for early_stopping function. #2049

Merged

Conversation

matsuken92
Copy link
Contributor

In order to apply early stopping with only first metric, applying first_metric_only flag for early_stopping function.

In order to apply early stopping with only first metric, applying first_metric_only flag for early_stopping function.
@matsuken92
Copy link
Contributor Author

The following error may be CI environment problem on appveyor. TravisCI and Azure DevOps have same issue. How to deal with it to solve...?

https://ci.appveyor.com/project/guolinke/lightgbm/builds/23063424/job/7unpekbvoh1gwt3r

jupyter nbconvert --ExecutePreprocessor.timeout=180 --to notebook --execute --inplace *.ipynb
Traceback (most recent call last):
  File "C:\Miniconda36-x64\envs\test-env\Scripts\jupyter-nbconvert-script.py", line 6, in <module>
    from nbconvert.nbconvertapp import main
  File "C:\Miniconda36-x64\envs\test-env\lib\site-packages\nbconvert\__init__.py", line 7, in <module>
    from . import postprocessors
  File "C:\Miniconda36-x64\envs\test-env\lib\site-packages\nbconvert\postprocessors\__init__.py", line 5, in <module>
    from .serve import ServePostProcessor
  File "C:\Miniconda36-x64\envs\test-env\lib\site-packages\nbconvert\postprocessors\serve.py", line 19, in <module>
    class ProxyHandler(web.RequestHandler):
  File "C:\Miniconda36-x64\envs\test-env\lib\site-packages\nbconvert\postprocessors\serve.py", line 21, in ProxyHandler
    @web.asynchronous
AttributeError: module 'tornado.web' has no attribute 'asynchronous'
Command exited with code 1

@guolinke
Copy link
Collaborator

@StrikerRUS can you help to review this? Thank you!

@StrikerRUS
Copy link
Collaborator

@matsuken92 Thanks for your contribution! Can you please write a test for the enhancement you are proposing? I think it'll be rather straightforward with self-defined dummy metrics.

@msftclas
Copy link

msftclas commented Mar 18, 2019

CLA assistant check
All CLA requirements met.

@StrikerRUS StrikerRUS force-pushed the feature/only_first_metrics_for_early_stop branch from 1c804f7 to 6ba245e Compare March 18, 2019 22:12
@StrikerRUS
Copy link
Collaborator

I went ahead and added simple test and updated docs with info about the new feature.

@StrikerRUS StrikerRUS changed the title Use first_metric_only flag for early_stopping function. [python] Use first_metric_only flag for early_stopping function. Mar 18, 2019
@guolinke guolinke merged commit 011cc90 into microsoft:master Mar 25, 2019
@guolinke
Copy link
Collaborator

one question:
when there are multi validation sets, the training will stop when any of them met early stopping? or just the first validation set?

@StrikerRUS
Copy link
Collaborator

I think that the behavior with multiple valid sets should remain unchanged:

Early stopping requires at least one set in valid_sets. If there is more than one, it will use all of them except the training data:

@guolinke
Copy link
Collaborator

@StrikerRUS @matsuken92 I think this implementation is only for the first metric of first validation set.

@StrikerRUS
Copy link
Collaborator

Then it should be addressed in #2127 to preserve documented behavior, I think.

@lock lock bot locked as resolved and limited conversation to collaborators Mar 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants