New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XGBoost 0.6a2 does not work inside the pynisher ... #271

Closed
mfeurer opened this Issue May 9, 2017 · 9 comments

Comments

Projects
None yet
5 participants
@mfeurer
Contributor

mfeurer commented May 9, 2017

... if the pynisher has to stop the execution of the target algorithm due to a timeout. I suspect that this is an issue of OpenMP together with multiprocessing.Process. The following code makes Auto-sklearn hang:

import multiprocessing
multiprocessing.set_start_method('fork')

import logging
import ConfigSpace
from autosklearn.pipeline.classification import SimpleClassificationPipeline
from test.test_evaluation.evaluation_util import get_abalone_datamanager
from autosklearn.metrics import accuracy
from autosklearn.evaluation import ExecuteTaFuncWithQueue
import unittest.mock
from smac.stats.stats import Stats


cs = SimpleClassificationPipeline().get_hyperparameter_search_space()


configuration = {"balancing:strategy": 'none',
                 "classifier:__choice__": 'xgradient_boosting',
                 "classifier:xgradient_boosting:base_score": 0.5,
                 "classifier:xgradient_boosting:colsample_bylevel": 1,
                 "classifier:xgradient_boosting:colsample_bytree": 1,
                 "classifier:xgradient_boosting:gamma": 0,
                 "classifier:xgradient_boosting:learning_rate": 0.07932363043971882,
                 "classifier:xgradient_boosting:max_delta_step": 0,
                 "classifier:xgradient_boosting:max_depth": 5,
                 "classifier:xgradient_boosting:min_child_weight": 3,
                 "classifier:xgradient_boosting:n_estimators": 323,
                 "classifier:xgradient_boosting:reg_alpha": 0,
                 "classifier:xgradient_boosting:reg_lambda": 1,
                 "classifier:xgradient_boosting:scale_pos_weight": 1,
                 "classifier:xgradient_boosting:subsample": 0.5431305206882203,
                 "imputation:strategy": 'mean',
                 "one_hot_encoding:use_minimum_fraction": 'False',
                 "preprocessor:__choice__": 'random_trees_embedding',
                 "preprocessor:random_trees_embedding:max_depth": 9,
                 "preprocessor:random_trees_embedding:max_leaf_nodes": 'None',
                 "preprocessor:random_trees_embedding:min_samples_leaf": 4,
                 "preprocessor:random_trees_embedding:min_samples_split": 19,
                 "preprocessor:random_trees_embedding:min_weight_fraction_leaf": 1.0,
                 "preprocessor:random_trees_embedding:n_estimators": 97,
                 "rescaling:__choice__": 'none'}
configuration = ConfigSpace.Configuration(configuration_space=cs,
                                          values=configuration)

cutoff = 60

scenario_mock = unittest.mock.Mock()
scenario_mock.wallclock_limit = cutoff
scenario_mock.algo_runs_timelimit = 1000
scenario_mock.ta_run_limit = 100
stats = Stats(scenario_mock)
stats.start_timing()

logging.basicConfig(format='%(asctime)s [%(levelname)s] %(message)s')
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

class BackendMock(object):
    def load_datamanager(self):
        return get_abalone_datamanager()

ta = ExecuteTaFuncWithQueue(backend=BackendMock(), autosklearn_seed=1,
                            resampling_strategy='holdout',
                            logger=logging.getLogger(),
                            stats=stats,
                            memory_limit=3072,
                            metric=accuracy)
print(ta.start(config=configuration, cutoff=cutoff, instance=None))

TPOT has observed similar issues: dmlc/xgboost#2163

XGBoost will be disabled until there is a simple solution for this.

@mfeurer mfeurer added the bug label May 9, 2017

mfeurer added a commit that referenced this issue May 9, 2017

mfeurer added a commit that referenced this issue May 9, 2017

mfeurer added a commit that referenced this issue May 9, 2017

mfeurer added a commit that referenced this issue May 10, 2017

@DCBoland

This comment has been minimized.

Show comment
Hide comment
@DCBoland

DCBoland Jun 15, 2017

Tried with latest xgboost (0db37c0) and this issue seems to be resolved. It's not clear how to get this latest version as a requirement though, without first updating PyPI.

DCBoland commented Jun 15, 2017

Tried with latest xgboost (0db37c0) and this issue seems to be resolved. It's not clear how to get this latest version as a requirement though, without first updating PyPI.

@mfeurer

This comment has been minimized.

Show comment
Hide comment
@mfeurer

mfeurer Jun 15, 2017

Contributor

Thanks a lot for checking this. It would be possible to require a special version of XGBoost from github at installation time, but this cannot be checked at runtime. To not introduce weird errors and keep the build simple I would like to wait for a new release of XGBoost before adding it back.

Contributor

mfeurer commented Jun 15, 2017

Thanks a lot for checking this. It would be possible to require a special version of XGBoost from github at installation time, but this cannot be checked at runtime. To not introduce weird errors and keep the build simple I would like to wait for a new release of XGBoost before adding it back.

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Jun 16, 2017

I think this might fix this problem: https://github.com/tomMoral/loky

amueller commented Jun 16, 2017

I think this might fix this problem: https://github.com/tomMoral/loky

@mfeurer

This comment has been minimized.

Show comment
Hide comment
@mfeurer

mfeurer Jun 19, 2017

Contributor

Thanks @amueller. Do you know if anyone tried to use XGBoost inside loky?

Contributor

mfeurer commented Jun 19, 2017

Thanks @amueller. Do you know if anyone tried to use XGBoost inside loky?

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Jun 19, 2017

@mfeurer I do not. Maybe @ogrisel knows. But I'm pretty sure it should work or you should let @ogrisel know that there's a bug ;)

amueller commented Jun 19, 2017

@mfeurer I do not. Maybe @ogrisel knows. But I'm pretty sure it should work or you should let @ogrisel know that there's a bug ;)

@ogrisel

This comment has been minimized.

Show comment
Hide comment
@ogrisel

ogrisel Jun 19, 2017

I have not tried with xgboost but loky was designed to avoid this family of problems.

ogrisel commented Jun 19, 2017

I have not tried with xgboost but loky was designed to avoid this family of problems.

@manuel-delverme

This comment has been minimized.

Show comment
Hide comment
@manuel-delverme

manuel-delverme Oct 25, 2017

Will xgboost ever be re-enabled?
if not it should be noted somewhere since "For a full list please have a look at the source code (in autosklearn/pipeline/components/)" is deceitful

manuel-delverme commented Oct 25, 2017

Will xgboost ever be re-enabled?
if not it should be noted somewhere since "For a full list please have a look at the source code (in autosklearn/pipeline/components/)" is deceitful

@mfeurer

This comment has been minimized.

Show comment
Hide comment
@mfeurer

mfeurer May 2, 2018

Contributor

XGBoost will be available again in the next release.

Contributor

mfeurer commented May 2, 2018

XGBoost will be available again in the next release.

@mfeurer mfeurer closed this May 2, 2018

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller commented May 7, 2018

sweet!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment