Replace scikit GradientBoostingClassifier with XGBClassifier

[tcfuji]

Also includes a test and dependency in setup.py.

[tcfuji]

@rhiever I'm not too familiar with travis. Should I just add pip install xgboost==$XGBOOST_VERSION to .travis_install?

[rhiever]

I believe you need to do the following. @rasbt can confirm.

• Add the required version of XGBoost to the lines around https://github.com/rhiever/tpot/blob/master/.travis.yml#L7
• pip install XGBoost similar to how DEAP is installed here: https://github.com/rhiever/tpot/blob/master/ci/.travis_install.sh#L46
• print the XGBoost version similar to how the others are printed here: https://github.com/rhiever/tpot/blob/master/ci/.travis_install.sh#L58
• Also print the XGBoost version similarly here: https://github.com/rhiever/tpot/blob/master/ci/.travis_test.sh#L17

[rasbt]

I am not using boosting and don't know if XGBoost has any other dependencies. However, if it can be installed via pip install xgboost then the steps Randy outlined above are the way to go. Maybe set up a fresh virtual environment with numpy, scipy, and scikit-learn installed and test if pip install xgboost is sufficient or if you need something like pip install xgboost -r requirements.txt

[rhiever]

Thanks @rasbt! You should give XGBoost a try. Even without OpenMP, it seems to be streets ahead of regular gradient boosting.

[rasbt]

You are welcome! I dunno, someday maybe, but I feel like XGBoost is more of a ML competition / Kaggle thing :P . Currently, I am more excited about toying around with TensorFlow :)

[rhiever]

To each their own. If you look at the examples above, XGBoost is very easy to use. Same interface as sklearn models. I hope the sklearn team will integrate XGBoost soon.

[rhiever]

@tcfuji: Will review this tomorrow. Looks good from a high-level glance.

[rhiever]

Looks good. Thanks again @tcfuji!

[rhiever]

@tcfuji, when you get a moment, can you please submit a separate PR to update the docs? Currently, they say we still use the sklearn GBC. We need to update that.

The docs are located here.

You can view the docs as you change them by running mkdocs serve in the docs directory. (This will require you to pip install mkdocs if you haven't already.)

[rhiever]

I think we should simply remove the sklearn GBC entry and make a new entry for XGBoost. You can likely use the sklearn GBC entry as a template so the changes will be minimal.