xgboost: eXtreme Gradient Boosting

An optimized general purpose gradient boosting (tree) library.

Contributors: https://github.com/tqchen/xgboost/graphs/contributors

Turorial and Documentation: https://github.com/tqchen/xgboost/wiki

Features

• Sparse feature format:
  • Sparse feature format allows easy handling of missing values, and improve computation efficiency.
• Push the limit on single machine:
  • Efficient implementation that optimizes memory and computation.
• Speed: XGBoost is very fast
  • IN demo/higgs/speedtest.py, kaggle higgs data it is faster(on our machine 20 times faster using 4 threads) than sklearn.ensemble.GradientBoostingClassifier
• Layout of gradient boosting algorithm to support user defined objective
• Python interface, works with numpy and scipy.sparse matrix

Supported key components

• Gradient boosting models:
  • regression tree (GBRT)
  • linear model/lasso
• Objectives to support tasks:
  • regression
  • classification
• OpenMP implementation

Planned components

• More objective to support tasks:
  • ranking
  • matrix factorization
  • structured prediction

Build

• Simply type make
• If your compiler does not come with OpenMP support, it will fire an warning telling you that the code will compile into single thread mode, and you will get single thread xgboost
  • You may get a error: -lgomp is not found, you can remove -fopenmp flag in Makefile to get single thread xgboost, or upgrade your compiler to compile multi-thread version

File extension convention

• .h are interface, utils and data structures, with detailed comment;
• .cpp are implementations that will be compiled, with less comment;
• .hpp are implementations that will be included by .cpp, with less comment