ml xgboost

XGBoost(Extreme Gradient Boosting)

XGBoost is an ensemble method, meaning that it is composed of different machine learning models that combine to work together. The individual models that make up the ensemble in XGBoost are called base learners.

Decision trees, the most commonly used XGBoost base learners, are unique in the machine learning landscape.

Decision trees are prone to overfitting the data. In other words, decision trees can map too closely to the training data, a problem explored later in this chapter in terms of variance and bias. Hyperparameter fine-tuning is one solution to prevent overfitting. Another solution is to aggregate the predictions of many trees, a strategy that Random Forests and XGBoost employ.

Like XGBoost, random forests are ensembles of decision trees. The difference is that random forests combine trees via bagging, while XGBoost combines trees via boosting. Random forests are a viable alternative to XGBoost

from xgboost import XGBClassifier
cross_val(XGBClassifier(n_estimators=5)