Python58-XGBoost学习与实战之初识XGBoost（二）sklearn 相结合.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost快速入门——与scikit-learn一起使用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 运行xgboost安装包中的实例程序\n",
    "from xgboost import XGBClassifier\n",
    "\n",
    "# 加载LibSVM格式数据模块\n",
    "from sklearn.datasets import load_svmlight_file\n",
    "from sklearn.metrics import accuracy_score\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据读取\n",
    "scikit-learn支持多种格式的数据，包括LibSVM格式数据\n",
    "XGBoost可以加载libsvm格式的文本数据，libsvm的文件格式（稀疏特征）如下：\n",
    "\n",
    "1 101:1.2 102:0.03\n",
    "\n",
    "0 1:2.1 10001:300 10002:400\n",
    "\n",
    "...\n",
    "\n",
    "每一行表示一个样本，第一行的开头的“1”是样本的标签。“101”和“102”为特征索引，'1.2'和'0.03' 为特征的值。\n",
    "在两类分类中，用“1”表示正样本，用“0” 表示负样本。也支持[0,1]表示概率用来做标签，表示为正样本的概率。\n",
    "\n",
    "下面的示例数据需要我们通过一些蘑菇的若干属性判断这个品种是否有毒。\n",
    "\n",
    "UCI数据描述：http://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/ ，\n",
    "\n",
    "每个样本描述了蘑菇的22个属性，比如形状、气味等等(加工成libsvm格式后变成了126维特征)，\n",
    "然后给出了这个蘑菇是否可食用。其中6513个样本做训练，1611个样本做测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(6513, 126)\n",
      "(1611, 126)\n"
     ]
    }
   ],
   "source": [
    "# read in data，数据在xgboost安装的路径下的demo目录,\n",
    "\n",
    "my_workpath = 'files/data/python58-data/'\n",
    "X_train,y_train = load_svmlight_file(my_workpath + 'agaricus.txt.train')\n",
    "X_test,y_test = load_svmlight_file(my_workpath + 'agaricus.txt.test')\n",
    "\n",
    "print(X_train.shape)\n",
    "print (X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练参数设置\n",
    "- max_depth： 树的最大深度。缺省值为6，取值范围为：[1,∞]\n",
    "- eta：为了防止过拟合，更新过程中用到的收缩步长。在每次提升计算之后，算法会直接获得新特征的权重。 \n",
    "- eta通过缩减特征的权重使提升计算过程更加保守。缺省值为0.3，取值范围为：[0,1]\n",
    "- silent：取0时表示打印出运行时信息，取1时表示以缄默方式运行，不打印运行时信息。缺省值为0\n",
    "- objective： 定义学习任务及相应的学习目标，“binary:logistic” 表示二分类的逻辑回归问题，输出为概率。\n",
    "\n",
    "- 其他参数取默认值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练模型\n",
    "有了参数列表和数据就可以训练模型了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "       colsample_bytree=1, gamma=0, learning_rate=1, max_delta_step=0,\n",
       "       max_depth=2, min_child_weight=1, missing=None, n_estimators=2,\n",
       "       n_jobs=1, nthread=None, object='binary.logistic',\n",
       "       objective='binary:logistic', random_state=0, reg_alpha=0,\n",
       "       reg_lambda=1, scale_pos_weight=1, seed=None, silent=True,\n",
       "       subsample=1)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置boosting迭代计算次数\n",
    "num_round=2\n",
    "\n",
    "# bst=XGBClassifier(**params)\n",
    "# bst=XGBClassifier()\n",
    "bst=XGBClassifier(max_depth=2,learning_rate=1,n_estimators=num_round,\n",
    "                  silent=True,object='binary.logistic')\n",
    "bst.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看模型在训练集上的分类性能\n",
    "\n",
    "XGBoost预测的输出是概率。这里蘑菇分类是一个二类分类问题，输出值是样本为第一类的概率。 我们需要将概率值转换为0或1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Accuary: 97.77%\n"
     ]
    }
   ],
   "source": [
    "train_preds = bst.predict(X_train)\n",
    "train_predictions = [round(value) for value in train_preds]\n",
    "\n",
    "train_accuracy = accuracy_score(y_train, train_predictions)\n",
    "print (\"Train Accuary: %.2f%%\" % (train_accuracy * 100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试\n",
    "模型训练好后，可以用训练好的模型对测试数据进行预测\n",
    "\n",
    "XGBoost预测的输出是概率，输出值是样本为第一类的概率。我们需要将概率值转换为0或1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy: 97.83%\n"
     ]
    }
   ],
   "source": [
    "# make prediction\n",
    "preds = bst.predict(X_test)\n",
    "predictions = [round(value) for value in preds]\n",
    "\n",
    "test_accuracy = accuracy_score(y_test, predictions)\n",
    "print(\"Test Accuracy: %.2f%%\" % (test_accuracy * 100.0))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}