20180415_Logistic回归.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.分类与预测——Logistic回归分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "回归分析是通过建立模型来研究变量之间相互关系的密切程度、结构状态及进行模型预测的一种有效工具。\n",
    "\n",
    "对于二分类的Logistic回归，因变量y只有“是、否”两个取值，记为1和0。假设在自变量x1，x2，…，xp作用下，y取“是”的概率是p，则取“否”的概率是1-p，研究的是当y取“是”发生的概率p与自变量x1，x2，…，xp的关系。\n",
    "\n",
    "二分类Logistic回归模型的原理如下：  \n",
    "\n",
    "**1. Logistic函数  **  \n",
    "\n",
    "Logistic回归模型中的因变量的只有1-0（如是和否、发生和不发生）两种取值。假设在p个独立自变量x1，x2，…，xp作用下，记y取1的概率是p=P(y=1|X)，取0概率是1-p，取1和取0的概率之比为p/(1-p)，称为事件的优势比（odds）,对odds取自然对数即得Logistic变换Logit(p)=ln[p/(1-p)]。  \n",
    "\n",
    "令$$Logit(p)=ln[\\frac{p}{1-p}]=z$$则$$p=\\frac{1}{1+e^{-z}}$$即为Logistic函数.  \n",
    "\n",
    "当p在(0，1)之间变化时，odds的取值范围是(0，+∞)，则ln[p/(1-p)]的取值范围是(-∞，+∞)。  \n",
    "\n",
    "**2. Logistic回归模型  **  \n",
    "\n",
    "Logistic回归模型是建立ln[p/（1-p）]与自变量的线性回归模型。  \n",
    "\n",
    "Logistic回归模型为：\n",
    "$$ln\\frac{p}{1-p}=\\beta_0+{\\beta_1}x_1+...+{\\beta_p}x_p+\\epsilon$$  \n",
    "\n",
    "因为ln[p/（1-p）]的取值范围是（-∞，+∞），这样，自变量x1，x2，…，xp可在任意范围内取值。  \n",
    "\n",
    "记$$g(x)=\\beta_0+{\\beta_1}x_1+...+{\\beta_p}x_p+\\epsilon$$得到：$$p=P(y=1|X)=\\frac{1}{1+e^{-g(x)}}$$ \n",
    "\n",
    "$$1-p=P(y=0|X)=\\frac{1}{1+e^{g(x)}}$$\n",
    "\n",
    "**3. Logistic回归模型解释**  \n",
    "$$\\frac{p}{1-p}=e^{\\beta_0+{\\beta_1}x_1+...+{\\beta_p}x_p+\\epsilon}$$\n",
    "\n",
    "βi:某自变量Xi变化一单位时，y＝1与y=0发生概率之比的对数的变化值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以以下某银行贷款拖欠率数据为例，我们想要知道贷款拖欠率与哪些因素有关"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>年龄</th>\n",
       "      <th>教育</th>\n",
       "      <th>工龄</th>\n",
       "      <th>地址</th>\n",
       "      <th>收入</th>\n",
       "      <th>负债率</th>\n",
       "      <th>信用卡负债</th>\n",
       "      <th>其他负债</th>\n",
       "      <th>违约</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>41</td>\n",
       "      <td>3</td>\n",
       "      <td>17</td>\n",
       "      <td>12</td>\n",
       "      <td>176</td>\n",
       "      <td>9.3</td>\n",
       "      <td>11.359392</td>\n",
       "      <td>5.008608</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>6</td>\n",
       "      <td>31</td>\n",
       "      <td>17.3</td>\n",
       "      <td>1.362202</td>\n",
       "      <td>4.000798</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>55</td>\n",
       "      <td>5.5</td>\n",
       "      <td>0.856075</td>\n",
       "      <td>2.168925</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>41</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>120</td>\n",
       "      <td>2.9</td>\n",
       "      <td>2.658720</td>\n",
       "      <td>0.821280</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>17.3</td>\n",
       "      <td>1.787436</td>\n",
       "      <td>3.056564</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   年龄  教育  工龄  地址   收入   负债率      信用卡负债      其他负债  违约\n",
       "0  41   3  17  12  176   9.3  11.359392  5.008608   1\n",
       "1  27   1  10   6   31  17.3   1.362202  4.000798   0\n",
       "2  40   1  15  14   55   5.5   0.856075  2.168925   0\n",
       "3  41   1  15  14  120   2.9   2.658720  0.821280   0\n",
       "4  24   2   2   0   28  17.3   1.787436  3.056564   1"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "#参数初始化\n",
    "filename = 'data/bankloan.xls'\n",
    "data = pd.read_excel(filename)\n",
    "x = data.iloc[:,:8].as_matrix()\n",
    "y = data.iloc[:,8].as_matrix()\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用Scikit-Learn对这个数据进行逻辑回归分析。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "通过随机逻辑回归模型筛选特征结束。\n"
     ]
    }
   ],
   "source": [
    "# 导入包\n",
    "from sklearn.linear_model import LogisticRegression as LR\n",
    "from sklearn.linear_model import RandomizedLogisticRegression as RLR\n",
    "rlr = RLR() #建立随机逻辑回归模型，筛选变量\n",
    "rlr.fit(x, y) #训练模型\n",
    "rlr.get_support() #获取特征筛选结果，也可以通过.scores_方法获取各个特征的分数\n",
    "print('通过随机逻辑回归模型筛选特征结束。')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "有效特征为：工龄,地址,负债率,信用卡负债\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\pandas\\indexes\\base.py:1275: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 9 but corresponding boolean dimension is 8\n",
      "  result = getitem(key)\n"
     ]
    }
   ],
   "source": [
    "print(u'有效特征为：%s' % ','.join(data.columns[rlr.get_support()]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这样，我们就得到了对违约有预测作用的变量。接下来，我们利用这些变量建模，并计算给出模型的平均正确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\pandas\\indexes\\base.py:1275: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 9 but corresponding boolean dimension is 8\n",
      "  result = getitem(key)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "逻辑回归模型训练结束。\n",
      "模型的平均正确率为：0.814285714286\n"
     ]
    }
   ],
   "source": [
    "x = data[data.columns[rlr.get_support()]].as_matrix() #筛选好特征\n",
    "lr = LR() #建立logistic模型\n",
    "lr.fit(x, y) #用筛选后的特征数据来训练模型\n",
    "print(u'逻辑回归模型训练结束。')\n",
    "print(u'模型的平均正确率为：%s' % lr.score(x, y)) #给出模型的平均正确率，"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}