DataScience/回帰分析の検定.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "78127756-ed03-482c-b2c4-d353e50d66f9",
   "metadata": {
    "tags": []
   },
   "source": [
    "# **回帰分析の検定**\n",
    "重回帰分析や判別分析を実行する際に変数選択手法の特徴を理解し、適用できる"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15345b81-ab5a-43cc-b84f-3daa7f3b43cd",
   "metadata": {
    "tags": []
   },
   "source": [
    "### **単回帰分析の偏回帰係数の検定**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f4b56cf-295c-4aca-98fe-7e81ebe5dd6c",
   "metadata": {},
   "source": [
    "標本回帰方程式 $ Y = \\hat\\beta_1 + \\hat\\beta_2 X $ において $ X $ が $ Y $ をどのように説明しているかはその傾き $ \\hat\\beta_2 $ で表される。この傾きの有意性について検定をしよう。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ac14a34-ffcb-4490-a603-83683439f594",
   "metadata": {},
   "source": [
    "帰無仮説と対立仮設を以下のように置く。\n",
    "- 帰無仮説 $ H_0 : \\beta_2 = a $ ($ a $ は指定された定数)\n",
    "- 対立仮設 $ H_1 : \\beta_2 ≠ a $ (両側検定) または $ H_1 : \\beta_2 > a $ (片側検定)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05b42f14-b2ae-4b6b-9c6d-0482b1d98c0d",
   "metadata": {},
   "source": [
    "まず、偏回帰係数 $ \\beta_2 $ の $t$値を以下の式で求める。\n",
    "\n",
    "$ t_2 = \\frac{\\hat\\beta_2 - a} {s.e.(\\hat\\beta_2)} $ ($ s.e. $ は標準誤差)\n",
    "\n",
    "この $t$値を求め、$t$分布表からから自由度 $ n - 2 $ のパーセント点を求めて検定を行うことで偏回帰係数の検定を行うことができる。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2ffaac3-8413-44a7-a754-26cb3a0218d0",
   "metadata": {},
   "source": [
    "$ X $ で $ Y $ を説明できるかどうかが重要であるから、以下の帰無仮説が特に重要である。\n",
    "\n",
    "$ H_0 : \\beta_2 = 0 $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6342a0cf-f6ca-42a4-bbfc-6c12113234aa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# モジュールとデータの読み込み\n",
    "import numpy as np\n",
    "import statsmodels.api as sm\n",
    "import scipy\n",
    "\n",
    "spector_data = sm.datasets.spector.load()\n",
    "\n",
    "# 変数 GPA のみを選択する\n",
    "spector_data.exog = spector_data.exog[\"GPA\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "c4a63e22-009e-4397-ad9f-f3b4a3029bca",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                 OLS Regression Results                                \n",
      "=======================================================================================\n",
      "Dep. Variable:                  GRADE   R-squared (uncentered):                   0.408\n",
      "Model:                            OLS   Adj. R-squared (uncentered):              0.389\n",
      "Method:                 Least Squares   F-statistic:                              21.37\n",
      "Date:                Mon, 14 Aug 2023   Prob (F-statistic):                    6.32e-05\n",
      "Time:                        09:51:21   Log-Likelihood:                         -19.932\n",
      "No. Observations:                  32   AIC:                                      41.86\n",
      "Df Residuals:                      31   BIC:                                      43.33\n",
      "Df Model:                           1                                                  \n",
      "Covariance Type:            nonrobust                                                  \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "GPA            0.1189      0.026      4.622      0.000       0.066       0.171\n",
      "==============================================================================\n",
      "Omnibus:                       22.056   Durbin-Watson:                   2.001\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):                5.376\n",
      "Skew:                           0.658   Prob(JB):                       0.0680\n",
      "Kurtosis:                       1.484   Cond. No.                         1.00\n",
      "==============================================================================\n",
      "\n",
      "Notes:\n",
      "[1] R² is computed without centering (uncentered) since the model does not contain a constant.\n",
      "[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
     ]
    }
   ],
   "source": [
    "# OLS モデルの当てはめとサマリ\n",
    "mod = sm.OLS(spector_data.endog, spector_data.exog)\n",
    "res = mod.fit()\n",
    "print(res.summary())\n",
    "\n",
    "# t がその偏回帰係数の t値、P>|t| が有意水準が p値 である。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91b4814a-b083-4174-8fa7-e129cb0de2a6",
   "metadata": {},
   "source": [
    "### **重回帰分析の偏回帰係数の検定**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4df899c-e6e2-46f6-91a1-15ed6534e26f",
   "metadata": {
    "tags": []
   },
   "source": [
    "重回帰分析においても、一つの偏回帰係数についての $ H_0 : \\beta_t = a $ の検定は、単回帰分析と同様に行うことができる。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2b521fd0-0ed9-49e5-87ed-bfb0b921ff65",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# データの読み込み\n",
    "spector_data = sm.datasets.spector.load()\n",
    "\n",
    "# 切片項を加えるとともに、変数 GPA のみを選択する\n",
    "spector_data.exog = sm.add_constant(spector_data.exog, prepend=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a1651daa-37a2-41bb-8bd0-6408da7558b3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                  GRADE   R-squared:                       0.416\n",
      "Model:                            OLS   Adj. R-squared:                  0.353\n",
      "Method:                 Least Squares   F-statistic:                     6.646\n",
      "Date:                Mon, 14 Aug 2023   Prob (F-statistic):            0.00157\n",
      "Time:                        05:49:00   Log-Likelihood:                -12.978\n",
      "No. Observations:                  32   AIC:                             33.96\n",
      "Df Residuals:                      28   BIC:                             39.82\n",
      "Df Model:                           3                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "GPA            0.4639      0.162      2.864      0.008       0.132       0.796\n",
      "TUCE           0.0105      0.019      0.539      0.594      -0.029       0.050\n",
      "PSI            0.3786      0.139      2.720      0.011       0.093       0.664\n",
      "const         -1.4980      0.524     -2.859      0.008      -2.571      -0.425\n",
      "==============================================================================\n",
      "Omnibus:                        0.176   Durbin-Watson:                   2.346\n",
      "Prob(Omnibus):                  0.916   Jarque-Bera (JB):                0.167\n",
      "Skew:                           0.141   Prob(JB):                        0.920\n",
      "Kurtosis:                       2.786   Cond. No.                         176.\n",
      "==============================================================================\n",
      "\n",
      "Notes:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
     ]
    }
   ],
   "source": [
    "# OLS モデルの当てはめとサマリ\n",
    "mod = sm.OLS(spector_data.endog, spector_data.exog)\n",
    "res = mod.fit()\n",
    "print(res.summary())\n",
    "\n",
    "# t がその偏回帰係数の t値、P>|t| が有意水準が p値 である。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad19e0e7-9f56-4cc4-8e7d-d086959b99e2",
   "metadata": {
    "tags": []
   },
   "source": [
    "一方、重回帰分析では複数の説明変数がある。いくつかの回帰係数についての仮説を同時に検定したい場合、回帰係数ごとの$t$検定では不十分であり、次に述べる$F$検定を用いる。\n",
    "\n",
    "たとえば、上記サンプルデータにおいて、帰無仮説と対立仮説を以下のように定義する。\n",
    "\n",
    "- 帰無仮説：$ H_0 : \\beta_{GPA} = 0 $ かつ $ \\beta_{TUCE} = 0 $ かつ $ \\beta_{PSI} = 0 $\n",
    "- 対立仮説：$ H_1 : \\beta_{GPA} ≠ 0 $ または $ \\beta_{TUCE} ≠ 0 $ または $ \\beta_{PSI} ≠ 0 $\n",
    "\n",
    "1. 説明変数を含まない重回帰方程式を推定し、回帰残差の平方和 $ \\sum \\hat{e}_i^2 $ を $ S_0 $ とする。\n",
    "2. すべての説明変数を含む重回帰方程式を推定し、その回帰残差の平方和 $ \\sum \\hat{e}_i^2 $ を $ S_1 $ とする。\n",
    "3. 帰無仮説に含まれる制約式の数を $ p $ とすると、統計量 $ F = \\frac{(S_0 - S_1) / p} {S_1 / (n-k)} $ は、帰無仮説が正しい場合に自由度 $ (p, n-k) $ の $F$分布 $F(p, n-k)$に従うことが知られている。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e685a34e-a17b-4026-8f8c-d282c7269863",
   "metadata": {
    "tags": []
   },
   "source": [
    "$ F ≧ F_a(p, n-k) $ の時に帰無仮説を棄却し、それ以外は棄却しない。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "f7e461ef-7f6f-47c7-955f-c3b43c080a4d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                  GRADE   R-squared:                       0.416\n",
      "Model:                            OLS   Adj. R-squared:                  0.353\n",
      "Method:                 Least Squares   F-statistic:                     6.646\n",
      "Date:                Mon, 14 Aug 2023   Prob (F-statistic):            0.00157\n",
      "Time:                        09:50:13   Log-Likelihood:                -12.978\n",
      "No. Observations:                  32   AIC:                             33.96\n",
      "Df Residuals:                      28   BIC:                             39.82\n",
      "Df Model:                           3                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "GPA            0.4639      0.162      2.864      0.008       0.132       0.796\n",
      "TUCE           0.0105      0.019      0.539      0.594      -0.029       0.050\n",
      "PSI            0.3786      0.139      2.720      0.011       0.093       0.664\n",
      "const         -1.4980      0.524     -2.859      0.008      -2.571      -0.425\n",
      "==============================================================================\n",
      "Omnibus:                        0.176   Durbin-Watson:                   2.346\n",
      "Prob(Omnibus):                  0.916   Jarque-Bera (JB):                0.167\n",
      "Skew:                           0.141   Prob(JB):                        0.920\n",
      "Kurtosis:                       2.786   Cond. No.                         176.\n",
      "==============================================================================\n",
      "\n",
      "Notes:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
     ]
    }
   ],
   "source": [
    "# 統計量Fはサマリの F-stasistic の項に値が表示される\n",
    "# 統計量Fにもとづく p値は Prob (F-statistic) の項に値が表示される\n",
    "print(res.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9174f1aa-cc4a-4d54-8d29-78866e5ac39b",
   "metadata": {
    "tags": []
   },
   "source": [
    "### **参考文献**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b539d28-6395-49a6-baae-4ac6a7b0a4ba",
   "metadata": {},
   "source": [
    "- 東京大学教養学部統計学教室編『統計学入門』(東京大学出版会、1991年)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67cea9bb-7c64-4647-bab5-197e260abd74",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}