OLS.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "MathJax.Hub.Config({\n",
       "    TeX: { equationNumbers: { autoNumber: \"AMS\" } }\n",
       "});\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%javascript\n",
    "MathJax.Hub.Config({\n",
    "    TeX: { equationNumbers: { autoNumber: \"AMS\" } }\n",
    "});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run for interactive plots with jupyter lab\n",
    "%matplotlib widget          \n",
    "\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import pandas as pd\n",
    "from scipy import stats\n",
    "import statsmodels.api as sm\n",
    "\n",
    "import matplotlib\n",
    "# matplotlib.use('nbagg')    # Doesn't work with jupyter lab\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ordinary Least Squares\n",
    "\n",
    "In multiple regression, we have a model of the form\n",
    "\\begin{equation}\n",
    "\\mathbf{y = X \\boldsymbol{\\beta + \\epsilon}} \\quad \\boldsymbol{\\epsilon} \\sim N(\\mathbf{0},\\sigma\\mathbf{I})\n",
    "\\end{equation}\n",
    "\n",
    "where $\\mathbf{y}$ is an $n\\times 1$ vector, $X$ is an $n \\times k$ design matrix, $\\boldsymbol{\\beta}$ is the vector of $k$ coefficients, and $\\boldsymbol{\\epsilon}$ is the $n \\times 1$ vector of standard normal random errors.\n",
    "\n",
    "We derive an estimate $\\mathbf{b}$ of $\\boldsymbol{\\beta}$ from a sample of observations for which\n",
    "\\begin{equation}\n",
    "\\label{eq-Reg}\n",
    "\\mathbf{y = Xb + e}\n",
    "\\end{equation}\n",
    "by minimizing the squared sum of errors $\\mathbf{e'e}$. The estimate is given by\n",
    "\\begin{equation}\n",
    "\\label{eq-b}\n",
    "\\mathbf{b = \\left(X'X\\right)^{-1} X'y}\n",
    "\\end{equation}\n",
    "\n",
    "Plugging equation (\\ref{eq-b}) into equation (\\ref{eq-Reg}), we get\n",
    "\\begin{equation}\n",
    "\\label{eq-H}\n",
    "\\mathbf{y = X \\left(X'X\\right)^{-1} X'y + e \\equiv Hy + e }\n",
    "\\end{equation}\n",
    "\n",
    "The predicted values $\\mathbf{Hy}$ are often denoted $\\mathbf{\\hat{y}}$, which explains the name \"hat matrix\" for $\\mathbf{H}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DescribeResult(nobs=30, minmax=(0.04995345894608716, 0.9648400471483856), mean=0.5558004906644833, variance=0.0770657116798012, skewness=-0.23634961375889468, kurtosis=-1.0681272762337057)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                      y   R-squared:                       0.182\n",
      "Model:                            OLS   Adj. R-squared:                  0.121\n",
      "Method:                 Least Squares   F-statistic:                     3.002\n",
      "Date:                Fri, 22 May 2020   Prob (F-statistic):             0.0665\n",
      "Time:                        16:25:24   Log-Likelihood:                -3.4473\n",
      "No. Observations:                  30   AIC:                             12.89\n",
      "Df Residuals:                      27   BIC:                             17.10\n",
      "Df Model:                           2                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "==============================================================================\n",
      "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
      "------------------------------------------------------------------------------\n",
      "const          0.5371      0.120      4.490      0.000       0.292       0.783\n",
      "x1             0.2963      0.177      1.674      0.106      -0.067       0.659\n",
      "x2             0.2649      0.192      1.379      0.179      -0.129       0.659\n",
      "==============================================================================\n",
      "Omnibus:                        4.062   Durbin-Watson:                   2.074\n",
      "Prob(Omnibus):                  0.131   Jarque-Bera (JB):                1.944\n",
      "Skew:                          -0.310   Prob(JB):                        0.378\n",
      "Kurtosis:                       1.918   Cond. No.                         4.94\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
     ]
    }
   ],
   "source": [
    "# Creating artificial data for regression\n",
    "\n",
    "np.random.seed(1)       # For reproducibility\n",
    "X = np.random.random((30,2))\n",
    "b = [0.2,0.3]\n",
    "e = np.random.random(30)\n",
    "\n",
    "y = np.dot(X,b) + e\n",
    "\n",
    "stats.describe(e)\n",
    "# Running the regression\n",
    "\n",
    "X = sm.add_constant(X)        # Adding an intercept to the regression\n",
    "\n",
    "m1 = sm.OLS(y,X).fit()\n",
    "print(m1.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Diagnostics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### R squared, F stat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    }
   },
   "source": [
    "The regression has $n$ observations, $\\mathbf{X}$ is an $n\\times k$ matrix that has $p=k-1$ predictors and includes an intercept.\n",
    "\n",
    "**R squared and adjusted R squared**\n",
    "\n",
    "Regression can be analyzed in terms of an attribution of the sum of squared deviations of the dependent or response variable $\\sum_i (y_i-\\bar{y})^2$. Fot this, since $y_i = \\hat{y}_i + e_i$, we note that\n",
    "$$\n",
    "\\sum_i (y_i-\\bar{y})^2 = \\sum_i (\\hat{y}_i - \\bar{y} +e_i)^2 = \\sum_i (\\hat{y}_i-\\bar{y})^2 + \\sum_i e_i^2 \n",
    "$$\n",
    "since $\\sum_i \\hat{y}_i e_i = 0$ and $\\sum_i e_i = 0$.\n",
    "\n",
    "This is often written as\n",
    "$$\n",
    "\\text{Total sum of squares (SST)} = \\text{Regression sum of squares (SSR)} + \\text{Error sum of squares (SSE)}\n",
    "$$\n",
    "\n",
    "The coefficient of determination, $R^2$ is defined as\n",
    "$$\n",
    "R^2 = \\frac{SSR}{SST} = 1 - \\frac{SSE}{SST}\n",
    "$$\n",
    "\n",
    "$R^2$ gives us the fraction of the total variation in $y$ accounted for by the regression and is used as a measure of the goodness of fit for the model. One of its drawbacks is that it always increases with the addition of a new variable, because SSE decreases [For a proof, see Greene, William H., *Econometric Analysis*,4th edition, p. 235]. For this reason, an adjusted value is defined:\n",
    "\n",
    "$$\n",
    "R^2_{adj} = 1 - \\frac{SSE/(n-k)}{SST/(n-1)} = 1 - \\frac{n-1}{n-k}\\;(1-R^2)\n",
    "$$\n",
    "\n",
    "\n",
    "**The F statistic in multiple regression**\n",
    "\n",
    " The F statistic is\n",
    "$$\n",
    "F = \\frac{\\frac{SSR}{k-1}}{\\frac{SSE}{[n-k]}} = \\frac{MSR}{MSE}\n",
    "$$\n",
    "\n",
    "where $SSR$ is the regression sum of squares, $SSE$ is the sum of squared errors, and $MSR$ and $MSE$ are their means respectively.\n",
    "\n",
    "Since $R^2 = \\frac{SSR}{SST}$,\n",
    "$$\n",
    "\\begin{split}\n",
    "F &= \\frac{SSR}{SSE} \\frac{[n-k]}{k-1}\\\\\n",
    "  & = \\frac{R^2}{1-R^2}\\frac{[n-k]}{k-1}\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "**Degrees of freedom**\n",
    "\n",
    "|Source | Degrees of Freedom |\n",
    "|---        |---   |\n",
    "| Regression | $p=k-1$ |\n",
    "| Error      | $n-k$   |\n",
    "| Total      | $n-1$   |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "jupyter": {
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SST =  2.7018948326204675 SSR =  0.4915620365070974 SSE =  2.21033279611337 SSR + SSE =  2.7018948326204675\n",
      "\n",
      "R-squared check:  0.18193233525316366 0.18193233525316366\n",
      "\n",
      "Adjusted R-squared check:  0.12133473045710164 0.12133473045710164\n",
      "\n",
      "F stat check:  3.0023024155071365 3.0023024155071365\n",
      "\n",
      "Prob(F) check:  0.06647442100717305 0.0664744210071729\n"
     ]
    }
   ],
   "source": [
    "# Sums of squares, R^2, adjusted R^2, and F stat for overall model fit\n",
    "\n",
    "# SSTotal = SSReg + SSError ;   R^2 = SSReg/SSTotal ; F = (SSReg/df_reg)/(SSError/df_resid) \n",
    "\n",
    "# statsmodels uses confusing variable names\n",
    "# SSReg --> ess   and SSError --> ssr\n",
    "# centered_tss = ess (explained sum of squares) + ssr (sum of squared residuals)\n",
    "\n",
    "print(\"SST = \",m1.centered_tss, \"SSR = \", m1.ess, \"SSE = \", m1.ssr, \"SSR + SSE = \", m1.ess+m1.ssr) \n",
    "\n",
    "#m1.df_model, m1.df_resid         # degrees of freedom in case we need them\n",
    "\n",
    "# Confirm rsquared calculations\n",
    "\n",
    "r2 = m1.ess/m1.centered_tss\n",
    "print(\"\\nR-squared check: \",r2,m1.rsquared)  \n",
    "\n",
    "r2a = 1-(m1.ssr/m1.df_resid)/(m1.centered_tss/(m1.df_model+m1.df_resid))\n",
    "print(\"\\nAdjusted R-squared check: \",r2a,m1.rsquared_adj)\n",
    "\n",
    "# Confirm F statistic calculation\n",
    "F = m1.mse_model/m1.mse_resid\n",
    "Fpvalue = 1-stats.f.cdf(F,m1.df_model,m1.df_resid)\n",
    "\n",
    "print(\"\\nF stat check: \",F, m1.fvalue   ) \n",
    "print(\"\\nProb(F) check: \",Fpvalue, m1.f_pvalue)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model selection: Log likelihood, AIC, BIC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normality of residuals: Jarque-Bera test\n",
    "\n",
    "[Wikipedia entry](https://en.wikipedia.org/wiki/Jarque%E2%80%93Bera_test).\n",
    "\n",
    "$$\n",
    "JB ={\\frac {n}{6}}\\left(S^{2}+{\\frac {1}{4}}(K-3)^{2}\\right)\n",
    "$$\n",
    "where\n",
    "$$\n",
    "\\begin{split}\n",
    "S &= {\\frac{{\\hat{\\mu }}_{3}}{{\\hat  {\\sigma }}^{3}}}={\\frac  {{\\frac  1n}\\sum _{{i=1}}^{n}(x_{i}-{\\bar  {x}})^{3}}{\\left({\\frac  1n}\\sum _{{i=1}}^{n}(x_{i}-{\\bar  {x}})^{2}\\right)^{{3/2}}}}\\\\\n",
    "K &= {\\frac{{\\hat{\\mu }}_{4}}{{\\hat  {\\sigma }}^{4}}} ={\\frac  {{\\frac  1n}\\sum _{{i=1}}^{n}(x_{i}-{\\bar  {x}})^{4}}{\\left({\\frac  1n}\\sum _{{i=1}}^{n}(x_{i}-{\\bar  {x}})^{2}\\right)^{{2}}}}\n",
    "\\end{split}\n",
    "$$\n",
    "\n",
    "The test statistic $JB$ is approximately $\\chi^2$ distributed with 2 degrees of freedom for large $n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "jupyter": {
     "source_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Skewness =  -0.31020796231133496 Kurtosis =  1.9180548173303456 Test Statistic =  1.944401622284343\n",
      "p value =  0.378249665313768\n"
     ]
    }
   ],
   "source": [
    "# JB statistic for regression residuals\n",
    "x = m1.resid\n",
    "n = np.shape(x)[0]\n",
    "\n",
    "xdev = x - np.mean(x)\n",
    "mvec = np.zeros(3)\n",
    "for i in range(len(mvec)):\n",
    "    mvec[i] = np.sum(xdev**(i+2))/n\n",
    "    \n",
    "M = dict(zip(['sigma2','mu3','mu4'],mvec))\n",
    "         \n",
    "S = M['mu3']/(M['sigma2']**(3/2))\n",
    "K = M['mu4']/(M['sigma2']**2)\n",
    "\n",
    "JB = (n/6)*(S**2 + (1/4)*(K-3)**2)   # H0: S=0, K-3=0 (sample data are from normal distribution)\n",
    "\n",
    "print(\"Skewness = \",S,\"Kurtosis = \",K,\"Test Statistic = \",JB)\n",
    "print(\"p value = \",1-stats.chi2.cdf(JB,2))   # p value = Prob (X >= JB) in chi-squared distribution with dof=2\n",
    "                                             # Large p-value indicates that we fail to reject the null hypothesis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Autocorrelation of residuals: Durbin-Watson test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Residual plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b20d04f5f2594de7b7c2e48a44f62041",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Residual vs. predicted value\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111,title=\"Residual plot\",ylabel=r\"$e_i$\",xlabel=r\"$\\haty$\")\n",
    "\n",
    "y = m1.resid\n",
    "x = m1.predict()\n",
    "_=plt.scatter(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Omnibus Test\n",
    "\n",
    "This test is for the normality of the distribution of residuals. [See Wikipedia entry](https://en.wikipedia.org/wiki/D%27Agostino%27s_K-squared_test)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NormaltestResult(statistic=4.062075177125114, pvalue=0.13119931958670275)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Omnibus test diagnostic reported by summary\n",
    "stats.normaltest(m1.resid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function fit in module statsmodels.regression.linear_model:\n",
      "\n",
      "fit(self, method='pinv', cov_type='nonrobust', cov_kwds=None, use_t=None, **kwargs)\n",
      "    Full fit of the model.\n",
      "    \n",
      "    The results include an estimate of covariance matrix, (whitened)\n",
      "    residuals and an estimate of scale.\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    method : str, optional\n",
      "        Can be \"pinv\", \"qr\".  \"pinv\" uses the Moore-Penrose pseudoinverse\n",
      "        to solve the least squares problem. \"qr\" uses the QR\n",
      "        factorization.\n",
      "    cov_type : str, optional\n",
      "        See `regression.linear_model.RegressionResults` for a description\n",
      "        of the available covariance estimators.\n",
      "    cov_kwds : list or None, optional\n",
      "        See `linear_model.RegressionResults.get_robustcov_results` for a\n",
      "        description required keywords for alternative covariance\n",
      "        estimators.\n",
      "    use_t : bool, optional\n",
      "        Flag indicating to use the Student's t distribution when computing\n",
      "        p-values.  Default behavior depends on cov_type. See\n",
      "        `linear_model.RegressionResults.get_robustcov_results` for\n",
      "        implementation details.\n",
      "    **kwargs\n",
      "        Additional keyword arguments that contain information used when\n",
      "        constructing a model using the formula interface.\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    RegressionResults\n",
      "        The model estimation results.\n",
      "    \n",
      "    See Also\n",
      "    --------\n",
      "    RegressionResults\n",
      "        The results container.\n",
      "    RegressionResults.get_robustcov_results\n",
      "        A method to change the covariance estimator used when fitting the\n",
      "        model.\n",
      "    \n",
      "    Notes\n",
      "    -----\n",
      "    The fit method uses the pseudoinverse of the design/exogenous variables\n",
      "    to solve the least squares minimization.\n",
      "\n",
      "Help on class RegressionResults in module statsmodels.regression.linear_model:\n",
      "\n",
      "class RegressionResults(statsmodels.base.model.LikelihoodModelResults)\n",
      " |  RegressionResults(model, params, normalized_cov_params=None, scale=1.0, cov_type='nonrobust', cov_kwds=None, use_t=None, **kwargs)\n",
      " |  \n",
      " |  This class summarizes the fit of a linear regression model.\n",
      " |  \n",
      " |  It handles the output of contrasts, estimates of covariance, etc.\n",
      " |  \n",
      " |  Parameters\n",
      " |  ----------\n",
      " |  model : RegressionModel\n",
      " |      The regression model instance.\n",
      " |  params : ndarray\n",
      " |      The estimated parameters.\n",
      " |  normalized_cov_params : ndarray\n",
      " |      The normalized covariance parameters.\n",
      " |  scale : float\n",
      " |      The estimated scale of the residuals.\n",
      " |  cov_type : str\n",
      " |      The covariance estimator used in the results.\n",
      " |  cov_kwds : dict\n",
      " |      Additional keywords used in the covariance specification.\n",
      " |  use_t : bool\n",
      " |      Flag indicating to use the Student's t in inference.\n",
      " |  **kwargs\n",
      " |      Additional keyword arguments used to initialize the results.\n",
      " |  \n",
      " |  Attributes\n",
      " |  ----------\n",
      " |  pinv_wexog\n",
      " |      See model class docstring for implementation details.\n",
      " |  cov_type\n",
      " |      Parameter covariance estimator used for standard errors and t-stats.\n",
      " |  df_model\n",
      " |      Model degrees of freedom. The number of regressors `p`. Does not\n",
      " |      include the constant if one is present.\n",
      " |  df_resid\n",
      " |      Residual degrees of freedom. `n - p - 1`, if a constant is present.\n",
      " |      `n - p` if a constant is not included.\n",
      " |  het_scale\n",
      " |      adjusted squared residuals for heteroscedasticity robust standard\n",
      " |      errors. Is only available after `HC#_se` or `cov_HC#` is called.\n",
      " |      See HC#_se for more information.\n",
      " |  history\n",
      " |      Estimation history for iterative estimators.\n",
      " |  model\n",
      " |      A pointer to the model instance that called fit() or results.\n",
      " |  params\n",
      " |      The linear coefficients that minimize the least squares\n",
      " |      criterion.  This is usually called Beta for the classical\n",
      " |      linear model.\n",
      " |  \n",
      " |  Method resolution order:\n",
      " |      RegressionResults\n",
      " |      statsmodels.base.model.LikelihoodModelResults\n",
      " |      statsmodels.base.model.Results\n",
      " |      builtins.object\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, model, params, normalized_cov_params=None, scale=1.0, cov_type='nonrobust', cov_kwds=None, use_t=None, **kwargs)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  compare_f_test(self, restricted)\n",
      " |      Use F test to test whether restricted model is correct.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      restricted : Result instance\n",
      " |          The restricted model is assumed to be nested in the\n",
      " |          current model. The result instance of the restricted model\n",
      " |          is required to have two attributes, residual sum of\n",
      " |          squares, `ssr`, residual degrees of freedom, `df_resid`.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      f_value : float\n",
      " |          The test statistic which has an F distribution.\n",
      " |      p_value : float\n",
      " |          The p-value of the test statistic.\n",
      " |      df_diff : int\n",
      " |          The degrees of freedom of the restriction, i.e. difference in\n",
      " |          df between models.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      See mailing list discussion October 17,\n",
      " |      \n",
      " |      This test compares the residual sum of squares of the two\n",
      " |      models.  This is not a valid test, if there is unspecified\n",
      " |      heteroscedasticity or correlation. This method will issue a\n",
      " |      warning if this is detected but still return the results under\n",
      " |      the assumption of homoscedasticity and no autocorrelation\n",
      " |      (sphericity).\n",
      " |  \n",
      " |  compare_lm_test(self, restricted, demean=True, use_lr=False)\n",
      " |      Use Lagrange Multiplier test to test a set of linear restrictions.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      restricted : Result instance\n",
      " |          The restricted model is assumed to be nested in the\n",
      " |          current model. The result instance of the restricted model\n",
      " |          is required to have two attributes, residual sum of\n",
      " |          squares, `ssr`, residual degrees of freedom, `df_resid`.\n",
      " |      demean : bool\n",
      " |          Flag indicating whether the demean the scores based on the\n",
      " |          residuals from the restricted model.  If True, the covariance of\n",
      " |          the scores are used and the LM test is identical to the large\n",
      " |          sample version of the LR test.\n",
      " |      use_lr : bool\n",
      " |          A flag indicating whether to estimate the covariance of the model\n",
      " |          scores using the unrestricted model. Setting the to True improves\n",
      " |          the power of the test.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      lm_value : float\n",
      " |          The test statistic which has a chi2 distributed.\n",
      " |      p_value : float\n",
      " |          The p-value of the test statistic.\n",
      " |      df_diff : int\n",
      " |          The degrees of freedom of the restriction, i.e. difference in df\n",
      " |          between models.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The LM test examines whether the scores from the restricted model are\n",
      " |      0. If the null is true, and the restrictions are valid, then the\n",
      " |      parameters of the restricted model should be close to the minimum of\n",
      " |      the sum of squared errors, and so the scores should be close to zero,\n",
      " |      on average.\n",
      " |  \n",
      " |  compare_lr_test(self, restricted, large_sample=False)\n",
      " |      Likelihood ratio test to test whether restricted model is correct.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      restricted : Result instance\n",
      " |          The restricted model is assumed to be nested in the current model.\n",
      " |          The result instance of the restricted model is required to have two\n",
      " |          attributes, residual sum of squares, `ssr`, residual degrees of\n",
      " |          freedom, `df_resid`.\n",
      " |      \n",
      " |      large_sample : bool\n",
      " |          Flag indicating whether to use a heteroskedasticity robust version\n",
      " |          of the LR test, which is a modified LM test.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      lr_stat : float\n",
      " |          The likelihood ratio which is chisquare distributed with df_diff\n",
      " |          degrees of freedom.\n",
      " |      p_value : float\n",
      " |          The p-value of the test statistic.\n",
      " |      df_diff : int\n",
      " |          The degrees of freedom of the restriction, i.e. difference in df\n",
      " |          between models.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The exact likelihood ratio is valid for homoskedastic data,\n",
      " |      and is defined as\n",
      " |      \n",
      " |      .. math:: D=-2\\log\\left(\\frac{\\mathcal{L}_{null}}\n",
      " |         {\\mathcal{L}_{alternative}}\\right)\n",
      " |      \n",
      " |      where :math:`\\mathcal{L}` is the likelihood of the\n",
      " |      model. With :math:`D` distributed as chisquare with df equal\n",
      " |      to difference in number of parameters or equivalently\n",
      " |      difference in residual degrees of freedom.\n",
      " |      \n",
      " |      The large sample version of the likelihood ratio is defined as\n",
      " |      \n",
      " |      .. math:: D=n s^{\\prime}S^{-1}s\n",
      " |      \n",
      " |      where :math:`s=n^{-1}\\sum_{i=1}^{n} s_{i}`\n",
      " |      \n",
      " |      .. math:: s_{i} = x_{i,alternative} \\epsilon_{i,null}\n",
      " |      \n",
      " |      is the average score of the model evaluated using the\n",
      " |      residuals from null model and the regressors from the\n",
      " |      alternative model and :math:`S` is the covariance of the\n",
      " |      scores, :math:`s_{i}`.  The covariance of the scores is\n",
      " |      estimated using the same estimator as in the alternative\n",
      " |      model.\n",
      " |      \n",
      " |      This test compares the loglikelihood of the two models.  This\n",
      " |      may not be a valid test, if there is unspecified\n",
      " |      heteroscedasticity or correlation. This method will issue a\n",
      " |      warning if this is detected but still return the results\n",
      " |      without taking unspecified heteroscedasticity or correlation\n",
      " |      into account.\n",
      " |      \n",
      " |      This test compares the loglikelihood of the two models.  This\n",
      " |      may not be a valid test, if there is unspecified\n",
      " |      heteroscedasticity or correlation. This method will issue a\n",
      " |      warning if this is detected but still return the results\n",
      " |      without taking unspecified heteroscedasticity or correlation\n",
      " |      into account.\n",
      " |      \n",
      " |      is the average score of the model evaluated using the\n",
      " |      residuals from null model and the regressors from the\n",
      " |      alternative model and :math:`S` is the covariance of the\n",
      " |      scores, :math:`s_{i}`.  The covariance of the scores is\n",
      " |      estimated using the same estimator as in the alternative\n",
      " |      model.\n",
      " |  \n",
      " |  conf_int(self, alpha=0.05, cols=None)\n",
      " |      Compute the confidence interval of the fitted parameters.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      alpha : float, optional\n",
      " |          The `alpha` level for the confidence interval. The default\n",
      " |          `alpha` = .05 returns a 95% confidence interval.\n",
      " |      cols : array_like, optional\n",
      " |          Columns to included in returned confidence intervals.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      array_like\n",
      " |          The confidence intervals.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The confidence interval is based on Student's t-distribution.\n",
      " |  \n",
      " |  get_prediction(self, exog=None, transform=True, weights=None, row_labels=None, **kwargs)\n",
      " |      Compute prediction results.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      exog : array_like, optional\n",
      " |          The values for which you want to predict.\n",
      " |      transform : bool, optional\n",
      " |          If the model was fit via a formula, do you want to pass\n",
      " |          exog through the formula. Default is True. E.g., if you fit\n",
      " |          a model y ~ log(x1) + log(x2), and transform is True, then\n",
      " |          you can pass a data structure that contains x1 and x2 in\n",
      " |          their original form. Otherwise, you'd need to log the data\n",
      " |          first.\n",
      " |      weights : array_like, optional\n",
      " |          Weights interpreted as in WLS, used for the variance of the predicted\n",
      " |          residual.\n",
      " |      row_labels : list\n",
      " |          A list of row labels to use.  If not provided, read `exog` is\n",
      " |          available.\n",
      " |      **kwargs\n",
      " |          Some models can take additional keyword arguments, see the predict\n",
      " |          method of the model for the details.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      linear_model.PredictionResults\n",
      " |          The prediction results instance contains prediction and prediction\n",
      " |          variance and can on demand calculate confidence intervals and summary\n",
      " |          tables for the prediction of the mean and of new observations.\n",
      " |  \n",
      " |  get_robustcov_results(self, cov_type='HC1', use_t=None, **kwargs)\n",
      " |      Create new results instance with robust covariance as default.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      cov_type : str\n",
      " |          The type of robust sandwich estimator to use. See Notes below.\n",
      " |      use_t : bool\n",
      " |          If true, then the t distribution is used for inference.\n",
      " |          If false, then the normal distribution is used.\n",
      " |          If `use_t` is None, then an appropriate default is used, which is\n",
      " |          `True` if the cov_type is nonrobust, and `False` in all other\n",
      " |          cases.\n",
      " |      **kwargs\n",
      " |          Required or optional arguments for robust covariance calculation.\n",
      " |          See Notes below.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      RegressionResults\n",
      " |          This method creates a new results instance with the\n",
      " |          requested robust covariance as the default covariance of\n",
      " |          the parameters.  Inferential statistics like p-values and\n",
      " |          hypothesis tests will be based on this covariance matrix.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The following covariance types and required or optional arguments are\n",
      " |      currently available:\n",
      " |      \n",
      " |      - 'fixed scale' and optional keyword argument 'scale' which uses\n",
      " |          a predefined scale estimate with default equal to one.\n",
      " |      - 'HC0', 'HC1', 'HC2', 'HC3' and no keyword arguments:\n",
      " |          heteroscedasticity robust covariance\n",
      " |      - 'HAC' and keywords\n",
      " |      \n",
      " |          - `maxlag` integer (required) : number of lags to use\n",
      " |          - `kernel` callable or str (optional) : kernel\n",
      " |                currently available kernels are ['bartlett', 'uniform'],\n",
      " |                default is Bartlett\n",
      " |          - `use_correction` bool (optional) : If true, use small sample\n",
      " |                correction\n",
      " |      \n",
      " |      - 'cluster' and required keyword `groups`, integer group indicator\n",
      " |      \n",
      " |          - `groups` array_like, integer (required) :\n",
      " |                index of clusters or groups\n",
      " |          - `use_correction` bool (optional) :\n",
      " |                If True the sandwich covariance is calculated with a small\n",
      " |                sample correction.\n",
      " |                If False the sandwich covariance is calculated without\n",
      " |                small sample correction.\n",
      " |          - `df_correction` bool (optional)\n",
      " |                If True (default), then the degrees of freedom for the\n",
      " |                inferential statistics and hypothesis tests, such as\n",
      " |                pvalues, f_pvalue, conf_int, and t_test and f_test, are\n",
      " |                based on the number of groups minus one instead of the\n",
      " |                total number of observations minus the number of explanatory\n",
      " |                variables. `df_resid` of the results instance is adjusted.\n",
      " |                If False, then `df_resid` of the results instance is not\n",
      " |                adjusted.\n",
      " |      \n",
      " |      - 'hac-groupsum' Driscoll and Kraay, heteroscedasticity and\n",
      " |          autocorrelation robust standard errors in panel data\n",
      " |          keywords\n",
      " |      \n",
      " |          - `time` array_like (required) : index of time periods\n",
      " |          - `maxlag` integer (required) : number of lags to use\n",
      " |          - `kernel` callable or str (optional). The available kernels\n",
      " |            are ['bartlett', 'uniform']. The default is Bartlett.\n",
      " |          - `use_correction` False or string in ['hac', 'cluster'] (optional).\n",
      " |            If False the the sandwich covariance is calculated without small\n",
      " |            sample correction. If `use_correction = 'cluster'` (default),\n",
      " |            then the same small sample correction as in the case of\n",
      " |            `covtype='cluster'` is used.\n",
      " |          - `df_correction` bool (optional) The adjustment to df_resid, see\n",
      " |            cov_type 'cluster' above\n",
      " |            # TODO: we need more options here\n",
      " |      \n",
      " |      - 'hac-panel' heteroscedasticity and autocorrelation robust standard\n",
      " |          errors in panel data.\n",
      " |          The data needs to be sorted in this case, the time series\n",
      " |          for each panel unit or cluster need to be stacked. The\n",
      " |          membership to a timeseries of an individual or group can\n",
      " |          be either specified by group indicators or by increasing\n",
      " |          time periods.\n",
      " |      \n",
      " |          keywords\n",
      " |      \n",
      " |          - either `groups` or `time` : array_like (required)\n",
      " |            `groups` : indicator for groups\n",
      " |            `time` : index of time periods\n",
      " |          - `maxlag` integer (required) : number of lags to use\n",
      " |          - `kernel` callable or str (optional)\n",
      " |                currently available kernels are ['bartlett', 'uniform'],\n",
      " |                default is Bartlett\n",
      " |          - `use_correction` False or string in ['hac', 'cluster'] (optional)\n",
      " |                If False the sandwich covariance is calculated without\n",
      " |                small sample correction.\n",
      " |          - `df_correction` bool (optional)\n",
      " |                adjustment to df_resid, see cov_type 'cluster' above\n",
      " |                # TODO: we need more options here\n",
      " |      \n",
      " |      Reminder:\n",
      " |      `use_correction` in \"hac-groupsum\" and \"hac-panel\" is not bool,\n",
      " |      needs to be in [False, 'hac', 'cluster']\n",
      " |      \n",
      " |      TODO: Currently there is no check for extra or misspelled keywords,\n",
      " |      except in the case of cov_type `HCx`\n",
      " |  \n",
      " |  scale(self)\n",
      " |      A scale factor for the covariance matrix.\n",
      " |      \n",
      " |      The Default value is ssr/(n-p).  Note that the square root of `scale`\n",
      " |      is often called the standard error of the regression.\n",
      " |  \n",
      " |  summary(self, yname=None, xname=None, title=None, alpha=0.05)\n",
      " |      Summarize the Regression Results.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      yname : str, optional\n",
      " |          Name of endogenous (response) variable. The Default is `y`.\n",
      " |      xname : list[str], optional\n",
      " |          Names for the exogenous variables. Default is `var_##` for ## in\n",
      " |          the number of regressors. Must match the number of parameters\n",
      " |          in the model.\n",
      " |      title : str, optional\n",
      " |          Title for the top table. If not None, then this replaces the\n",
      " |          default title.\n",
      " |      alpha : float\n",
      " |          The significance level for the confidence intervals.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      Summary\n",
      " |          Instance holding the summary tables and text, which can be printed\n",
      " |          or converted to various output formats.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      statsmodels.iolib.summary.Summary : A class that holds summary results.\n",
      " |  \n",
      " |  summary2(self, yname=None, xname=None, title=None, alpha=0.05, float_format='%.4f')\n",
      " |      Experimental summary function to summarize the regression results.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      yname : str\n",
      " |          The name of the dependent variable (optional).\n",
      " |      xname : list[str], optional\n",
      " |          Names for the exogenous variables. Default is `var_##` for ## in\n",
      " |          the number of regressors. Must match the number of parameters\n",
      " |          in the model.\n",
      " |      title : str, optional\n",
      " |          Title for the top table. If not None, then this replaces the\n",
      " |          default title.\n",
      " |      alpha : float\n",
      " |          The significance level for the confidence intervals.\n",
      " |      float_format : str\n",
      " |          The format for floats in parameters summary.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      Summary\n",
      " |          Instance holding the summary tables and text, which can be printed\n",
      " |          or converted to various output formats.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      statsmodels.iolib.summary2.Summary\n",
      " |          A class that holds summary results.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  HC0_se\n",
      " |      White's (1980) heteroskedasticity robust standard errors.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      Defined as sqrt(diag(X.T X)^(-1)X.T diag(e_i^(2)) X(X.T X)^(-1)\n",
      " |      where e_i = resid[i].\n",
      " |      \n",
      " |      When HC0_se or cov_HC0 is called the RegressionResults instance will\n",
      " |      then have another attribute `het_scale`, which is in this case is just\n",
      " |      resid**2.\n",
      " |  \n",
      " |  HC1_se\n",
      " |      MacKinnon and White's (1985) heteroskedasticity robust standard errors.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      Defined as sqrt(diag(n/(n-p)*HC_0).\n",
      " |      \n",
      " |      When HC1_se or cov_HC1 is called the RegressionResults instance will\n",
      " |      then have another attribute `het_scale`, which is in this case is\n",
      " |      n/(n-p)*resid**2.\n",
      " |  \n",
      " |  HC2_se\n",
      " |      MacKinnon and White's (1985) heteroskedasticity robust standard errors.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)) X(X.T X)^(-1)\n",
      " |      where h_ii = x_i(X.T X)^(-1)x_i.T\n",
      " |      \n",
      " |      When HC2_se or cov_HC2 is called the RegressionResults instance will\n",
      " |      then have another attribute `het_scale`, which is in this case is\n",
      " |      resid^(2)/(1-h_ii).\n",
      " |  \n",
      " |  HC3_se\n",
      " |      MacKinnon and White's (1985) heteroskedasticity robust standard errors.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)^(2)) X(X.T X)^(-1)\n",
      " |      where h_ii = x_i(X.T X)^(-1)x_i.T.\n",
      " |      \n",
      " |      When HC3_se or cov_HC3 is called the RegressionResults instance will\n",
      " |      then have another attribute `het_scale`, which is in this case is\n",
      " |      resid^(2)/(1-h_ii)^(2).\n",
      " |  \n",
      " |  aic\n",
      " |      Akaike's information criteria.\n",
      " |      \n",
      " |      For a model with a constant :math:`-2llf + 2(df\\_model + 1)`. For a\n",
      " |      model without a constant :math:`-2llf + 2(df\\_model)`.\n",
      " |  \n",
      " |  bic\n",
      " |      Bayes' information criteria.\n",
      " |      \n",
      " |      For a model with a constant :math:`-2llf + \\log(n)(df\\_model+1)`.\n",
      " |      For a model without a constant :math:`-2llf + \\log(n)(df\\_model)`.\n",
      " |  \n",
      " |  bse\n",
      " |      The standard errors of the parameter estimates.\n",
      " |  \n",
      " |  centered_tss\n",
      " |      The total (weighted) sum of squares centered about the mean.\n",
      " |  \n",
      " |  condition_number\n",
      " |      Return condition number of exogenous matrix.\n",
      " |      \n",
      " |      Calculated as ratio of largest to smallest eigenvalue.\n",
      " |  \n",
      " |  cov_HC0\n",
      " |      Heteroscedasticity robust covariance matrix. See HC0_se.\n",
      " |  \n",
      " |  cov_HC1\n",
      " |      Heteroscedasticity robust covariance matrix. See HC1_se.\n",
      " |  \n",
      " |  cov_HC2\n",
      " |      Heteroscedasticity robust covariance matrix. See HC2_se.\n",
      " |  \n",
      " |  cov_HC3\n",
      " |      Heteroscedasticity robust covariance matrix. See HC3_se.\n",
      " |  \n",
      " |  eigenvals\n",
      " |      Return eigenvalues sorted in decreasing order.\n",
      " |  \n",
      " |  ess\n",
      " |      The explained sum of squares.\n",
      " |      \n",
      " |      If a constant is present, the centered total sum of squares minus the\n",
      " |      sum of squared residuals. If there is no constant, the uncentered total\n",
      " |      sum of squares is used.\n",
      " |  \n",
      " |  f_pvalue\n",
      " |      The p-value of the F-statistic.\n",
      " |  \n",
      " |  fittedvalues\n",
      " |      The predicted values for the original (unwhitened) design.\n",
      " |  \n",
      " |  fvalue\n",
      " |      F-statistic of the fully specified model.\n",
      " |      \n",
      " |      Calculated as the mean squared error of the model divided by the mean\n",
      " |      squared error of the residuals if the nonrobust covariance is used.\n",
      " |      Otherwise computed using a Wald-like quadratic form that tests whether\n",
      " |      all coefficients (excluding the constant) are zero.\n",
      " |  \n",
      " |  mse_model\n",
      " |      Mean squared error the model.\n",
      " |      \n",
      " |      The explained sum of squares divided by the model degrees of freedom.\n",
      " |  \n",
      " |  mse_resid\n",
      " |      Mean squared error of the residuals.\n",
      " |      \n",
      " |      The sum of squared residuals divided by the residual degrees of\n",
      " |      freedom.\n",
      " |  \n",
      " |  mse_total\n",
      " |      Total mean squared error.\n",
      " |      \n",
      " |      The uncentered total sum of squares divided by the number of\n",
      " |      observations.\n",
      " |  \n",
      " |  nobs\n",
      " |      Number of observations n.\n",
      " |  \n",
      " |  resid\n",
      " |      The residuals of the model.\n",
      " |  \n",
      " |  resid_pearson\n",
      " |      Residuals, normalized to have unit variance.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      array_like\n",
      " |          The array `wresid` normalized by the sqrt of the scale to have\n",
      " |          unit variance.\n",
      " |  \n",
      " |  rsquared\n",
      " |      R-squared of the model.\n",
      " |      \n",
      " |      This is defined here as 1 - `ssr`/`centered_tss` if the constant is\n",
      " |      included in the model and 1 - `ssr`/`uncentered_tss` if the constant is\n",
      " |      omitted.\n",
      " |  \n",
      " |  rsquared_adj\n",
      " |      Adjusted R-squared.\n",
      " |      \n",
      " |      This is defined here as 1 - (`nobs`-1)/`df_resid` * (1-`rsquared`)\n",
      " |      if a constant is included and 1 - `nobs`/`df_resid` * (1-`rsquared`) if\n",
      " |      no constant is included.\n",
      " |  \n",
      " |  ssr\n",
      " |      Sum of squared (whitened) residuals.\n",
      " |  \n",
      " |  uncentered_tss\n",
      " |      Uncentered sum of squares.\n",
      " |      \n",
      " |      The sum of the squared values of the (whitened) endogenous response\n",
      " |      variable.\n",
      " |  \n",
      " |  wresid\n",
      " |      The residuals of the transformed/whitened regressand and regressor(s).\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from statsmodels.base.model.LikelihoodModelResults:\n",
      " |  \n",
      " |  cov_params(self, r_matrix=None, column=None, scale=None, cov_p=None, other=None)\n",
      " |      Compute the variance/covariance matrix.\n",
      " |      \n",
      " |      The variance/covariance matrix can be of a linear contrast of the\n",
      " |      estimated parameters or all params multiplied by scale which will\n",
      " |      usually be an estimate of sigma^2.  Scale is assumed to be a scalar.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      r_matrix : array_like\n",
      " |          Can be 1d, or 2d.  Can be used alone or with other.\n",
      " |      column : array_like, optional\n",
      " |          Must be used on its own.  Can be 0d or 1d see below.\n",
      " |      scale : float, optional\n",
      " |          Can be specified or not.  Default is None, which means that\n",
      " |          the scale argument is taken from the model.\n",
      " |      cov_p : ndarray, optional\n",
      " |          The covariance of the parameters. If not provided, this value is\n",
      " |          read from `self.normalized_cov_params` or\n",
      " |          `self.cov_params_default`.\n",
      " |      other : array_like, optional\n",
      " |          Can be used when r_matrix is specified.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      ndarray\n",
      " |          The covariance matrix of the parameter estimates or of linear\n",
      " |          combination of parameter estimates. See Notes.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      (The below are assumed to be in matrix notation.)\n",
      " |      \n",
      " |      If no argument is specified returns the covariance matrix of a model\n",
      " |      ``(scale)*(X.T X)^(-1)``\n",
      " |      \n",
      " |      If contrast is specified it pre and post-multiplies as follows\n",
      " |      ``(scale) * r_matrix (X.T X)^(-1) r_matrix.T``\n",
      " |      \n",
      " |      If contrast and other are specified returns\n",
      " |      ``(scale) * r_matrix (X.T X)^(-1) other.T``\n",
      " |      \n",
      " |      If column is specified returns\n",
      " |      ``(scale) * (X.T X)^(-1)[column,column]`` if column is 0d\n",
      " |      \n",
      " |      OR\n",
      " |      \n",
      " |      ``(scale) * (X.T X)^(-1)[column][:,column]`` if column is 1d\n",
      " |  \n",
      " |  f_test(self, r_matrix, cov_p=None, scale=1.0, invcov=None)\n",
      " |      Compute the F-test for a joint linear hypothesis.\n",
      " |      \n",
      " |      This is a special case of `wald_test` that always uses the F\n",
      " |      distribution.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      r_matrix : {array_like, str, tuple}\n",
      " |          One of:\n",
      " |      \n",
      " |          - array : An r x k array where r is the number of restrictions to\n",
      " |            test and k is the number of regressors. It is assumed\n",
      " |            that the linear combination is equal to zero.\n",
      " |          - str : The full hypotheses to test can be given as a string.\n",
      " |            See the examples.\n",
      " |          - tuple : A tuple of arrays in the form (R, q), ``q`` can be\n",
      " |            either a scalar or a length k row vector.\n",
      " |      \n",
      " |      cov_p : array_like, optional\n",
      " |          An alternative estimate for the parameter covariance matrix.\n",
      " |          If None is given, self.normalized_cov_params is used.\n",
      " |      scale : float, optional\n",
      " |          Default is 1.0 for no scaling.\n",
      " |      \n",
      " |          .. deprecated:: 0.10.0\n",
      " |      \n",
      " |      invcov : array_like, optional\n",
      " |          A q x q array to specify an inverse covariance matrix based on a\n",
      " |          restrictions matrix.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      ContrastResults\n",
      " |          The results for the test are attributes of this results instance.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      t_test : Perform a single hypothesis test.\n",
      " |      wald_test : Perform a Wald-test using a quadratic form.\n",
      " |      statsmodels.stats.contrast.ContrastResults : Test results.\n",
      " |      patsy.DesignInfo.linear_constraint : Specify a linear constraint.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The matrix `r_matrix` is assumed to be non-singular. More precisely,\n",
      " |      \n",
      " |      r_matrix (pX pX.T) r_matrix.T\n",
      " |      \n",
      " |      is assumed invertible. Here, pX is the generalized inverse of the\n",
      " |      design matrix of the model. There can be problems in non-OLS models\n",
      " |      where the rank of the covariance of the noise is not full.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      >>> import numpy as np\n",
      " |      >>> import statsmodels.api as sm\n",
      " |      >>> data = sm.datasets.longley.load(as_pandas=False)\n",
      " |      >>> data.exog = sm.add_constant(data.exog)\n",
      " |      >>> results = sm.OLS(data.endog, data.exog).fit()\n",
      " |      >>> A = np.identity(len(results.params))\n",
      " |      >>> A = A[1:,:]\n",
      " |      \n",
      " |      This tests that each coefficient is jointly statistically\n",
      " |      significantly different from zero.\n",
      " |      \n",
      " |      >>> print(results.f_test(A))\n",
      " |      <F test: F=array([[ 330.28533923]]), p=4.984030528700946e-10, df_denom=9, df_num=6>\n",
      " |      \n",
      " |      Compare this to\n",
      " |      \n",
      " |      >>> results.fvalue\n",
      " |      330.2853392346658\n",
      " |      >>> results.f_pvalue\n",
      " |      4.98403096572e-10\n",
      " |      \n",
      " |      >>> B = np.array(([0,0,1,-1,0,0,0],[0,0,0,0,0,1,-1]))\n",
      " |      \n",
      " |      This tests that the coefficient on the 2nd and 3rd regressors are\n",
      " |      equal and jointly that the coefficient on the 5th and 6th regressors\n",
      " |      are equal.\n",
      " |      \n",
      " |      >>> print(results.f_test(B))\n",
      " |      <F test: F=array([[ 9.74046187]]), p=0.005605288531708235, df_denom=9, df_num=2>\n",
      " |      \n",
      " |      Alternatively, you can specify the hypothesis tests using a string\n",
      " |      \n",
      " |      >>> from statsmodels.datasets import longley\n",
      " |      >>> from statsmodels.formula.api import ols\n",
      " |      >>> dta = longley.load_pandas().data\n",
      " |      >>> formula = 'TOTEMP ~ GNPDEFL + GNP + UNEMP + ARMED + POP + YEAR'\n",
      " |      >>> results = ols(formula, dta).fit()\n",
      " |      >>> hypotheses = '(GNPDEFL = GNP), (UNEMP = 2), (YEAR/1829 = 1)'\n",
      " |      >>> f_test = results.f_test(hypotheses)\n",
      " |      >>> print(f_test)\n",
      " |      <F test: F=array([[ 144.17976065]]), p=6.322026217355609e-08, df_denom=9, df_num=3>\n",
      " |  \n",
      " |  normalized_cov_params(self)\n",
      " |      See specific model class docstring\n",
      " |  \n",
      " |  remove_data(self)\n",
      " |      Remove data arrays, all nobs arrays from result and model.\n",
      " |      \n",
      " |      This reduces the size of the instance, so it can be pickled with less\n",
      " |      memory. Currently tested for use with predict from an unpickled\n",
      " |      results and model instance.\n",
      " |      \n",
      " |      .. warning::\n",
      " |      \n",
      " |         Since data and some intermediate results have been removed\n",
      " |         calculating new statistics that require them will raise exceptions.\n",
      " |         The exception will occur the first time an attribute is accessed\n",
      " |         that has been set to None.\n",
      " |      \n",
      " |      Not fully tested for time series models, tsa, and might delete too much\n",
      " |      for prediction or not all that would be possible.\n",
      " |      \n",
      " |      The lists of arrays to delete are maintained as attributes of\n",
      " |      the result and model instance, except for cached values. These\n",
      " |      lists could be changed before calling remove_data.\n",
      " |      \n",
      " |      The attributes to remove are named in:\n",
      " |      \n",
      " |      model._data_attr : arrays attached to both the model instance\n",
      " |          and the results instance with the same attribute name.\n",
      " |      \n",
      " |      result.data_in_cache : arrays that may exist as values in\n",
      " |          result._cache (TODO : should privatize name)\n",
      " |      \n",
      " |      result._data_attr_model : arrays attached to the model\n",
      " |          instance but not to the results instance\n",
      " |  \n",
      " |  save(self, fname, remove_data=False)\n",
      " |      Save a pickle of this instance.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      fname : {str, handle}\n",
      " |          A string filename or a file handle.\n",
      " |      remove_data : bool\n",
      " |          If False (default), then the instance is pickled without changes.\n",
      " |          If True, then all arrays with length nobs are set to None before\n",
      " |          pickling. See the remove_data method.\n",
      " |          In some cases not all arrays will be set to None.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      If remove_data is true and the model result does not implement a\n",
      " |      remove_data method then this will raise an exception.\n",
      " |  \n",
      " |  t_test(self, r_matrix, cov_p=None, scale=None, use_t=None)\n",
      " |      Compute a t-test for a each linear hypothesis of the form Rb = q.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      r_matrix : {array_like, str, tuple}\n",
      " |          One of:\n",
      " |      \n",
      " |          - array : If an array is given, a p x k 2d array or length k 1d\n",
      " |            array specifying the linear restrictions. It is assumed\n",
      " |            that the linear combination is equal to zero.\n",
      " |          - str : The full hypotheses to test can be given as a string.\n",
      " |            See the examples.\n",
      " |          - tuple : A tuple of arrays in the form (R, q). If q is given,\n",
      " |            can be either a scalar or a length p row vector.\n",
      " |      \n",
      " |      cov_p : array_like, optional\n",
      " |          An alternative estimate for the parameter covariance matrix.\n",
      " |          If None is given, self.normalized_cov_params is used.\n",
      " |      scale : float, optional\n",
      " |          An optional `scale` to use.  Default is the scale specified\n",
      " |          by the model fit.\n",
      " |      \n",
      " |          .. deprecated:: 0.10.0\n",
      " |      \n",
      " |      use_t : bool, optional\n",
      " |          If use_t is None, then the default of the model is used. If use_t\n",
      " |          is True, then the p-values are based on the t distribution. If\n",
      " |          use_t is False, then the p-values are based on the normal\n",
      " |          distribution.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      ContrastResults\n",
      " |          The results for the test are attributes of this results instance.\n",
      " |          The available results have the same elements as the parameter table\n",
      " |          in `summary()`.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      tvalues : Individual t statistics for the estimated parameters.\n",
      " |      f_test : Perform an F tests on model parameters.\n",
      " |      patsy.DesignInfo.linear_constraint : Specify a linear constraint.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      >>> import numpy as np\n",
      " |      >>> import statsmodels.api as sm\n",
      " |      >>> data = sm.datasets.longley.load(as_pandas=False)\n",
      " |      >>> data.exog = sm.add_constant(data.exog)\n",
      " |      >>> results = sm.OLS(data.endog, data.exog).fit()\n",
      " |      >>> r = np.zeros_like(results.params)\n",
      " |      >>> r[5:] = [1,-1]\n",
      " |      >>> print(r)\n",
      " |      [ 0.  0.  0.  0.  0.  1. -1.]\n",
      " |      \n",
      " |      r tests that the coefficients on the 5th and 6th independent\n",
      " |      variable are the same.\n",
      " |      \n",
      " |      >>> T_test = results.t_test(r)\n",
      " |      >>> print(T_test)\n",
      " |                                   Test for Constraints\n",
      " |      ==============================================================================\n",
      " |                       coef    std err          t      P>|t|      [0.025      0.975]\n",
      " |      ------------------------------------------------------------------------------\n",
      " |      c0         -1829.2026    455.391     -4.017      0.003   -2859.368    -799.037\n",
      " |      ==============================================================================\n",
      " |      >>> T_test.effect\n",
      " |      -1829.2025687192481\n",
      " |      >>> T_test.sd\n",
      " |      455.39079425193762\n",
      " |      >>> T_test.tvalue\n",
      " |      -4.0167754636411717\n",
      " |      >>> T_test.pvalue\n",
      " |      0.0015163772380899498\n",
      " |      \n",
      " |      Alternatively, you can specify the hypothesis tests using a string\n",
      " |      \n",
      " |      >>> from statsmodels.formula.api import ols\n",
      " |      >>> dta = sm.datasets.longley.load_pandas().data\n",
      " |      >>> formula = 'TOTEMP ~ GNPDEFL + GNP + UNEMP + ARMED + POP + YEAR'\n",
      " |      >>> results = ols(formula, dta).fit()\n",
      " |      >>> hypotheses = 'GNPDEFL = GNP, UNEMP = 2, YEAR/1829 = 1'\n",
      " |      >>> t_test = results.t_test(hypotheses)\n",
      " |      >>> print(t_test)\n",
      " |                                   Test for Constraints\n",
      " |      ==============================================================================\n",
      " |                       coef    std err          t      P>|t|      [0.025      0.975]\n",
      " |      ------------------------------------------------------------------------------\n",
      " |      c0            15.0977     84.937      0.178      0.863    -177.042     207.238\n",
      " |      c1            -2.0202      0.488     -8.231      0.000      -3.125      -0.915\n",
      " |      c2             1.0001      0.249      0.000      1.000       0.437       1.563\n",
      " |      ==============================================================================\n",
      " |  \n",
      " |  t_test_pairwise(self, term_name, method='hs', alpha=0.05, factor_labels=None)\n",
      " |      Perform pairwise t_test with multiple testing corrected p-values.\n",
      " |      \n",
      " |      This uses the formula design_info encoding contrast matrix and should\n",
      " |      work for all encodings of a main effect.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      term_name : str\n",
      " |          The name of the term for which pairwise comparisons are computed.\n",
      " |          Term names for categorical effects are created by patsy and\n",
      " |          correspond to the main part of the exog names.\n",
      " |      method : {str, list[str]}\n",
      " |          The multiple testing p-value correction to apply. The default is\n",
      " |          'hs'. See stats.multipletesting.\n",
      " |      alpha : float\n",
      " |          The significance level for multiple testing reject decision.\n",
      " |      factor_labels : {list[str], None}\n",
      " |          Labels for the factor levels used for pairwise labels. If not\n",
      " |          provided, then the labels from the formula design_info are used.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      MultiCompResult\n",
      " |          The results are stored as attributes, the main attributes are the\n",
      " |          following two. Other attributes are added for debugging purposes\n",
      " |          or as background information.\n",
      " |      \n",
      " |          - result_frame : pandas DataFrame with t_test results and multiple\n",
      " |            testing corrected p-values.\n",
      " |          - contrasts : matrix of constraints of the null hypothesis in the\n",
      " |            t_test.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      Status: experimental. Currently only checked for treatment coding with\n",
      " |      and without specified reference level.\n",
      " |      \n",
      " |      Currently there are no multiple testing corrected confidence intervals\n",
      " |      available.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      >>> res = ols(\"np.log(Days+1) ~ C(Weight) + C(Duration)\", data).fit()\n",
      " |      >>> pw = res.t_test_pairwise(\"C(Weight)\")\n",
      " |      >>> pw.result_frame\n",
      " |               coef   std err         t         P>|t|  Conf. Int. Low\n",
      " |      2-1  0.632315  0.230003  2.749157  8.028083e-03        0.171563\n",
      " |      3-1  1.302555  0.230003  5.663201  5.331513e-07        0.841803\n",
      " |      3-2  0.670240  0.230003  2.914044  5.119126e-03        0.209488\n",
      " |           Conf. Int. Upp.  pvalue-hs reject-hs\n",
      " |      2-1         1.093067   0.010212      True\n",
      " |      3-1         1.763307   0.000002      True\n",
      " |      3-2         1.130992   0.010212      True\n",
      " |  \n",
      " |  wald_test(self, r_matrix, cov_p=None, scale=1.0, invcov=None, use_f=None, df_constraints=None)\n",
      " |      Compute a Wald-test for a joint linear hypothesis.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      r_matrix : {array_like, str, tuple}\n",
      " |          One of:\n",
      " |      \n",
      " |          - array : An r x k array where r is the number of restrictions to\n",
      " |            test and k is the number of regressors. It is assumed that the\n",
      " |            linear combination is equal to zero.\n",
      " |          - str : The full hypotheses to test can be given as a string.\n",
      " |            See the examples.\n",
      " |          - tuple : A tuple of arrays in the form (R, q), ``q`` can be\n",
      " |            either a scalar or a length p row vector.\n",
      " |      \n",
      " |      cov_p : array_like, optional\n",
      " |          An alternative estimate for the parameter covariance matrix.\n",
      " |          If None is given, self.normalized_cov_params is used.\n",
      " |      scale : float, optional\n",
      " |          Default is 1.0 for no scaling.\n",
      " |      \n",
      " |          .. deprecated:: 0.10.0\n",
      " |      \n",
      " |      invcov : array_like, optional\n",
      " |          A q x q array to specify an inverse covariance matrix based on a\n",
      " |          restrictions matrix.\n",
      " |      use_f : bool\n",
      " |          If True, then the F-distribution is used. If False, then the\n",
      " |          asymptotic distribution, chisquare is used. If use_f is None, then\n",
      " |          the F distribution is used if the model specifies that use_t is True.\n",
      " |          The test statistic is proportionally adjusted for the distribution\n",
      " |          by the number of constraints in the hypothesis.\n",
      " |      df_constraints : int, optional\n",
      " |          The number of constraints. If not provided the number of\n",
      " |          constraints is determined from r_matrix.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      ContrastResults\n",
      " |          The results for the test are attributes of this results instance.\n",
      " |      \n",
      " |      See Also\n",
      " |      --------\n",
      " |      f_test : Perform an F tests on model parameters.\n",
      " |      t_test : Perform a single hypothesis test.\n",
      " |      statsmodels.stats.contrast.ContrastResults : Test results.\n",
      " |      patsy.DesignInfo.linear_constraint : Specify a linear constraint.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The matrix `r_matrix` is assumed to be non-singular. More precisely,\n",
      " |      \n",
      " |      r_matrix (pX pX.T) r_matrix.T\n",
      " |      \n",
      " |      is assumed invertible. Here, pX is the generalized inverse of the\n",
      " |      design matrix of the model. There can be problems in non-OLS models\n",
      " |      where the rank of the covariance of the noise is not full.\n",
      " |  \n",
      " |  wald_test_terms(self, skip_single=False, extra_constraints=None, combine_terms=None)\n",
      " |      Compute a sequence of Wald tests for terms over multiple columns.\n",
      " |      \n",
      " |      This computes joined Wald tests for the hypothesis that all\n",
      " |      coefficients corresponding to a `term` are zero.\n",
      " |      `Terms` are defined by the underlying formula or by string matching.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      skip_single : bool\n",
      " |          If true, then terms that consist only of a single column and,\n",
      " |          therefore, refers only to a single parameter is skipped.\n",
      " |          If false, then all terms are included.\n",
      " |      extra_constraints : ndarray\n",
      " |          Additional constraints to test. Note that this input has not been\n",
      " |          tested.\n",
      " |      combine_terms : {list[str], None}\n",
      " |          Each string in this list is matched to the name of the terms or\n",
      " |          the name of the exogenous variables. All columns whose name\n",
      " |          includes that string are combined in one joint test.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      WaldTestResults\n",
      " |          The result instance contains `table` which is a pandas DataFrame\n",
      " |          with the test results: test statistic, degrees of freedom and\n",
      " |          pvalues.\n",
      " |      \n",
      " |      Examples\n",
      " |      --------\n",
      " |      >>> res_ols = ols(\"np.log(Days+1) ~ C(Duration, Sum)*C(Weight, Sum)\", data).fit()\n",
      " |      >>> res_ols.wald_test_terms()\n",
      " |      <class 'statsmodels.stats.contrast.WaldTestResults'>\n",
      " |                                                F                P>F  df constraint  df denom\n",
      " |      Intercept                        279.754525  2.37985521351e-22              1        51\n",
      " |      C(Duration, Sum)                   5.367071    0.0245738436636              1        51\n",
      " |      C(Weight, Sum)                    12.432445  3.99943118767e-05              2        51\n",
      " |      C(Duration, Sum):C(Weight, Sum)    0.176002      0.83912310946              2        51\n",
      " |      \n",
      " |      >>> res_poi = Poisson.from_formula(\"Days ~ C(Weight) * C(Duration)\",                                            data).fit(cov_type='HC0')\n",
      " |      >>> wt = res_poi.wald_test_terms(skip_single=False,                                          combine_terms=['Duration', 'Weight'])\n",
      " |      >>> print(wt)\n",
      " |                                  chi2             P>chi2  df constraint\n",
      " |      Intercept              15.695625  7.43960374424e-05              1\n",
      " |      C(Weight)              16.132616  0.000313940174705              2\n",
      " |      C(Duration)             1.009147     0.315107378931              1\n",
      " |      C(Weight):C(Duration)   0.216694     0.897315972824              2\n",
      " |      Duration               11.187849     0.010752286833              3\n",
      " |      Weight                 30.263368  4.32586407145e-06              4\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Class methods inherited from statsmodels.base.model.LikelihoodModelResults:\n",
      " |  \n",
      " |  load(fname) from builtins.type\n",
      " |      Load a pickled results instance\n",
      " |      \n",
      " |      .. warning::\n",
      " |      \n",
      " |         Loading pickled models is not secure against erroneous or\n",
      " |         maliciously constructed data. Never unpickle data received from\n",
      " |         an untrusted or unauthenticated source.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      fname : {str, handle}\n",
      " |          A string filename or a file handle.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      Results\n",
      " |          The unpickled results instance.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from statsmodels.base.model.LikelihoodModelResults:\n",
      " |  \n",
      " |  llf\n",
      " |      Log-likelihood of model\n",
      " |  \n",
      " |  pvalues\n",
      " |      The two-tailed p values for the t-stats of the params.\n",
      " |  \n",
      " |  tvalues\n",
      " |      Return the t-statistic for a given parameter estimate.\n",
      " |  \n",
      " |  use_t\n",
      " |      Flag indicating to use the Student's distribution in inference.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Methods inherited from statsmodels.base.model.Results:\n",
      " |  \n",
      " |  initialize(self, model, params, **kwargs)\n",
      " |      Initialize (possibly re-initialize) a Results instance.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      model : Model\n",
      " |          The model instance.\n",
      " |      params : ndarray\n",
      " |          The model parameters.\n",
      " |      **kwargs\n",
      " |          Any additional keyword arguments required to initialize the model.\n",
      " |  \n",
      " |  predict(self, exog=None, transform=True, *args, **kwargs)\n",
      " |      Call self.model.predict with self.params as the first argument.\n",
      " |      \n",
      " |      Parameters\n",
      " |      ----------\n",
      " |      exog : array_like, optional\n",
      " |          The values for which you want to predict. see Notes below.\n",
      " |      transform : bool, optional\n",
      " |          If the model was fit via a formula, do you want to pass\n",
      " |          exog through the formula. Default is True. E.g., if you fit\n",
      " |          a model y ~ log(x1) + log(x2), and transform is True, then\n",
      " |          you can pass a data structure that contains x1 and x2 in\n",
      " |          their original form. Otherwise, you'd need to log the data\n",
      " |          first.\n",
      " |      *args\n",
      " |          Additional arguments to pass to the model, see the\n",
      " |          predict method of the model for the details.\n",
      " |      **kwargs\n",
      " |          Additional keywords arguments to pass to the model, see the\n",
      " |          predict method of the model for the details.\n",
      " |      \n",
      " |      Returns\n",
      " |      -------\n",
      " |      array_like\n",
      " |          See self.model.predict.\n",
      " |      \n",
      " |      Notes\n",
      " |      -----\n",
      " |      The types of exog that are supported depends on whether a formula\n",
      " |      was used in the specification of the model.\n",
      " |      \n",
      " |      If a formula was used, then exog is processed in the same way as\n",
      " |      the original data. This transformation needs to have key access to the\n",
      " |      same variable names, and can be a pandas DataFrame or a dict like\n",
      " |      object that contains numpy arrays.\n",
      " |      \n",
      " |      If no formula was used, then the provided exog needs to have the\n",
      " |      same number of columns as the original exog in the model. No\n",
      " |      transformation of the data is performed except converting it to\n",
      " |      a numpy array.\n",
      " |      \n",
      " |      Row indices as in pandas data frames are supported, and added to the\n",
      " |      returned prediction.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors inherited from statsmodels.base.model.Results:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(sm.OLS.fit)\n",
    "help(sm.regression.linear_model.RegressionResults)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}