chapter-03/3.2.4-multiple-outputs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.2.4 Multiple Outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we have multiple outputs $Y_1, Y_2, ..., Y_k$ that we wish to predict from our inputs $X_0, X_1, ..., X_p$. We assume a linear model for each output (3.34, 3.35):\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "Y_k &= \\beta_{0k} + \\sum_{j=1}^{p} X_j\\beta_{jk} + \\varepsilon_k\\\\\n",
    "&= f_k(X) + \\varepsilon_k\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "We can write the model in matrix notation:\n",
    "\n",
    "$$\n",
    "\\mathbf{Y} = \\mathbf{XB} + \\mathbf{E}\n",
    "$$\n",
    "\n",
    "Here **Y** is the $N\\times K $ response matrix, **X** is the $N\\times(p+1)$ input matrix, **B** is the $(p+1)\\times K$ matrix and **E** is the $N\\times K$ matrix of errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A straightforward generalization of the univariate loss functio is (3.37, 3.38):\n",
    "$$\n",
    "\\begin{align}\n",
    "RSS(\\mathbf{B}) &= \\sum_{k=1}^K{\\sum_{i=1}^N (y_{ik} - f_k(x_i))^2}\\\\\n",
    "&= tr\\left[(\\mathbf{Y}-\\mathbf{XB})^T(\\mathbf{Y}-\\mathbf{XB})\\right]\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The least squares estimates is (3.39):\n",
    "$$\\hat{\\mathbf{B}} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the errors $\\varepsilon=(\\varepsilon_1,...,\\varepsilon_K)$ are correlated, then (3.40):\n",
    "\n",
    "$$\n",
    "RSS(B; \\Sigma)=\\sum_{i=1}^N(y_i - f(x_i))^T\\Sigma^{-1}(y_i - f(x_i))\n",
    "$$\n",
    "\n",
    "arises from multivariate Gaussian theory."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}