chapter-02/2.7-structured-regression-models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.7 Structured Regression Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have seen that although kNN and other local methods focus directly on estimating the function at a point, they face problems in high dimensions. They may also be inappropriate even in low dimensions in cases where more structured approaches can make more efficient use of the data. This section introduces classes of such approaches."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.7.1 Difficulty of the Problem\n",
    "\n",
    "(2.37) RSS criterian for an arbitrary function $f$: \n",
    "\n",
    "$$RSS(f) = \\sum_{i=1}^N(y_i-f(x_i))^2$$\n",
    "\n",
    "Minimizing (2.37) leads to infinitely many solutions: any function $\\hat{f}$ passing through the training points is a solution. It might be a poor predictor at test points different from the training points. If there are multiple observations at each point $x_i$, i.e $y_{il}, l = 1 ... N$, the risk is limited.\n",
    "\n",
    "In order to obtain useful results for finite N, we must restrict the eligible solutions to a smaller set of functions."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}