basis/2、autograd机制.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# autograd机制 （0.4.0之前版本的）\n",
    "\n",
    "在机器学习中，我们通常使用梯度下降（gradient descent）来更新模型参数从而求解。损失函数关于模型参数的梯度指向一个可以降低损失函数值的方向，我们不断地沿着梯度的方向更新模型从而最小化损失函数。虽然梯度计算比较直观，但对于复杂的模型，例如多达数十层的神经网络，手动计算梯度非常困难。\n",
    "\n",
    "PyTorch中提供给了我们自动化求导的包 —— autograd\n",
    "\n",
    "导入之后，**我们用一个 Variable $X$ 对象来包装Tensor变量。然后 $X$.data 就是该Tensor变量，$X$.grad 就是梯度咯**\n",
    "\n",
    "PyTorch的 Variable 跟 Tensor 有着几乎一样的API，在 Tensor 上进行的操作也基本都能在 Variable 上进行。**区别在于：使用 Variable 对象，你会定义一张动态图，它可以帮助你进行自动求导**\n",
    "\n",
    "![Variable对象](../images/Variable.png)\n",
    "\n",
    "!!!! **利好消息，pytorch之后的版本会逐步废弃掉 Variable 变量，全程使用tensor就可以了。** !!!!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 把必要的包给导入\n",
    "import torch\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们想计算 $f = 3 \\times x^3 + 4 \\times x^2 + 6$ 的导数，该如何做呢？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable containing:\n",
      " 13\n",
      "[torch.FloatTensor of size 1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def fn(x):\n",
    "    y = 3 * x.pow(3) + 4 * x.pow(2) + 6\n",
    "    return y\n",
    "\n",
    "x1 = Variable(torch.Tensor([1]), requires_grad=True)\n",
    "\n",
    "y1 = fn(x1)\n",
    "\n",
    "print(y1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y1.backward() # 自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       " 17\n",
       "[torch.FloatTensor of size 1]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1.grad # 查看梯度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过调用 backward() 函数，我们自动求出了在 x = 1 的时候的导数\n",
    "\n",
    "需要注意的一点是：如果我们输入的 Tensor 不是一个标量，而是矢量（多个值）。\n",
    "\n",
    "那么，我们在调用backward()之前，需要让**结果变成标量** 才能求出导数。\n",
    "\n",
    "也就是说如果不将 Y 的值变成标量，就会报错。（可以尝试把mean()给取消，看看是不是报错了）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x2 = Variable(torch.Tensor([[1, 2], [3, 4]]), requires_grad=True)\n",
    "\n",
    "y2 = fn(x2).mean() # 将结果变成标量，这样就不会报错了\n",
    "\n",
    "y2.backward() # 自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Variable containing:\n",
       "  4.2500  13.0000\n",
       " 26.2500  44.0000\n",
       "[torch.FloatTensor of size 2x2]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x2.grad # 查看梯度"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}