基本的/Tensor操作学习.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor 学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/5d9273db0477677da5f5d91d13e2c96c.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=torch.tensor([1,2])\n",
    "y=torch.tensor([3,4])\n",
    "z=x.add(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([1, 2]), tensor([3, 4]), tensor([4, 6]))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x,y,z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([4, 6])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.add_(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到的是x在进行add操作后是没有改变的,如果要使值改变需要使用x.add_()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1., 2., 3., 4., 5., 6.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据list数据生成tensor\n",
    "torch.Tensor([1,2,3,4,5,6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[4.2934e-08, 1.0499e-08, 1.6985e+22],\n",
       "        [3.3207e-09, 3.3709e-06, 2.6941e+23]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据指定形状生成tensor\n",
    "torch.Tensor(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 2., 3.],\n",
       "        [4., 5., 6.]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据给定的tensor的形状\n",
    "t=torch.Tensor([[1,2,3],[4,5,6]])\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看tensor的形状\n",
    "t.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#shape与size()等价方式\n",
    "t.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.3107e-08, 7.9873e+20, 4.1960e-08],\n",
       "        [2.1863e+23, 4.1245e-11, 7.1450e+31]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据已有形状创建tensor\n",
    "torch.Tensor(t.size())\n",
    "# 生成的结果是随机的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t1的值tensor([1.4013e-45]),t1的数据类型torch.FloatTensor\n",
      "t2的值1,t2的数据类型torch.LongTensor\n"
     ]
    }
   ],
   "source": [
    "t1=torch.Tensor(1)\n",
    "t2=torch.tensor(1)\n",
    "print(\"t1的值{},t1的数据类型{}\".format(t1,t1.type()))\n",
    "print(\"t2的值{},t2的数据类型{}\".format(t2,t2.type()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensor是一个生成张量的函数，默认传入的是一个大小，自动生成一个size的tensor，tensor里的数据类型默认float\n",
    "而tensor是默认输入的值为创建tensor的值，自动判断其类型，生成一个tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 生成特定tensor "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 0.],\n",
       "        [0., 1.]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成一个单位矩阵\n",
    "torch.eye(2,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#自动生成全是0的矩阵\n",
    "torch.zeros(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 1.,  4.,  7., 10.])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据规则生成数据1~10之间的4个数\n",
    "torch.linspace(1,10,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.8662, 0.3090, 0.0922],\n",
       "        [0.3363, 0.9156, 0.4157]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成满足均匀分布随机数\n",
    "torch.rand(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-1.5119,  0.5444, -0.7975],\n",
       "        [-1.0350,  0.2358,  0.4778]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成满足标准分布随机数\n",
    "torch.randn(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#返回所给数据形状相同，值全为0的张量\n",
    "torch.zeros_like(torch.rand(2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 改变形状\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/b78a5eb320ca85865498ef331663acd6.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成一个形状为2x3的矩阵\n",
    "x = torch.randn(2, 3)\n",
    "#查看矩阵的形状\n",
    "x.size()   #结果为torch.Size([2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看x的维度\n",
    "x.dim()    #结果为2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0580,  0.1934],\n",
       "        [-0.2943, -0.2764],\n",
       "        [ 1.9054,  1.1999]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#把x变为3x2的矩阵\n",
    "x.view(3,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.0580,  0.1934, -0.2943, -0.2764,  1.9054,  1.1999])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#把x展平为1维向量\n",
    "y=x.view(-1)  \n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([6])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape# 1x6的tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([6])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "size()和shape是一样的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 6])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#添加一个维度\n",
    "z=torch.unsqueeze(y,0)\n",
    "#查看z的形状\n",
    "z.size()   #结果为torch.Size([1, 6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0580,  0.1934, -0.2943, -0.2764,  1.9054,  1.1999]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到的是tensor中多了一层[]即增加了一个维度，可以试一下在增加一个维度看看，会发现有多了一层[]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0580],\n",
       "        [ 0.1934],\n",
       "        [-0.2943],\n",
       "        [-0.2764],\n",
       "        [ 1.9054],\n",
       "        [ 1.1999]])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = torch.unsqueeze(y,1)\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z.dim()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.dim()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果后一个参数写成1的话，发现y首先被拆分成了一个个tensor后又组成了一个维度加一的tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([6, 1])"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看z的形状\n",
    "z.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#计算Z的元素个数\n",
    "z.numel()   #结果为6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 索引操作\n",
    "Tensor的索引操作与Numpy类似，一般情况下索引结果与源数据共享内存。从tensor获取元素除了可以通过索引，也可借助一些函数\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/ca5b6ebfbff3968e030d3c7b92c445f0.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3607, -0.2859, -0.3938],\n",
       "        [ 0.2429, -1.3833, -2.3134]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#设置一个随机种子\n",
    "torch.manual_seed(100) \n",
    "#生成一个形状为2x3的矩阵\n",
    "x = torch.randn(2, 3)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.3607, -0.2859, -0.3938])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#根据索引获取第1行，所有数据\n",
    "x[0,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.3938, -2.3134])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#获取最后一列数据\n",
    "x[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.3607, 0.2429])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成是否大于0的Byter张量\n",
    "mask=x>0\n",
    "#获取大于0的值\n",
    "torch.masked_select(x,mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 0],\n",
       "        [1, 0]])"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#获取非0值的下标,即行，列索引\n",
    "torch.nonzero(mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.dim()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "gather(input,dim,index),input是一个tensor，dim是待操作数的维度，index就是对应操作维度上的下标\n",
    "\n",
    "获取指定索引对应的值,输出根据以下规则得到\n",
    "\n",
    " out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0\n",
    " \n",
    " out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1\n",
    " \n",
    " out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2\n",
    "\n",
    "以下面代码为例;\n",
    "output的格式和index是一样的:\n",
    "\n",
    "output[0][0]=input[index[0][0]][0]=iput[0][0]=0.3607\n",
    "\n",
    "output[0][1]=input[index[0][1]][1]=input[1][1]=-1.3833\n",
    "\n",
    "output[0][2]=input[index[0][2]][2]=input[1][2]=-2.3134"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 1, 1]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#获取指定索引对应的值,输出根据以下规则得到\n",
    "# out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0\n",
    "# out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1\n",
    "# out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2\n",
    "index=torch.LongTensor([[0,1,1]])\n",
    "index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3607, -1.3833, -2.3134]])"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.gather(x,0,index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3607, -0.2859, -0.2859],\n",
       "        [-1.3833, -1.3833, -1.3833]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index=torch.LongTensor([[0,1,1],[1,1,1]])\n",
    "a=torch.gather(x,1,index)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "out[0][0]=input[0][index[0][0]]=input[0][0]=0.3607\n",
    "\n",
    "out[0][1]=input[0][index[0][1]]=input[0][1]=-0.2859\n",
    "\n",
    "那么gather到底有啥好处呢？我的理解是这样的对于input的tensor，我们可以确定需要操作的维数，即只取该维度上的数据，而index帮助确定该维度的数据，不用使用循环，减少了麻烦。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#把a的值返回到一个2x3的0矩阵中\n",
    "z=torch.zeros(2,3)\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3607, -0.2859,  0.0000],\n",
       "        [ 0.0000, -1.3833,  0.0000]])"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z.scatter_(1,index,a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scatter的操作方法\n",
    "\n",
    "self[ index[i][j][k] ][ j ][ k ] = src[i][j][k]  # if dim == 0\n",
    "\n",
    "self[ i ][ index[i][j][k] ][ k ] = src[i][j][k]  # if dim == 1\n",
    "\n",
    "self[ i ][ j ][ index[i][j][k] ] = src[i][j][k]  # if dim == 2\n",
    "\n",
    "以上题为例：\n",
    "\n",
    "z[0][index[0][0]] = z[0][0] = a[0][0]=0.3607\n",
    "\n",
    "z[0][index[0][1]] = z[0][1] = a[0][1]=-0.2859\n",
    "\n",
    "............."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 广播机制"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2],\n",
       "        [10, 11, 12],\n",
       "        [20, 21, 22],\n",
       "        [30, 31, 32]], dtype=torch.int32)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.arange(0, 40,10).reshape(4, 1)\n",
    "B = np.arange(0, 3)\n",
    "#把ndarray转换为Tensor\n",
    "A1=torch.from_numpy(A)  #形状为4x1\n",
    "B1=torch.from_numpy(B)  #形状为3\n",
    "#Tensor自动实现广播\n",
    "C=A1+B1\n",
    "C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2],\n",
       "        [10, 11, 12],\n",
       "        [20, 21, 22],\n",
       "        [30, 31, 32]], dtype=torch.int32)"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#我们可以根据广播机制，手工进行配置\n",
    "#根据规则1，B1需要向A1看齐，把B变为（1,3）\n",
    "B2=B1.unsqueeze(0)  #B2的形状为1x3\n",
    "#使用expand函数重复数组，分别的4x3的矩阵\n",
    "A2=A1.expand(4,3)\n",
    "B3=B2.expand(4,3)\n",
    "#然后进行相加,C1与C结果一致\n",
    "C1=A2+B3\n",
    "C1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 逐元素操作\n",
    "与Numpy一样，tensor也有逐元素操作（element-wise），操作内容相似，但使用函数可能不尽相同。大部分数学运算都属于逐元操作，逐元素操作输入与输出的形状相同\n",
    "![jupyte](http://www.feiguyunai.com/wp-content/uploads/2019/09/3fbded1e5a9c9c5330566b41cf6f20fd.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.3861, -0.1857,  0.6969],\n",
       "        [-0.3915, -0.1830,  0.7061],\n",
       "        [-0.1649, -0.2989,  0.3154]])"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.randn(1, 3)\n",
    "t1 = torch.randn(3, 1)\n",
    "t2 = torch.randn(1, 3)\n",
    "#t+0.1*(t1/t2)\n",
    "torch.addcdiv(t, 0.1, t1, t2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.4357, 0.4376, 0.6170]])"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#计算sigmoid\n",
    "torch.sigmoid(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0000, 0.0000, 0.4770]])"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将t限制在[0,1]之间\n",
    "torch.clamp(t,0,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.7414, 1.7490, 2.4770]])"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#t+2进行就地运算\n",
    "t.add_(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = torch.Tensor([[1,2,3,4],[5,6,7,8]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.0000, 1.4142, 1.7321, 2.0000],\n",
       "        [2.2361, 2.4495, 2.6458, 2.8284]])"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.sqrt(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 归并操作\n",
    "归并操作顾名思义，就是对输入进行归并或合计等操作，这类操作的输入输出形状一般不相同，而且往往是输入大于输出形状。归并操作可以对整个tensor，也可以沿着某个维度进行归并\n",
    "\n",
    "归并操作一般涉及一个dim参数，指定沿哪个维进行归并。另一个参数是keepdim，说明输出结果中是否保留维度1，缺省情况是False，即不保留。\n",
    "以下为归并操作的部分代码\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/625f6b4e0a1446f77392fc93a847f5a1.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 6., 10., 14.])"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#生成一个含6个数的向量\n",
    "a=torch.linspace(0,10,6)\n",
    "#使用view方法，把a变为2x3矩阵\n",
    "a=a.view((2,3))\n",
    "#沿y轴方向累加，即dim=0\n",
    "b=a.sum(dim=0)   #b的形状为[3]\n",
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 6., 10., 14.]])"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#沿y轴方向累加，即dim=0,并保留含1的维度\n",
    "b=a.sum(dim=0,keepdim=True) #b的形状为[1,3]\n",
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 比较操作\n",
    "比较操作一般进行逐元素比较，有些是按指定方向比较。\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/c8d86d5a90943e7be3ecc22885bb28f0.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=torch.linspace(0,10,6).view(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.,  2.,  4.],\n",
       "        [ 6.,  8., 10.]])"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor(10.),\n",
       " torch.return_types.max(\n",
       " values=tensor([ 6.,  8., 10.]),\n",
       " indices=tensor([1, 1, 1])),\n",
       " torch.return_types.topk(\n",
       " values=tensor([[ 6.,  8., 10.]]),\n",
       " indices=tensor([[1, 1, 1]])))"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#求所有元素的最大值\n",
    "max = torch.max(x)   #结果为10\n",
    "#求y轴方向的最大值\n",
    "max1 = torch.max(x,dim=0)  #结果为[6,8,10]\n",
    "#求最大的2个元素\n",
    "max2 = torch.topk(x,1,dim=0)  #结果为[6,8,10],对应索引为tensor([[1, 1, 1]\n",
    "max,max1,max2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵操作\n",
    "机器学习和深度学习中存在大量的矩阵运算，用的比较多的有两种，一种是逐元素乘法，另外一种是点积乘法。\n",
    "\n",
    "【说明】\n",
    "\n",
    "①torch的dot与Numpy的dot有点不同，torch中dot对两个为1D张量进行点积运算，Numpy中的dot无此限制。\n",
    "\n",
    "②mm是对2D的矩阵进行点积，bmm对含batch的3D进行点积运算。\n",
    "\n",
    "③转置运算会导致存储空间不连续，需要调用contiguous方法转为连续。\n",
    "\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/883ab00d77e328c75eae26fc46abf6ad.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "普通点乘"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(18)"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=torch.tensor([2, 3])\n",
    "b=torch.tensor([3, 4])\n",
    " \n",
    "torch.dot(a,b)  #运行结果为18"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对两个二维的矩阵点成。类似于矩阵点乘运算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[5, 8, 2],\n",
      "        [5, 5, 2]]) tensor([[5, 3, 5, 2],\n",
      "        [2, 3, 4, 0],\n",
      "        [1, 4, 3, 0]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[43, 47, 63, 10],\n",
       "        [37, 38, 51, 10]])"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x=torch.randint(10,(2,3))\n",
    "y=torch.randint(6,(3,4))\n",
    "print(x,y)\n",
    "torch.mm(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对两个三维的矩阵点乘"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[1, 8, 6],\n",
      "         [5, 4, 1]],\n",
      "\n",
      "        [[1, 3, 9],\n",
      "         [4, 7, 6]]]) tensor([[[2, 1, 1, 0],\n",
      "         [4, 4, 2, 1],\n",
      "         [1, 3, 1, 2]],\n",
      "\n",
      "        [[0, 3, 2, 1],\n",
      "         [1, 1, 2, 3],\n",
      "         [0, 5, 2, 3]]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[[40, 51, 23, 20],\n",
       "         [27, 24, 14,  6]],\n",
       "\n",
       "        [[ 3, 51, 26, 37],\n",
       "         [ 7, 49, 34, 43]]])"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x=torch.randint(10,(2,2,3))\n",
    "y=torch.randint(6,(2,3,4))\n",
    "print(x,y)\n",
    "torch.bmm(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch与Numpy比较\n",
    "Pytorch与Numpy有很多类似的地方，并且有很多相同的操作函数名称，或虽然函数名称不同但含义相同；当然也有一些虽然函数名称相同，但含义不尽相同。对此，有时很容易混淆，下面我们把一些主要的区别进行汇总，具体可参考表\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/995d56a6c02fd906dcbd71e3e09c703b.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor与Autograd\n",
    "在神经网络中，一个重要内容就是进行参数学习，而参数学习离不开求导，Pytorch是如何进行求导的呢？\n",
    "现在大部分深度学习架构都有自动求导的功能，Pytorch也不列外，torch.autograd包就是用来自动求导的。autograd包为张量上所有的操作提供了自动求导功能，而torch.Tensor和torch.Function为autograd上的两个核心类，他们相互连接并生成一个有向非循环图。接下来我们先简单介绍tensor如何实现自动求导，然后介绍计算图，最后用代码实现这些功能。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "autograd包为对tensor进行自动求导，为实现对tensor自动求导，需考虑如下事项：\n",
    "\n",
    "（1）创建叶子节点(leaf node)的tensor，使用requires_grad参数指定是否记录对其的操作，以便之后利用backward()方法进行梯度求解。requires_grad参数缺省值为False，如果要对其求导需设置为True，与之有依赖关系的节点自动变为True。\n",
    "\n",
    "（2）可利用requires_grad_()方法修改tensor的requires_grad属性。可以调用.detach()或with torch.no_grad():将不再计算张量的梯度，跟踪张量的历史记录。这点在评估模型、测试模型阶段常常使用。\n",
    "\n",
    "（3）通过运算创建的tensor（即非叶子节点），会自动被赋于grad_fn属性。该属性表示梯度函数。叶子节点的grad_fn为None。\n",
    "\n",
    "（4）最后得到的tensor执行backward()函数，此时自动计算各变在量的梯度，并将累加结果保存grad属性中。计算完成后，非叶子节点的梯度自动释放。\n",
    "\n",
    "（5）backward()函数接受参数，该参数应和调用backward()函数的Tensor的维度相同，或者是可broadcast的维度。如果求导的tensor为标量（即一个数字），backward中参数可省略。\n",
    "\n",
    "（6）反向传播的中间缓存会被清空，如果需要进行多次反向传播，需要指定backward中的参数retain_graph=True。多次反向传播时，梯度是累加的。\n",
    "\n",
    "（7）非叶子节点的梯度backward调用后即被清空。\n",
    "\n",
    "（8）可以通过用torch.no_grad()包裹代码块来阻止autograd去跟踪那些标记为.requesgrad=True的张量的历史记录。这步在测试阶段经常使用。\n",
    "整个过程中，Pytorch采用计算图的形式进行组织，该计算图为动态图，它的计算图在每次前向传播时，将重新构建。其他深度学习架构，如TensorFlow、Keras一般为静态图。接下来我们介绍计算图，用图的形式来描述就更直观了，该计算图为有向无环图（DAG）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 计算图\n",
    "计算图是一种有向无环图像，用图形方式表示算子与变量之间的关系，直观高效。如图所示，圆形表示变量，矩阵表示算子。如表达式：z=wx+b，可写成两个表示式：y=wx，则z=y+b，其中x、w、b为变量，是用户创建的变量，不依赖于其他变量，故又称为叶子节点。为计算各叶子节点的梯度，需要把对应的张量参数requires_grad属性设置为True，这样就可自动跟踪其历史记录。y、z是计算得到的变量，非叶子节点，z为根节点。mul和add是算子（或操作或函数）。由这些变量及算子，就构成一个完整的计算过程（或前向传播过程）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/74d0cafe66686492ce6228e5fa0f81fc.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们的目标是更新各叶子节点的梯度，根据复合函数导数的链式法则，不难算出各叶子节点的梯度。\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/cd31ef2b0f71dea92de540acfe16cb5c.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pytorch调用backward()，将自动计算各节点的梯度，这是一个反向传播过程，这个过程可用图表示。在反向传播过程中，autograd沿着图，从当前根节点z反向溯源，利用导数链式法则，计算所有叶子节点的梯度，其梯度值将累加到grad属性中。对非叶子节点的计算操作（或function）记录在grad_fn属性中，叶子节点的grad_fn值为None。\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/b6fd0cb6acbc93e2c5e5cdace1bfcb4e.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 标量反向传播\n",
    "假设x、w、b都是标量，z=wx+b，对标量z调用backward()，我们无需对backward()传入参数。以下是实现自动求导的主要步骤：\n",
    "\n",
    "(1)定义叶子节点及算子节点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x,w,b的require_grad属性分别为：False,True,True\n"
     ]
    }
   ],
   "source": [
    "#定义输入张量x\n",
    "x=torch.Tensor([2])\n",
    "#初始化权重参数W,偏移量b、并设置require_grad属性为True，为自动求导\n",
    "w=torch.randn(1,requires_grad=True)\n",
    "b=torch.randn(1,requires_grad=True)\n",
    "#实现前向传播\n",
    "y=torch.mul(w,x)  #等价于w*x\n",
    "z=torch.add(y,b)  #等价于y+b\n",
    "#查看x,w，b页子节点的requite_grad属性\n",
    "print(\"x,w,b的require_grad属性分别为：{},{},{}\".format(x.requires_grad,w.requires_grad,b.requires_grad))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）查看叶子节点、非叶子节点的其他属性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y，z的requires_grad属性分别为：True,True\n",
      "x，w，b，y，z的是否为叶子节点：True,True,True,False,False\n",
      "x，w，b的grad_fn属性：None,None,None\n",
      "y，z的grad_fn属性：<MulBackward0 object at 0x00000168DE1536D8>,<AddBackward0 object at 0x00000168DE153748>\n"
     ]
    }
   ],
   "source": [
    "#查看非叶子节点的requres_grad属性,\n",
    "print(\"y，z的requires_grad属性分别为：{},{}\".format(y.requires_grad,z.requires_grad))\n",
    "#因与w，b有依赖关系，故y，z的requires_grad属性也是：True,True\n",
    "#查看各节点是否为叶子节点\n",
    "print(\"x，w，b，y，z的是否为叶子节点：{},{},{},{},{}\".format(x.is_leaf,w.is_leaf,b.is_leaf,y.is_leaf,z.is_leaf))\n",
    "#x，w，b，y，z的是否为叶子节点：True,True,True,False,False\n",
    "#查看叶子节点的grad_fn属性\n",
    "print(\"x，w，b的grad_fn属性：{},{},{}\".format(x.grad_fn,w.grad_fn,b.grad_fn))\n",
    "#因x，w，b为用户创建的，为通过其他张量计算得到，故x，w，b的grad_fn属性：None,None,None\n",
    "#查看非叶子节点的grad_fn属性\n",
    "print(\"y，z的grad_fn属性：{},{}\".format(y.grad_fn,z.grad_fn))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3）自动求导，实现梯度方向传播，即梯度的反向传播。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "参数w,b的梯度分别为:tensor([2.]),tensor([1.]),None\n",
      "非叶子节点y,z的梯度分别为:None,None\n"
     ]
    }
   ],
   "source": [
    "#基于z张量进行梯度反向传播,执行backward之后计算图会自动清空，\n",
    "z.backward()\n",
    "#如果需要多次使用backward，需要修改参数retain_graph为True，此时梯度是累加的\n",
    "#z.backward(retain_graph=True)\n",
    " \n",
    "#查看叶子节点的梯度，x是叶子节点但它无需求导，故其梯度为None\n",
    "print(\"参数w,b的梯度分别为:{},{},{}\".format(w.grad,b.grad,x.grad))\n",
    "#参数w,b的梯度分别为:tensor([2.]),tensor([1.]),None\n",
    " \n",
    "#非叶子节点的梯度，执行backward之后，会自动清空\n",
    "print(\"非叶子节点y,z的梯度分别为:{},{}\".format(y.grad,z.grad))\n",
    "#非叶子节点y,z的梯度分别为:None,None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 非标量反向传播\n",
    "上面调用backward()无需传入参数。目标张量一般是标量，如我们经常使用的损失值Loss，一般都是一个标量。但也有非标量的情况，后面我们介绍的Deep Dream的目标值就是一个含多个元素的张量。如何对非标量进行反向传播呢？Pytorch有个简单的规定，不让张量(tensor)对张量求导，只允许标量对张量求导，因此，如果目标张量对一个非标量调用backward()，需要传入一个gradient参数,该参数也是张量，而且需要与调用backward()的张量形状相同。为什么要传入一个张量gradient？\n",
    "传入这个参数就是为了把张量对张量求导转换为标量对张量求导。这有点拗口，我们举一个例子来说，假设目标值为loss=(y_1,y_2,…,y_m)传入的参数为v=(v_1,v_2,…,v_m),那么就可把对loss的求导，转换为对loss*v^T标量的求导。即把原来∂loss/∂x得到雅可比矩阵(Jacobian)乘以张量v^T，便可得到我们需要的梯度矩阵。\n",
    "backward函数的格式为：\n",
    "\n",
    "backward(gradient=None, retain_graph=None, create_graph=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（1）定义叶子叶子节点及计算节点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#定义叶子节点张量x，形状为1x2\n",
    "x= torch.tensor([[2, 3]], dtype=torch.float, requires_grad=True)\n",
    "#初始化Jacobian矩阵\n",
    "J= torch.zeros(2 ,2)\n",
    "#初始化目标张量，形状为1x2\n",
    "y = torch.zeros(1, 2)\n",
    "#定义y与x之间的映射关系：\n",
    "#y1=x1**2+3*x2，y2=x2**2+2*x1\n",
    "y[0, 0] = x[0, 0] ** 2 + 3 * x[0 ,1]\n",
    "y[0, 1] = x[0, 1] ** 2 + 2 * x[0, 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）手工计算y对x的梯度\n",
    "我们先手工计算一下y对x的梯度，为了验证Pytorch的backward的结果是否正确。\n",
    "y对x的梯度是一个雅可比矩阵，各项的值，我们可通过以下方法进行计算。\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/d8be64de0eb29951ef711cf7ecb0adcd.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（3）调用backward获取y对x的梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[6., 9.]])\n"
     ]
    }
   ],
   "source": [
    "y.backward(torch.Tensor([[1, 1]]))\n",
    "print(x.grad)\n",
    "#结果为tensor([[6., 9.]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个结果与我们手工运算的不符，显然这个结果是错误的，错在哪里呢？这个结果的计算过程是：\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/ec9974b67295400cb3f4de58ca9e28de.png)\n",
    "由此，错在v的取值错误，通过这种方式得的到并不是y对x的梯度。这里我们可以分成两步的计算。首先让v=(1,0)得到y_1对x的梯度，然后使v=(0,1)，得到y_2对x的梯度。这里因需要重复使用backward(),需要使参数retain_graph=True,具体代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[4., 3.],\n",
      "        [2., 6.]])\n"
     ]
    }
   ],
   "source": [
    "y.backward(torch.Tensor([[1, 0]]),retain_graph=True)\n",
    "J[0]=x.grad\n",
    "#梯度是累加的，故需要对x的梯度清零\n",
    "x.grad = torch.zeros_like(x.grad)\n",
    "#生成y2对x的梯度\n",
    "y.backward(torch.Tensor([[0, 1]]))\n",
    "J[1]=x.grad\n",
    "#显示jacobian矩阵的值\n",
    "print(J)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与手写运算的结果一致了，需要注意的是backward需要第一次就把retain_graph设为true，否则是不能运行的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用Numpy实现机器学习\n",
    "前面我们介绍了Numpy、Tensor的基础内容，对如何用Numpy、Tensor操作数组有了一定认识。为了加深大家对Pytorch是如何进行完成机器学习、深度学习，本章剩余章节将分别用Numpy、Tensor、autograd、nn及optimal实现同一个机器学习任务，比较他们之间的异同及各自优缺点，从而加深对Pytorch的理解。\n",
    "首先，我们用最原始的Numpy实现有关回归的一个机器学习任务，不用Pytorch中的包或类。这种方法代码可能多一点，但每一步都是透明的，有利于理解每步的工作原理。\n",
    "主要步骤包括：\n",
    "\n",
    "首先，是给出一个数组x，然后基于表达式：y=3x^2+2，加上一些噪音数据到达另一组数据y。\n",
    "\n",
    "然后，构建一个机器学习模型，学习表达式y=wx^2+b的两个参数w，b。利用数组x，y的数据为训练数据\n",
    "\n",
    "最后，采用梯度梯度下降法，通过多次迭代，学习到w、b的值。\n",
    "\n",
    "以下为具体步骤：\n",
    "\n",
    "（1）导入需要的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# -*- coding: utf-8 -*-\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）生成输入数据x及目标数据y\n",
    "设置随机数种子，生成同一个份数据，以便用多种方法进行比较。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(100) \n",
    "x = np.linspace(-1, 1, 100).reshape(100,1) \n",
    "y = 3*np.power(x, 2) +2+ 0.2*np.random.rand(x.size).reshape(100,1)       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（3）查看x，y数据分布情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHoxJREFUeJzt3X+QXeV93/H3R2KNljphwVYSWJAljyltGmIp3iG0mklsxQMYe0DFuKgzpODaw9hNG0NiUhF3XMJMBlFmAu2QCcV2GmInthxsU9nYpTiCSc0Y3JVZm2AFW/5VENQIwxIzktWV+PaPe644OnvOvefePffHOffzmtnR3XvPvfvo7N3vfc7zfJ/vo4jAzMyaZdWoG2BmZtVzcDczayAHdzOzBnJwNzNrIAd3M7MGcnA3M2sgB3czswZycDczayAHdzOzBjphVD/4ta99baxfv35UP97MrJb27NnzXESs7XbcyIL7+vXrmZ+fH9WPNzOrJUk/LHOch2XMzBrIwd3MrIEc3M3MGsjB3cysgRzczcwayMHdzKyBRpYKaWY2Ke55dD+33PcETy8e4vSZaa674Gy2bpod6M90cDczG6B7Ht3P9Z99jENLRwHYv3iI6z/7GMBAA3wtg/soPgXNzPpxy31PHAvsbYeWjnLLfU84uKeN6lPQzKwfTy8e6un+qtRuQrXTp6CZ2bg5fWa6p/urUrvgPqpPQTOzflx3wdlMT60+7r7pqdVcd8HZA/25tQvuo/oUNDPrx9ZNs9x06TnMzkwjYHZmmpsuPcfZMlnXXXD2cWPuMJxPQTOzfm3dNDv0OcFSwV3SD4CfAEeBIxExl3lcwH8GLgIOAldFxNerbWpL+wQ5W8bMrFgvPfe3RMRzBY+9DTgr+fpV4E+SfwdiFJ+CZmZ1UtWY+yXAn0fLw8CMpNMqem0zM+tR2eAewP+UtEfS1TmPzwJPpr5/KrnPzMxGoOywzOaIeFrSzwH3S/q7iPib1OPKeU5k70g+GK4GWLduXc+NNTOzckoF94h4Ovn3WUmfA84F0sH9KeDM1PdnAE/nvM6dwJ0Ac3Nzy4J/P1yKwMxsua7BXdI/AFZFxE+S2+cDN2YO2wX8W0mfojWR+mJEPFN5azNcisDMxtWoO55leu4/D3yule3ICcBfRsT/kPQ+gIi4A/girTTIfbRSId89mOYeb1QFeczM8rQD+v7FQ4hXxqZH0fHsGtwj4nvAG3PuvyN1O4DfqrZp3bkUgZmNi+xIQnbcedgdz9qVH0hzKQIzGxd5IwlZw+x41jq45xXkEa1LoM07dnPPo/tH0zAzmzhlAvcwO561Du7pgjxA7hiXA7yZDUO3wD3sGli1Du7QCvAPbd/C7Mx07hjXNTsX3Is3s4ErGkmA4VWCTKtdVcginS6JnCJpZoM2bkUNGxPcT5+ZZn+HAO8USTMbtHEqalj7YZm2vEuiLKdImtmkaEzPPX1JVNSDd4qkmU2KxvTc4ZXJ1dsu3ziSPQvNzMZFY3ruaeM2sWFmNmyNDO4wXhMbZmbD1tjgbmY2DKOu/ljEwd3MrE/jXHbcwd3MrEfp0r5Z47KmxsHdzKwH2d56nnFYU9OoVEgzs0ErU9p3HNbUOLibmfWgW698XNbUOLibmfWgU698FNUfizi4m5n1IK+O1fTUam67fCMPbd8yFoEdPKFqZtaTuqyAd3A3M+tRHVbAT0RwH9cVZGZmg9L44D7OK8jMrD7q1kksPaEqabWkRyV9IeexqyQdkLSQfL232mb2Ly8ntb2CzMysjHYncf/iIYJXOonjvDdzL9kyHwD2dnh8Z0RsTL4+usJ2VaYoJ3UcVpCZWT3UsZNYKrhLOgN4OzA2QbusopzUcVhBZmb1UMdOYtme+23A7wEvdzjmnZK+KeluSWfmHSDpaknzkuYPHDjQa1v7UpSTet0FZ3PPo/vZvGM3G7bfy+Ydu8f6EsvMRqeOncSuwV3SO4BnI2JPh8M+D6yPiF8GvgzclXdQRNwZEXMRMbd27dq+GtyrrZtmuenSc5idmUbAzPQUa6ZWcc3OBa7duVCrMTQzG41OncRxVabnvhm4WNIPgE8BWyR9In1ARPw4Ig4n334EeFOlrVyh9t6qt16+kcNHXuaFg0sAROa4cR9DM7PRyHYSx6nMQJGuqZARcT1wPYCkNwMfjIgr0sdIOi0inkm+vZjOE68jU6aa2ziPoZnZ6NRh4VJa33nukm4E5iNiF/Dbki4GjgDPA1dV07xqlQnc4zyGZmZWliKygxPDMTc3F/Pz80P9mZt37M7dOaVNtIZqZmuwQMHMBm8cFy5J2hMRc92Om6iqkHmTIkr92/6Y8+SqmdVx4VLaRAX3vEmRWy/fyOzMtCdXzew4dVy4lNb42jJZeZMi1+5cyD3Wk6tmk6uOC5fSJqrnXqSOCxTMbLDqHhcc3KnnAgUzG6y6x4WJG5bJU5edVcxseOoeFyYqFdLMrO6cCmlmNsE8LGNmljKOC5f64eBuZpZo0racHpYxM0vUfeFSmoO7mVmi7guX0jwsY2YTrz3OXpQ7WJeFS2kO7mY20bLj7Fl1WriU5uBuZhOt0yY+dS7/7eCeI50KdfL0FBIsHlyqdVqUmeUrGk8X8ND2LcNtTIUc3DOyl2iLh5aOPVbntCgzO14Tx9nTnC2T0W2f1bqmRZnZK9IbceSp6zh7moN7RpmUp/2Lh9i8Y3dtdmQxs+N1G2e/6dJzan917mGZjNNnpjvus9rmIRqz+mrqOHuae+4ZeTWci3iIxqye6r4RRxnuuWdkazi3s2VeOLiUe3wdV66ZTar2JOr+xUMIjptMbcI4e5qDe468fVY379idO1zTpE96sybLZsIFHAvwdc5nL1J6WEbSakmPSvpCzmMnStopaZ+kRyStr7KR46DuW26ZTbq8SdR2YH9o+5ZGBXbobcz9A8DegsfeA7wQEW8AbgVuXmnDxs3WTbPcdOk5zM5MI5ozo242KZpUFKyMUsMyks4A3g78IfA7OYdcAtyQ3L4buF2SYlR7+A1I3nCNmdVDUSZcU4dWy/bcbwN+D3i54PFZ4EmAiDgCvAi8JnuQpKslzUuaP3DgQB/NNTPrz6QNrXYN7pLeATwbEXs6HZZz37Jee0TcGRFzETG3du3aHpppZrYykza0WmZYZjNwsaSLgDXAz0r6RERckTrmKeBM4ClJJwAnA89X3lozsxWYpKHVrj33iLg+Is6IiPXANmB3JrAD7AKuTG5flhzTqPF2M7M66TvPXdKNwHxE7AI+Bnxc0j5aPfZtFbXPzMz60FNwj4gHgQeT2x9O3f9T4F1VNszMrF/ek8ErVPuWfvNM0hvGbNx5T4YWFw7rQ7oWdPDKG8YlgM1Gz3sytDi49yHvzTMpbxizcVdmxWlTV6WmObj3YdKWMZvVSZkVp01dlZrm4N6HSagFbVY39zy6/1j11rxVlW1NXpWa5uDeh0lbxmw27rJ7orbL+QLMTE9xyklTE7EqNc3ZMn3IbujhbBmz0epWzncSObj3Kb2MuZ0Wee3OBQd6sxHwPNhyHpZZIadFmo2e58GWc3BfIadFmo2e58GW87DMCvly0Gz0PA+2nIP7Ck3a7i5m42qSyvmW4WGZFfLloJmNIwf3Fcru7jIzPcWaqVVcu3OBzTt2H5tYbS+w2LD93uPuNzMbBI1qT425ubmYn58fyc8elGw1Omj14t/5plk+s2f/svsnZTGFmVVH0p6ImOt2nHvuFSrKnPnkI086o8bMhsoTqhUqypA5WnB15Iwas5XxvgrF3HOvUFGGzGrllzFyRo1Z/7yAsDMH9wrlZc6IVs89G96dUWNWXl5CghcQduZhmQqlF1K0y462B2TaVeraxYx8+WhWTjZRod1DL9ptycOdLe65V2zrplke2r6F2ZlpsiPt6Sp1Duxm5RT10D3c2Zl77gPSrSyBJ4LMyumUqJC+OgYPd6Z17blLWiPpa5K+IelxSX+Qc8xVkg5IWki+3juY5tZHpyp1nggyK69TTzy9KcckbcRRRplhmcPAloh4I7ARuFDSeTnH7YyIjcnXRyttZQ11KkvgiSCz8vL+ltI83Jmva3CPlpeSb6eSr9Esa62RbFmCdK/ClSTNykv/LRXx385ypcbcJa0G9gBvAP44Ih7JOeydkn4N+DZwbUQ8WV0z66moSp0rSZr1pv231N4AO8t/O8uVypaJiKMRsRE4AzhX0i9lDvk8sD4ifhn4MnBX3utIulrSvKT5AwcOrKTdteZKkmb98d9OeT2lQkbEIvAgcGHm/h9HxOHk248Abyp4/p0RMRcRc2vXru2juc3QacjGzIr5b6e8rsMyktYCSxGxKGkaeCtwc+aY0yLimeTbi4G9lbe0YbyxgFl//LdTTpkx99OAu5Jx91XApyPiC5JuBOYjYhfw25IuBo4AzwNXDarBZjYZvBZkZVzP3czGTtHeCB6CKV/P3StUx4x7K2bFJQduue8J/z2U5OA+RooKJAF+Q9tE8VqQlXPhsDHilatmLZ3Kd1g5Du5joF2rOm9xBri3YpPH+ewr52GZEcubOMpyb8UmTXpvBM8/9cfBfcTyhmLS3FuxSeV89pVxcB+xTkMu3rHJzPrl4D5iRUXE2iVMzcz64QnVEfPEkZkNgnvuI+aJI5t06YV7J09PIcHiwSX/LayQg/sY8MSRTapsttjioaVjj3kR38p4WMbMRqZbtpgX8fXPPfcxVlRnxvVnrCnKLNDzIr7+OLiPqaI6M/M/fJ7P7Nnv+jPWCEXZYtljrHcO7mOqqM7MJx7+P8uOdbU8q5PsBOrUarF0NL/0uDPH+ufgPqZ6vRT1pavVQd4E6tQqccpJUyweXHK2TIUc3MdUmcvV7PFm4y7vinTp5eCkV53Aox8+f0StaiZny4ypvMVNRXzpanXhOu3D4+A+ptK7vHcyMz3FmqlVXLtzgc07dnPPo/uH1EKz3rlO+/A4uI+xrZtmeWj7Fm67fGNuiYIrzlvH4SMv88LBJYJXMmcc4G1cudzG8Di410C6Fy9aRcVuuvQcHvi7A965yWql6L3sSdPqKSI/BWnQ5ubmYn5+fiQ/uyk2bL+XvN+egO/vePuwm2NmQyBpT0TMdTvOPfca8/ilmRXpGtwlrZH0NUnfkPS4pD/IOeZESTsl7ZP0iKT1g2isHc/jl1YX7X2CN2y/1xP/Q1Imz/0wsCUiXpI0BXxF0pci4uHUMe8BXoiIN0jaBtwMXD6A9lqKywXbuMkr3/vCwSUEx4YQXTJjOLoG92gNyr+UfDuVfGWHei8Bbkhu3w3cLkkxqgH9CeJywTYuOpXvzQYCl8wYvFJj7pJWS1oAngXuj4hHMofMAk8CRMQR4EXgNVU21MzGW7fyvVleuDRYpYJ7RByNiI3AGcC5kn4pc4jynpa9Q9LVkuYlzR84cKD31prZ2Oo1WHvif7B6ypaJiEXgQeDCzENPAWcCSDoBOBl4Puf5d0bEXETMrV27tq8Gm9l46iVYe+J/8LqOuUtaCyxFxKKkaeCttCZM03YBVwJfBS4Ddnu8ffi8iYcNS5mJ06z2Y7N+bw5FmWyZ04C7JK2m1dP/dER8QdKNwHxE7AI+Bnxc0j5aPfZtA2ux5Sra3AOckWDV6jZx2g7iMy7fO1JlsmW+CWzKuf/Dqds/Bd5VbdOsF0WbezgjwarWbeK03Tt/aPuW4TXKlnE994ZwKVUbtPZQTJl9Bvy+Gz2XH2gIlyKwQWoPxZTdQMbvu9FzcG8IlyKwQeolh93vu/Hg4N4Q2VKq3sTDqtRpmGVmeopTTppyCd8x45K/DZTNZgCnodnKbN6xO3dIxhOnw+eSvxMs7xI6W7TJPXnrhYf96sfBvYG6ZSp4tybrlXdQqh+nQjbQ6TPTXbManKpmvXIF0npxz72B8i6hs5yqZtZs7rk3UHoTj/2Lh5bV+8gbK3VdGrNmcXBvqPQldLfA7bo0Zs3j4D4Buo2Vui6NWfM4uE+YvF6869KYNY+D+wTJG365dudCYf1tT7qa1ZeD+wTptLgpywtUDDzRXmcO7hOk7DCLSxQYeKK97hzcJ0iZxU0C1wqZcJ3qtnuivT68iGmCeHGTdVOmbrsn2uvBPfcJ4sVN1k2Zuu3uANSDg/uE8eImy1N2Cz1PtNeHg/sE8+Img/z6/3k80V4vDu5WqGhsdf/iITbv2O0/9IboNhQzPbXa5X1ryBOqVqjT2Ko3/WiOThOkrtteX12Du6QzJT0gaa+kxyV9IOeYN0t6UdJC8vXhwTTXhuGeR/cf21ZNHY7zph/11v49Fy1ka2+h58BeT2WGZY4AvxsRX5f0M8AeSfdHxLcyx/2viHhH9U20YcqOvwYsy6pJc1pcvaQnTjv9Xj1xWn9dg3tEPAM8k9z+iaS9wCyQDe7WAEUlClZLHM3ZTN1pceOj1+ynTj12z6fUX08TqpLWA5uAR3Ie/qeSvgE8DXwwIh5fcets6Ip64kcjmJ5afVzgd+9ufJRJWy2Tw+4Vys1ROrhLejXwGeCaiPj7zMNfB14XES9Jugi4Bzgr5zWuBq4GWLduXd+NtsEpKlHQ7s15QdN46VYq4JqdC9yw63EkeOHgUtfX85VYcyhyLrWXHSRNAV8A7ouIPypx/A+AuYh4ruiYubm5mJ+f76GpNgx5Oc/9psJ5detglc1PL8spj/UgaU9EzHU7rmvPXZKAjwF7iwK7pF8AfhQRIelcWlk4P+6xzTYG0pfwZYNyXhAHcmvHX7NzwWO6FSkzzNJNe1LVv5PmKTMssxn4TeAxSQvJfb8PrAOIiDuAy4D3SzoCHAK2RZlLAhtL3VauQnHWRXusd83UqsLa8S5jUI2VZio5oDdbmWyZr0DHdGci4nbg9qoaZeOtW9bFoaWjXXuULmOwcmVKOBdp57Bbc3mFqvWsiuEAcI78SuWVcJ6eWs0V563rWNrZWU6TwbVlrGdlgvLM9BSHj7zc8UPAmRkr02l+ZO51px67/+TpKSRYPLjkie0J4uBuPes2HDA9tZobLv4nQLna8c6q6V/R/EiZeRNrNgd369l1F5y9LAWvKOuiW+1414wvxx+A1qtSee6D4Dz3eusUrHsJQu0CZVme8HtFlWsPrP4qy3M3y5N32d9PL7xo/N6Tra/wpinWD2fLWGU6BaEiRZOqnmw9vvRyHn8AWicO7laZfnrhRel8111w9rHgtmH7vWzesXuiNgZpXwV1mrj2B6B14mEZq0xRFk2nIJRN52un7V2zcyF35Wv6OU3WbS2B8HaH1pl77laZTr3wTrZumuWh7Vu49fKNHD7y8rHqhXkrXydl56dOVzt5H3qTdFVj5bjnbpXpp+hYWpmVr03sraYzjNpXLkU5bHmbpnhy1fI4uFulyi6eyUuZLDtBOOghmmHmlGczjBYPFddcz26WkubJVctynrsNXVHe9pqpVaU2lEirurJhXtsGWRa3UzZMWnqzFK8LmGzOc7exVZQyeeIJq5b1Tjtt4gzlevFleuKddjQa5KRumR53duu7vA9GFwKzLE+o2tAVBbQXDy1x06XnMDszjWj1Rm+9fCOzXVL+Ok20plMKg/wJyDJph2V+Vj/KpDOmj9m6aXbZOfJKVcvjnrsNXaeUyaIx+27byWU/MLrtLZqegOy1hPHTi4cqG5fPq9OTltcrd1EwK8PB3YYuL6B1GlpIB+Gi3nW6d1tmb9H0h0Gvk5EnT0/1XGah6MOgKM/f5XltpTyhaiPRb8+3TBGtMpOU6QnITsfnlSruNPE7kxOcYfmVx9Qq8eo1JziIW8/KTqg6uFvtdPtg2LD93o6TsNkPg04fGLA8b//anQsdXz+t24RwUZvMiji428TpNM7eVpTOuNKMmio4ndHKcCqkTYR0wO3US872xK/dudDT+HaZcfyV8kIkq5KDu9VWNuAWBfbZgrHv9GrQbpOiVW0K3omrPFqVnOdutVUm4LYXAG3dNNv1+E457J161dliad3MTE8xtVrLXsMLkaxKXYO7pDMlPSBpr6THJX0g5xhJ+i+S9kn6pqRfGUxzzV5RZhgj3Rsuc3zRMUW96vYiovaiopnpKU45aQpofbCkTU+t5rbLN7LwH8/nlsve6IVINlBlhmWOAL8bEV+X9DPAHkn3R8S3Use8DTgr+fpV4E+Sf80GpmgxVFu2N9zt+PYxeTrl5hctKuo0SeuFSDZoXYN7RDwDPJPc/omkvcAskA7ulwB/Hq3Um4clzUg6LXmu2UDkBdxORb76WQ3a1k85YwdwG6WeJlQlrQc2AY9kHpoFnkx9/1Ryn4O7DUyvAXelq0EdrK1OSgd3Sa8GPgNcExF/n3045ynLkhckXQ1cDbBu3boemmmWr9eA6wBtk6JUtoykKVqB/S8i4rM5hzwFnJn6/gzg6exBEXFnRMxFxNzatWv7aa+ZmZVQJltGwMeAvRHxRwWH7QL+VZI1cx7wosfbzcxGp8ywzGbgN4HHJC0k9/0+sA4gIu4AvghcBOwDDgLvrr6pZmZWVplsma+QP6aePiaA36qqUWZmtjJeoWpm1kAO7mZmDTSykr+SDgA/XOHLvBZ4roLmVGkc2wRuV6/GsV3j2CZwu3pRRZteFxFd0w1HFtyrIGm+TF3jYRrHNoHb1atxbNc4tgncrl4Ms00eljEzayAHdzOzBqp7cL9z1A3IMY5tArerV+PYrnFsE7hdvRham2o95m5mZvnq3nM3M7McYx/cJb0r2QHqZUmFs8ySLpT0RLIb1PbU/RskPSLpO5J2SnpVBW06VdL9yWveL+mUnGPeImkh9fVTSVuTx/5M0vdTj21caZvKtis57mjqZ+9K3V/5uSrbLkkbJX01+V1/U9LlqccqO19F75PU4ycm//d9yblYn3rs+uT+JyRd0G8b+mzX70j6VnJu/lrS61KP5f4+h9SuqyQdSP3896YeuzL5nX9H0pVDbNOtqfZ8W9Ji6rGBnCtJfyrpWUl/W/C4VLBb3aDOExEx1l/APwbOBh4E5gqOWQ18F3g98CrgG8AvJo99GtiW3L4DeH8FbfpPwPbk9nbg5i7Hnwo8D5yUfP9nwGUDOFel2gW8VHB/5eeqbLuAfwicldw+ndZeADNVnq9O75PUMf8GuCO5vQ3Ymdz+xeT4E4ENyeusruj8lGnXW1Lvn/e329Xp9zmkdl0F3J7z3FOB7yX/npLcPmUYbcoc/++APx3Cufo14FeAvy14/CLgS7RKuZwHPDLI8xQR499zj4i9EZG/a/ErzgX2RcT3IuL/AZ8CLpEkYAtwd3LcXcDWCpp1SfJaZV/zMuBLEXGwgp/dSa/tOmaA56pUuyLi2xHxneT208CzQNV1oXPfJx3aejfwG8m5uQT4VEQcjojv0yqSd+6w2hURD6TePw/TKqs9aGXOV5ELgPsj4vmIeAG4H7hwBG36l8AnK/i5HUXE39DqwBU5tltdRDwMzEg6jcGdp/EP7iUV7QT1GmAxIo5k7l+pn4+kpHHy7891OX4by99gf5hcnt0q6cQK2tRLu9ZImpf0cHuoiMGdq17aBYCkc2n1yr6buruK81X0Psk9JjkXL9I6N2We269eX/s9tHqBbXm/z2G2653J7+ZuSe19HQZ1vkq/bjJ0tQHYnbp7UOeqm6J2D+x91dM2e4Mi6cvAL+Q89KGI+O9lXiLnvuhw/4raVOb5qdc5DTgHuC919/XA/6UVwO4E/j1w4xDbtS4inpb0emC3pMeA7O5aUPJcVdiu9vn6OHBlRLyc3N33+cq+fM592f9j5e+lEkq/tqQrgDng11N3L/t9RsR3854/gHZ9HvhkRByW9D5aVz1bSj53UG1q2wbcHRHpTXMHda66Gfr7aiyCe0S8dYUvUbQT1HO0Ln9OSHphuTtE9domST9SsgF4Eoye7fBS/wL4XEQspV67vZHJYUn/DfhgmTZV1a5k2IOI+J6kB2nti/sZ+jxXVbVL0s8C9wL/Ibl0bb923+cro8yOYe1jnpJ0AnAyrcvtUruNDbBdSHorrQ/LX4+Iw+37C36fVQSsru2KiB+nvv0IcHPquW/OPPfBYbQpZRuZUuQDPFfdFLV7UOepMcMy/xs4S61sj1fR+qXuitaMxQO0xrwBrgTKXAl0syt5rTKvuWzMLwlw7XHurUDuDPsg2iXplPawhqTX0tqM5VsDPFdl2/Uq4HO0xiX/KvNYVecr933Soa2XAbuTc7ML2KZWNs0G4Czga322o+d2SdoE/Ffg4oh4NnV/7u9ziO06LfXtxcDe5PZ9wPlJ+04Bzuf4q9eBtSlp19m0Jii/mrpvkOeqm6Ld6gZ1nmqRLfPPaX26HQZ+BNyX3H868MXUcRcB36b1Kfyh1P2vp/VHuA/4K+DECtr0GuCvge8k/56a3D8HfDR13HpgP7Aq8/zdwGO0gtQngFdXdK66tgv4Z8nP/kby73sGea56aNcVwBKwkPraWPX5ynuf0BriuTi5vSb5v+9LzsXrU8/9UPK8J4C3Vfw+79auLyfv//a52dXt9zmkdt0EPJ78/AeAf5R67r9OzuM+4N3DalPy/Q3AjszzBnauaHXgnknew0/Rmhd5H/C+5HEBf5y0+TFSmX+DOk9eoWpm1kBNGZYxM7MUB3czswZycDczayAHdzOzBnJwNzNrIAd3M7MGcnA3M2sgB3czswb6/47iH2oXbal9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画图\n",
    "plt.scatter(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（4）初始化权重参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 随机初始化参数\n",
    "w1 = np.random.rand(1,1)\n",
    "b1 = np.random.rand(1,1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（5）训练模型\n",
    "定义损失函数，假设批量大小为100：\n",
    "![jupyter](http://www.feiguyunai.com/wp-content/uploads/2019/09/2e4ba2ba407b80439b5888d310093c18.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 代码实现\n",
    "lr =0.001 # 学习率\n",
    " \n",
    "for i in range(800):\n",
    "    # 前向传播\n",
    "    y_pred = np.power(x,2)*w1 + b1\n",
    "    # 定义损失函数\n",
    "    loss = 0.5 * (y_pred - y) ** 2\n",
    "    loss = loss.sum()\n",
    "    #计算梯度\n",
    "    grad_w=np.sum((y_pred - y)*np.power(x,2))\n",
    "    grad_b=np.sum((y_pred - y))\n",
    "    #使用梯度下降法，是loss最小\n",
    "    w1 -= lr * grad_w\n",
    "    b1 -= lr * grad_b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（6）可视化结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4U1X6wPHv241SEZCCCGJbGFFAWcSKKMyMu4D7iIoC4oJlccEdEUFA6+jIKKIooKCIEVTccQMUf66goKAioCC0FBwpoICUAm3f3x83qWmatikkTdK8n+fJ0+Tec29Ob5L3nnvOueeIqmKMMSa2xIU7A8YYY2qeBX9jjIlBFvyNMSYGWfA3xpgYZMHfGGNikAV/Y4yJQQEFfxFpKCJzRGSViKwUkZN81ouITBSRNSLynYh0Dk12jTHGBENCgOkeA95X1d4ikgSk+KzvCbR2P04EnnL/NcYYE4GqLPmLSH3gH8A0AFXdq6p/+CS7AHheHYuAhiLSLOi5NcYYExSBlPxbAfnAsyLSEVgKDFPVXV5pDgc2eL3Ocy/71XtHIpIFZAEcdNBBx7dp0+YAsm6MMbFn6dKlW1S1yYHuJ5DgnwB0Bm5U1cUi8hhwFzDKK4342a7cuBGqOhWYCpCZmalLliypfo6NMSaGiUhOMPYTSINvHpCnqovdr+fgnAx80xzh9boFsOnAs2eMMSYUqgz+qvo/YIOIHO1edDrwo0+yt4Ar3b1+ugLbVfVXjDHGRKRAe/vcCLjcPX1+Aa4WkcEAqjoZeBfoBawBCoCrQ5BXY4wxQRJQ8FfVZUCmz+LJXusVuD6I+TLGxIh9+/aRl5dHYWFhuLMSUZKTk2nRogWJiYkh2X+gJX9jjAmJvLw8Dj74YDIyMhDx13ck9qgqW7duJS8vj5YtW4bkPWx4B2NMWBUWFpKammqB34uIkJqaGtKrIQv+xpiws8BfXqiPiQV/Y4yJQRb8jTEmyOrVqwfApk2b6N27d6VpJ0yYQEFBQU1kqwwL/sYYE4Di4uJqb9O8eXPmzJlTaRoL/sYYEybr16+nTZs2DBgwgA4dOtC7d28KCgrIyMhg3LhxdO/enVdeeYW1a9fSo0cPjj/+eP7+97+zatUqANatW8dJJ53ECSecwKhRo8rs99hjjwWck8ftt99O+/bt6dChA48//jgTJ05k06ZNnHrqqZx66qk1+j9bV09jTOS4+WZYtiy4++zUCSZMqDLZ6tWrmTZtGt26deOaa67hySefBJz+9p999hkAp59+OpMnT6Z169YsXryYoUOH8tFHHzFs2DCGDBnClVdeyaRJk/zuf+rUqaxbt45vv/2WhIQEtm3bRqNGjXjkkUdYuHAhjRs3Dt7/HAAL/sYYAxxxxBF069YNgH79+jFx4kQALrvsMgD+/PNPvvjiCy655JLSbfbs2QPA559/zquvvgpA//79GT58eLn9L1iwgMGDB5OQ4ITdRo0ahe6fCYAFf2NM5AighB4qvl0rPa8POuggAEpKSmjYsCHLKrgyqaprpqpGVJdWq/M3xhggNzeXL7/8EoBZs2bRvXv3Muvr169Py5YteeWVVwAnmC9fvhyAbt26MXv2bABcLpff/Z911llMnjyZoqIiALZt2wbAwQcfzM6dO4P/D1XBgr8xxgBt27ZlxowZdOjQgW3btjFkyJByaVwuF9OmTaNjx44cc8wxvPnmmwA89thjTJo0iRNOOIHt27f73f/AgQNJS0ujQ4cOdOzYkRdffBGArKwsevbsWeMNvuKMyVbzbDIXYwzAypUradu2bVjzsH79es4991x++OGHsObDl79jIyJLVdV3oM1qs5K/McbEIAv+xpiYl5GREXGl/lCz4G+MMTHIgr8xxsQgC/7GGBODArrJS0TWAzuBYqDIt6VZRE4B3gTWuRe9pqrjgpdNY4wxwVSdkv+pqtqpki5Gn7rXd7LAb4yJFn/88UfpOD6xxKp9jDExraLgvz9DOEeTQIO/AvNEZKmIZFWQ5iQRWS4i74nIMUHKnzHGlOFyQUYGxMU5fysYTSFgd911F2vXrqVTp06ccMIJnHrqqVxxxRW0b9++zJDMAOPHj2fMmDEAFQ7vHC0CHditm6puEpFDgfkiskpVP/Fa/w2Qrqp/ikgv4A2gte9O3CeOLIC0tLQDzLoxJta4XJCVBZ65T3JynNcAffvu3z4ffPBBfvjhB5YtW8bHH3/MOeecww8//EDLli1Zv359hdtlZWX5Hd45WgQU/FV1k/vvZhF5HegCfOK1fofX83dF5EkRaayqW3z2MxWYCs7wDkHIvzEmhowc+Vfg9ygocJbvb/D31aVLF1q2bFlpmsqGd44WVQZ/ETkIiFPVne7nZwHjfNIcBvymqioiXXCqk7aGIsPGmNiVm1u95fvDM4QzQEJCAiUlJaWvCwsLgaqHd44GgdT5NwU+E5HlwFfAO6r6vogMFpHB7jS9gR/caSYCfTRcI8YZY2qtimqLD6QWubIhlZs2bcrmzZvZunUre/bsYe7cuUDlwztHiypL/qr6C9DRz/LJXs+fAJ4IbtaMMaas7Oyydf4AKSnO8v2VmppKt27dOPbYY6lbty5NmzYtXZeYmMjo0aM58cQTadmyJW3atCld53K5GDJkCPfffz/79u2jT58+dOxYLlRGLBvS2RgTVtUd0tnlcur4c3OdEn92dvDq+yNNKId0tmkcjTFRpW/f2hvsa5Ld5GWMMTHIgr8xJuysf0h5oT4mFvyNMWGVnJzM1q1b7QTgRVXZunUrycnJIXsPq/M3xoRVixYtyMvLIz8/P9xZiSjJycm0aNEiZPu34G+MCavExMQq76g1wWfVPsYYE4Ms+BtjTAyy4G+MMTHIgr8xxsQgC/7GGBODLPgbY0wU8MxgBscfH4z9hS34L10anCnYjDGmtvPMYJaTE7x9hrXk75mCzU4AxhhTMX8zmB2osFf7eKZgM8YY499fM5UFbwiMsAX/o1iN5x8J5hRsxhhT23hmKnuKIUHbZ9iC/8H8yb94DTiwKdiMMaa2y86G9sk/M5BngrbPsAX/QpK5j1HUq1t8QFOwGWNMbde3L7x13L3slTpB22dAwV9E1ovI9yKyTETKzb0ojokiskZEvhORzlXtcxPNacdKPhjwos3KY4wxlfn+ezIWzSZl+E04fSUPXEBz+IrIeiBTVbdUsL4XcCPQCzgReExVT6xsn5mZmbqkuBh27ICVKyEpqdqZN8aYmHDhhbBwIaxbh6SmBmUO32BV+1wAPK+ORUBDEWlW5VbZ2fDLLzB9epCyYYwxtczixfDmm3DHHdCoUdB2G2jwV2CeiCwVkSw/6w8HNni9znMvK0NEskRkiYgsyc/Ph549yW99Mv+74T5SZLfd9GWMMb7uuQcaN4Zhw4K620CDfzdV7Qz0BK4XkX/4rBc/25SrT1LVqaqaqaqZTZo0wfWi0D83m8OKNzGUSXbTlzHGeFkwciEsWMBtW0aQ0f7goMbGgOr8y2wgMgb4U1XHey2bAnysqrPcr1cDp6jqrxXtJzMzU7dsWUJODrzP2WSyhFb8wg4akJ4O69fvz79jjDG1g+sF5agBXTmsZBNH8ROF1CUlBQoKmqxTzW91oPuvsuQvIgeJyMGe58BZwA8+yd4CrnT3+ukKbK8s8Ht4bu4awb9JZRu3M77McmOMiSWewdvi4uC1AW9wQslXjGEMhdQFPEM8NC9Xpb4/qiz5i0gr4HX3ywTgRVXNFpHBAKo6WUQEeALoARQAV6tquS6h3rxL/gCz6MN5vE0rfqFuelMr+RtjYopn8LaCAoiniO/oQBwlHMsPFJeZbj0T1SX+qtqrpcqSv6r+oqod3Y9jVDXbvXyyqk52P1dVvV5V/6aq7asK/B7Z2ZCS4jwfxX3UYQ+juJ+cHBvx0xgTW7wHb+vPTNqxkpFk+wR+gH17g/F+YR3YrW9fmDoV0tNhDa2ZxkCymEJLfrHGX2NMTPFUd9ehkLHcy9dk8hr/KpPGKSxv2hiM9wv7qJ59+zqNu+npMJbRFBPPOEYDzlmwXz+7CjDG1H6eMc6G8BRpbOAuHgSE+HgQcWLk1KkAW7YF4/3CHvw9cnPhV5ozgZu5ghfpyLLSdXYVYIyp7bKzoVndP7iH+5nPGXzE6aSkwIwZUFLiFJKDORROxAR/z1nvIYbzO4fwEMPLrLdx/40xtVnfvjD/jIdIZRt38VBpST9UY59FTPD3NP5upyHZjORs5nE6C8qksS6gxphaKy+PY+ZPgCuuYKl2DnpJ31fEBH/vxt9JXM960vkPdyKUlKaxcf+NMbXWvfc69Tv3318jbxcxwR/+avyd/kId7ku6n858y2W8BDhXBTbuvzGmVlqxAp57DoYOhZYta+QtIyr4e/TtC6c9cwU/JnYkm5G0TtsT0rovY4wJF5cL5p8wgu0l9Thuzsga69gSkcEfoG//ONrN/Q+tWMdPNz9pgd8YU+u4XDDz2o85c/fbPMhdLMtrXGM9G6s9sFuwZGZm6pIlAdwI3KMHfPUVrFkT1LGsjTEmHFwup+dibi7ESwlflnThUDZzNKtLx/CpbHBLEYmoyVxC5+GHYft2q/A3xkQ9z/g9OTmgCpeWzCKTpYwkuzTwQ830bIz84N++PVx9NTz+uDPrlzHGRCnv8XvqUMgD3M03HIeLsvXaNdGzMfKDP8C4cZCYCCNGhDsnxhiz37xL9DcxkXRyuZ3xqFcorqmejdER/Js3d+avfPll+PLLcOfGGGP2i6dEn8oWRpLN25zLQk4rN35PTXRwiY7gD3D77XDYYXDrrU5lmTHGRBnPSAZjGMNB7GI4D4V0/J7KRE/wr1fPOXKLFsHs2eHOjTHGVFvfvjD73h8ZzGSmMoiC9HZhu4cpaoK/ywWtxg7gWzqxsf9wZj+7O9xZMsaYajtv4W0kNKjH0PyxNVrS9xUVwd/TPWpdbjy38CiHF29g1aBHbIhnY0xU8MzN21Peh/ffZ+k5o6Bx47DmKeDgLyLxIvKtiMz1s+4qEckXkWXux8BgZtK7e9T/cQqvcyG37/s3E4ZXOUe8McaElafwmpdTxHhu42eO5PTXbwx74bU6Jf9hwMpK1r+kqp3cj2cOMF9l+N7wcAcPk8Rehmy8J5hvY4wxQecpvGYxlWP4kTt4mO27k8I+P0lAwV9EWgDnAEEN6oHyveFhLUcykZu4imdh6dJwZMkYYwKSmwuHsI1xjGYhp/AmF5QuD6dAS/4TgDvBa3D98i4Wke9EZI6IHOEvgYhkicgSEVmSn58fcCY93aO83c8ottCYRV2H0ThViYuzuX6NMZEnLQ3uZSyH8DvDeAyQ0uXhVGXwF5Fzgc2qWlkR+20gQ1U7AAuAGf4SqepUVc1U1cwmTZoEnEnviV6cPMF2GnA3D9C16HPO2PYSqjbXrzEm8jwxZAXXM4kpDOJ7OgCRMT9JlaN6isi/gf5AEZAM1AdeU9V+FaSPB7apaoPK9hvwqJ4+MjKcIA8QRzFfcwJNyOdoVrMb5/KgshHxjDGmxqjC2Wez5/Ov6droZ5ZvbExamhP497eLZ42N6qmqI1S1hapmAH2Aj3wDv4g083p5PpU3DB8Q73qyEuK5iYkcQR7DechvGmOMCQeXCwY2fRvmz+eBOuO4/cHGNX4Xb2X2u5+/iIwTkfPdL28SkRUishy4CbgqGJnzx7ee7HO6M4s+3Ml/SMO5JIiLw9oAjDFh43LBjdcVclf+raygHQ/8PjjiqqQjfzIXH54+s55+/wAt2MAq2vAePbmEOWXSp6TU3EBJxhgDTsHzipwHeICRnMk8FnAmEJwq6diZzMWHd+OvCKSmwu7UI3iAkfTmVc5gfpn0BQWEvT+tMSa2aE4u93A/c7i4NPBDZFVJR13Jv0J79vBz8rEUkUBHlrOPpNJVIs6IecYYUxPmHnQppxXMpS0rySW9dLmV/EOhTh0eaDKBtqziJiaWWRXu/rTGmBjy4YecW/AK4xNHlAn8kdC901vtCf7AGY+ew7tx53IvY2nGJiDyDrgxphbbtw9uvBFataL1lDtKq6drcpKWQNWq4N+3L+x7eAJJ7OVh7ojIA26MqcUmToSVK2HCBC6/Opn162t+kpZA1argD3DBrX+jzqjh9OVF1j+7MOIOuDGmdnptYh5/3jGGuZxDxg3nRlS3Tn9qXfAHnIneW7aEoUNh795w58YYU8u5XKC33kq8FnETE8nJlYjr1++rdgb/unXh8cdh1Sp49NFw58YYU8u9f+s8Li5+hQe4m3W0AiK/m3nt6erpz0UXwbx5Th2cdfkxxoTCnj38lNweQWnP9+whuXRVKLqZW1fPQEyY4AysNGxYuHNijKmFXC4Y3/RhjuJnrmdSmcAPkV3mrN3BPz0dRo+GN96AueVmnzTGmP3mcsG/B67l+u3ZvEJv5nNWmfWR3s28Vgd/lwtaP+UMrJR34fXcfN0uMjJs0DdjzIEbebfy38Kh7CORm5lQZl00dDOvtcHfMwDcmtwkBjGFFsW5NH9mLDk52MQvxpj95nI5hceuuS9xNvMYSTabOLx0vUhk9uv3VWuDv2fSZHCGfX6Ga7mVR2jPd6VpCgqgXz+7CjDGBMZTqPwj5w8mcDNfk8mTDC2TJpLr+b3V2uDvO3recB7idw5hCoMQn6mI7SrAGBMIT6HyAe6mCfkMYgolxJeuj/R6fm+1Nvj7nn23kcpt/JeTWEQWU8ulj/Q+ucaY8MvNhRNZxGAm8zg38i2dS9dFQz2/t1ob/LOznbOwt5n0ZwGn8xDDac7GcttE0ljbxpjI4Knjj4uDOrKXZxhIHi0YxX2laTxDNUdL4IdaHPx9J31JT4chQ4RxzaeQxF6e4IZy20RLXZ0xpmZ46vg9HUVuL3mIY1nBUJ7kTw4GoquqpwxVDegBxAPfAnP9rKsDvASsARYDGVXt7/jjj9dw+abPQ6qgF/GqOh+pakqK6gsvhC1LxpgIlJ6upTHiaFZqIUk6i8s0Pl5VxFlf03EDWKIBxu3KHgEP7yAitwKZQH1VPddn3VCgg6oOFpE+wEWqelll+6uR4R0qUlTEtiNPYF/eb7Qp/pEG6Q3Jzo6uSzZjTOjFxTmhXyjhY07hWH6gLSvJl6Zhmx2wRod3EJEWwDnAMxUkuQCY4X4+BzhdRORAMxcyCQk0evUZmupv/J41POrq6owxNcNTFTyQZ/gHn3I749lM01pRRRxonf8E4E6gonPd4cAGAFUtArYDqb6JRCRLRJaIyJL8/Pz9yG4QHX883HKL0zCwcGF482KMiUjZ2dA6eQPjuZ0POY1nuTp66/h9VBn8ReRcYLOqLq0smZ9l5eqTVHWqqmaqamaTJk2qkc0QGTcOjjwSBg6EXbvCnRtjTITpe4Wy8OjBJEgxWTxNerpEVXfOygRS8u8GnC8i64HZwGki8oJPmjzgCAARSQAaANuCmM/QSEmBZ56BX36BUaPCnRtjTKRxuTh8+bukPJLNWm1Vq6qIqwz+qjpCVVuoagbQB/hIVfv5JHsLGOB+3tudJjwTBVTXP/8JQ4Y4wz8vWhTu3BhjIsVvvznDwZ90kjMpey2z3/38RWSciJzvfjkNSBWRNcCtwF3ByFyNefBBaNECrrkG9uwJd26MMTXM+0aujAxnBti5rW5kz7Y/OT1nGq7Z8VXtIuokVCexqn4MfOx+PtpreSFwSTAzVpNcb9dn3p6pzNjQk0mHjqXhkw/Umks7Y0zlPDdyeQaCzMmB/Kde4Vxe4W6y+WhTWxZlOetqU1yo3dM4BsD7g3+agVzNs5xa50sGTetSqz5oY4x/GRlOwPdowmZWcAzryeAkvqTYXUb2DOEQbsHq5x/zwd/7g6/Pdr6nPbs4iAvTvmV1TnKl2xpjop/nRi6H8gqXcB5vcxzfspJ2pelCMR/v/rA5fIPEezC3HTRgIM/QllUMzB1d8UbGmKjnqef3Lv9exkv05lVGM65M4IfaN/ZXzAd/3w90PmcxhSxuYzx88UV4MmWMCSnvAds8mvI/JnE9iziR/3JbmfS15cYubzEf/P0N/TwiYTx5cWn83G0AbdN22SQvxtQy3jP9OZSnuY4UChjZ/DmyhiSUGRG4ttzY5a1avX1qI88HOnKkUwXUqBHs3Hkw/YtmsJBTuXHDHWRlPVkmrTEmuvnO3XEt0ziPudzMBD7c2CY8maphMV/yByeor1/vNObUqwd798In/JNHuYWhPEX3gg9sli9jahHv6t6W/MKj3MKHnMababXvZq6KWPD34V0iGEk2K2jHs1zNzpzIH63CGBMYT3VvHMXMYAAlxDE0+VnufyB2QmLs/KcB8i4R7CGZ/sykCfk8lzK0bLcAY0zU8sz0d3/D//J3PmNs6uOMfiYtpqp2Lfj78G0A/pbOZCeO4byCl7CWX2Nqj77tvmXErnvgoot4JL9/TAV+sOBfju/cv6mp8OTBd/Ep3dlx5fW8MWF9aVrf8UDs3GBMlCgocH7sjRvD0087P/YYY8HfD08D8MyZsHs35G+Lpz8zUYVDb+vHi88XlZvYOSfHeW0nAGOiwJ13wsqVMGOGU8KLQRb8K+HdFziHDK5nEieXfM6mmx7000/YeW29goyJXC4XXNX0XZg0iWcOvgXX5jPDnaWwifmxfSpTdswPAMVFXy7lZbrzOYs5sdw2kTL+hzGmLJcL7rnuNxbt7sBvNKULXxGfkhx1N3DZ2D41oPxYHsJQniSPFrzI5dRnewDbGGNqUkVtcffcXcJTuwdQnx1cwYvsITmmr9Yt+FfC39AP22nIFbxIGrlMZjDeUxXXxvE/jIkmlbXF9c59hB58wC08ygqOLd3G927fWGHBvxK+PX/i3ZP5fMnJ3MtYLmc2V/Ec4LQZ1a0L/ftbzx9jwqWitrgJ/b7mAUbwGhcxhUFl1sfq1brV+VeDdxtAHMXM50xOZDGd+Ya8lKPLfOlSUmrnYFDGRLLy7XRwMDv4hs4ksZdOLON3GpWui8bfaY3V+YtIsoh8JSLLRWSFiIz1k+YqEckXkWXux8ADzVgk8i4hlOB0/9xNXV7iMkoKdpdJG8t1icaES/lSvDKZwbRkHVfwIr/TiPj42j1aZ6ACqfbZA5ymqh2BTkAPEenqJ91LqtrJ/XgmqLmMEL5tAJs4nKw6z9OJ5TzKLeXSx2pdojHh4vsbvY6nuYJZjOI+Pqc74PTGKylx7uWJ1cAPAQR/dfzpfpnofsTkIDe+bQDp6XDxtF5Mrn8ng5lCH2aVSR+rdYnGhIv3b7QDy5nITXzAWTzIXaVp7HfpCKjBV0TiRWQZsBmYr6qL/SS7WES+E5E5InJEBfvJEpElIrIkPz//ALIdPt7DP3tKDvUn3s+XcSczlSxa8xNgPX+MCZe+fWH99zv59LBL+F0aOXfnu0Od/S7/ElDwV9ViVe0EtAC6iMixPkneBjJUtQOwAJhRwX6mqmqmqmY2adLkQPIdUa4YkMj/Hp1NUVwSL3MpRx2xO6brEo0JK1UYNIj6m9fyw92zSEk/1Or4/ah2bx8RuRfYparjK1gfD2xT1QaV7Scae/tU6d134Zxz4JprYNq0cOfGmNg0aRLccAPcf3+t7HVRk719mohIQ/fzusAZwCqfNM28Xp4PrDzQjEWlXr3gnntg+nQL/sbUIM9dvSfJIvbecAsbO50DI0aEO1sRLZBqn2bAQhH5Dvgap85/roiME5Hz3WlucncDXQ7cBFwVmuxGgTFj4Iwz4Prr4ZtvABv62ZhQ8tzVuysnn5e5hI0cTpfVM3HNsntYK2M3eYVCfj507gwJCbxy11KuurWR3QBmTIhkZMCGnGLepwd/51NO5gu+pTPp6U6njNrGBnaLZE2awJw5sHEjjW/uR2FBcZnVdgOYMcGTmwv3MYozWcANPMG3dC5dbipmwT9EXGtO5O6DH+fUwvcYy73l1tsX05jgGJQ6h7v5N1PIYhp/DS5g/fkrlxDuDNRGnjrIgoIsWrKEe8hmKcfzBheVprEvpjFBsGIFE3dexeK4rtxUMrF0sfXnr5qV/EPgr5EFhRt4gsV04XmupI27E5R9MY0Jgj/+gIsuIrFhPTY+9irN0utYf/5qsAbfEPAdWfBw8ljK8WynAb1bLGb4g4fYF9OYA1FcDOefD/PmwcKF0L17uHNUY6zBN4L5VulspAUX8yoZrOe7tpfR97Ki8GTMmCjm3WV6cqMRzk2Vjz8eU4E/mCz4h4C/GcC+TenONwOfgvnz4Y47wpMxY6KU9wxdfXUmg3c8zJMMRYYMtntn9pM1+IaAp0pn5EinV09amnNC6Nr3Wjjoe5gwAdq3d4aBMMZUydOO1oXFPM11fMSpDGMC8NdUjWD1/NVhdf417MXni2gxqBddCz+mb9MPufC/f7cvrDFViIuDw3UDX9GFAlLowldsI7VMmtp6U5cvq/OPQi4XXDckgQsKX+IXWvHUbxfx4MA1pZesNgyEMf61bbGTtzmPFAo4n7fKBX6we2eqy4J/DfJcuv7BIZzLXARlTuE5DLvyd0Scyd9zcpyeQp5LWTsBmFjlKQwlSDGP/u9yjuUHLuEVfuQYv+nt3pnqseBfg7xLJms5kgt5g5as46WS3iSwr9zE0zYMhIlV3g2847mNs/a9w81xj/NN6lmAM5OeN7t3pvos+Ncg35LJZ/yda5nG6XzEFAbhb3ZMu5Q1schzlXw9T3Azj/EItzCpZAj16jlXxjNnlp1O1W7qqj4L/jXIXxfQF+jPWEZzDc8ymnHltrFLWROLcnPhAt5gIjfxJudzBw+XLgf/06ma6rHgX4N8J4CPj3eWj2EMz3IVYxnDVTxbmj4xEf780xqATey5oOkiZnE5X9GFy5lFCc6PxQpDwWPBv4Z5l1hmzPBcCQhZTGUeZzKVLM5kHqmpzgli61ZrADYxZs0aZu06j01yOOfxNrtxLpetXj+4LPiHkfeVQLEkcusRc9iZdgzz6l1M18Sl7N1bNr01AJta77ffoEcPkuvAioff46D0JlavHyJ2k1ek2bQJunVj8/pddONz1tC6zGoR56rBmFpnxw445RRYvRo++ghOPDHcOYpINTmBe7KIfCUiy93z9I71k6aOiLwkImtEZLGIZBxoxmJW8+bwwQfExynzOIvD+LXMaqvzNNHO782MhYVw4YWUfPc9V9Z7jbiTTrSROJsvAAAW/ElEQVR2rhALpNpnD3CaqnYEOgE9RKSrT5prgd9V9UjgUeCh4GYzxhx1FF+NeY8m5PM+PWjAH4DVeZro591/39OWde1VxbzZoD8sXMiVxc8xc/PZ1s5VA6oM/ur40/0y0f3wrSu6AJjhfj4HOF3E9zYMUx09R2WyePjrtGUl73AObY7YZXWeJur9NdGRh/JE0SAu2DuHW3gEF2W/4NbOFToBNfiKSLyILAM2A/NVdbFPksOBDQCqWgRsh/KDb4hIlogsEZEl+fn5B5bzGHD6g2eSNGcW3eIWsfLoC+l7cWG4s2TMASl706LyCLcykGmMZTQTuCWAbUywBBT8VbVYVTsBLYAuInKsTxJ/pfxyLcmqOlVVM1U1s0mTJtXPbSy6+GKYPh0WLGBeoz4kyT6rCzURr6JBCr3brO5lLLcwgUe5mTGMqXBf1s4VGtXq6qmqfwAfAz18VuUBRwCISALQANgWhPwZwJUwgFsTH+es3W/yLFexIafY6kJNxPJXr3/11dC4sfNcBG5jPGMYyzSu4VYewX/50dq5QkpVK30ATYCG7ud1gU+Bc33SXA9Mdj/vA7xc1X6PP/54NYFJT1cF1eH8WxV0OlepUKzp6eHOmTHleb6vFT2G8agq6Cwu0yaNijQpqex6EedverrqCy+E+7+JPMASrSK+BvIIZCavZsAMEYnHuVJ4WVXnisg4dybeAqYBM0VkDU6Jv08Qz08xz1Pn+RB3UYc9jGUMRSQwOGcKdp+eiTSV1dEPZRITuIU5XMw9aTPZnBOPy1V+1jvr2BB6VQZ/Vf0OOM7P8tFezwuBS4KbNeORluZcLgOMYzSJ7OMesqlbLwH0yfLj2xoTBp4gXtF9o1lMYRI38CbncwUvUrQhEXACvQX7mmfFxihQdjRQYRT38UjCnfT7czIMGWK3/Jqw867n92cwTzGFwbxDLy7lZfaRZA25YWbBPwr4jgaamio8UP9BHuQumDKFNaddB8XF4c6miWHl++//5a6UiTzFUN7iPP7Fa+yljjXkRgAL/lHCMxrozJmwezds3SaM4AHGMpoj/286L6VcTYIUWzdQExYV1fPfxn/5d8EwcjMv4ra0OeyTOjZIW4QIpMHXRJCyJSxhDGMpIoH79o4G9tA/ZyZZWUmA/bhMzfFul3Iod/MA2dwDl1xCmsvFz4mJ4cqe8cNK/lHGXwnrfkZxB//hMl7mdS6ipGC33RJvalTZdinlIYaTzT2s69YPXnzRmZnIRBQL/lGmokay8dzBICbTk/d4j578nrOjZjNmYpqnXaplWjGTGcKdPMxPZwyl5SczIMEqGCKRBf8o428eYI+pDKIvLrrzGZ8mnQabN5dZX9Et98YEQ9/ee/jl5H4MYgqMGMFR855wvmwmItknE2XK9/yBpKS/1s/mcvokvU47XQHdusG6dYD/W+5tiAgTNDt2wDnnwOzZ8NBD8MADdv9JhLPgH4W85wHessUZ981zMkhPh4umn0fCwgXOBMAnnwzLlvntimfD5Zqg+O03Zwaujz+G556DO+8Mc4ZMICz41wLeJ4PsbCegx/29G2ckf8bvOxPY2fkfHJmzwO+2NlyuOSA//eQUMFavhrffhgEDwp0jEyAL/rWIb9XOh7+2o/2uL1mv6bxHT65merlt7C5Ls98++YQ9nbuyZf1OuhZ8RMaQnlaNGEUs+Nci/qp2NtKCbnzOR5zGdK4lm7sRnOEg7C5Ls99cLopPP5N1BU3pUrKIxZxo7UhRxoJ/LVJRFc5O6nMuc5nKddzNv5lNH44+osDusjTV5ppZwmMN74V+/fi06GRO0i9YR6vS9daOFD2sA24tUv4uy78UkcggpvATR/Ef7uTSJmvhH2/gnoPHmCrNnraLlEEDGFb8Ks9yFYOYwj6SyqWzdqToYCX/WqSyewAcwlMpt/N/t8+FNWvghBPgyy8BuwfAVCEnhw5DunF+8evcwiNcw3S/gR+sHSlaWPCvRXzvAUhPd0Z89n49dSqc+nAvWLQI6tWDU05h8bVTybpO7R4A49eCuz9ia6tMmu9bzzm8455o3aZdjHrBmA5sfx42jWME2LpVtUcPVdBnuEaTKSg35Z5NpRfDSkr0mz4PaRFx+gPt9ChW+Z2WMT7emXrRvis1gyBN42gl/1jWqBHMnct9jOJapvMZ3clgXZkkdhUQo3bsgEsu4bjZw3mVizmRxfzE0eWSpaTAjBnOPSbr11sHgmhSZfAXkSNEZKGIrBSRFSIyzE+aU0Rku4gscz9G+9uXiRyldfyJ8YyNH8e5vM3fWMu3HMdFvFYmrfXgiDFLl7KzdWeKXn2D2xjPZbzELuqVS2bj8ke3QEr+RcBtqtoW6ApcLyLt/KT7VFU7uR/jgppLE1S+N4MVF8M7nMtxfMtPHMVrXMxEbiSJPaXbWA+O2s3lgox05SZ5nD2ZJ/PH5j38k//jEW7DX/1+erqV9KNdlcFfVX9V1W/cz3cCK4HDQ50xEzoVTbm3Ib4l3fmMR7iFG3mCRXSlDSsB68ERrQLpxeVywd3X5TMh9yImchPzOItOLOMLuvndpzXq1hLVaSAAMoBcoL7P8lOArcBy4D3gmKr2ZQ2+4SNSvtEOnOUvvKCakqJ6Lm/pZhprAcl6c+IT+sLMknBn21ST57P0/Yw9DflDhjh/z+Y93cRhWkiS3swjCiV+vx/WASAyEKQG3+oE/nrAUuBfftbVB+q5n/cCfq5gH1nAEmBJWlpaSA+QqVh6esU/bFXnx52ernoYv+rCZKc3kPbqpbpxY6X79WxnPT/Cy/M5VBTAPY+67NLHuFEV9DuO1fYsrzS95/thwqtGgz+QCHwA3Bpg+vVA48rSWMk/fPyVCFNSKgjWJSWqjz+umpys2rCh6vPPO8sC2Kd3KdNOBDXD3+fg79GdT/Rn/qYK+ijDtA67K01f4ffD1LgaC/44rT3PAxMqSXMYIO7nXdxVQ1LZfi34h1egpXRPuqNYrV/XOVkVdF7d87Q5G8tsV1VJ04JHzajyc+BPncBNWozoGlrpP/jYb7rERNXUVLuKi0Q1Gfy7Awp8ByxzP3oBg4HB7jQ3ACvcdf6LgJOr2q8F/8jnW4qMo0hv5hEtIFn/oL4OYZLWq1ukL7xQcTuCVRvUrMo+h568o+tIVwWdyA2awp9Wrx+FghX8PaX1GpeZmalLliwJy3ubwGRk+B8o7m+sYTKDOYMPWcSJjGk2lVVJHSocVM5DxLkZyISOv8/sMH7lMYZxKa+wgnYMYgqf073ctikp1m8/GojIUlXNPND92B2+pkIV9e1fy5GcyXz6MZO/sZa5v3bm/aNvolndPyrdn6e7qA0iFzreg/slspfbGM9qjubixLdY3vs+Lkz7li+ke4XjPlngjyHBuHzYn4dV+0S+QHqMNGKLPl9viGpcnO4+uLHe2ehpjaOoXPWDp86/Wo3NZr+88IJqv0M/0JUcrQqa1+kc1Z9+Cne2TJBQ0109g/2w4B/5Auk54gnyvZp9o78d1d150bGjLhj+gd8G5aq6mZrAVNhgv3x56WB9euSRqnPnhjGXJhQs+Jsa4RtkPDcGeQf+0hJ83RL99IbZqi1bOgvOOkv1m2/K7K+yG8xMYPydlFsn5+qaf1ztHMiGDVXHj1ctLAx3Vk0IWPA3YVVpCb6wUPXRR1UbNXIW/utfqt9/X/V2pkLeJ+H4+L+OWzM26kRu0EKStJAk1dtuc4bqNrVWsIK/Nfia/VJRY3BuLlCnDtx8M6xdC/feCwsWQIcOcNllTMpaXm62MRGnh0rjxs7DGoLL8jcQ3+Hk8Sg3s5a/MZjJzGAAR/ETGXPG43qvUbizbKJBMM4g+/Owkn90q1YJfutW1REjVOvVUwXN69hLezf9xG/VkTUEl+d9rI9ilU7jat1Dou4jXqdzlbZkrR23GIJV+5hw2q9eO9u2qd5/v2rjxqqg3yZ10St4QRPZU+EJINZmifKu3klNdR5QomcwT9/iXC1GtIBkncgNmsZ6u6EuBlnwN2FX3SEiPOlmTdulOmmSruIoVdBNHKb3cq8eQU6lPYtqqkQbrgHqfE+oDdmmQ3lCV9BWFfR/HKpjGK1N+K30xFhZLyxTO1nwN1GhsiuEjLRiPZv39F16aDGixYi+Q0+9iFc1icJKS7WhCsj+8ltT49ykp6sKxfoPPtbn6acFJKuCfs3x2p8ZZY6J5xhaA3rsseBvokJlwck70Gbwi45llG7gcFXQbTTUqQzUf7JQheL9vgqo7tVJZVceIbsCKSlRXb5cH+JOzeEIVdA/qK+TGKLHsbTSk5/dNBd7LPibqFBVv37fOu4mjYq0B+/p89Jfd+A0EP9KU51Mlvbg3TKl36pKt4EGxkCHQfYXgPe7eqi4WPXzz1XvuEO1dWtV0L0k6Nuco5fj0rrsCrhEb/MoxBYL/iYq7G+1xAsvqKbW3aWXMltnc2npiWAnB+nbnKM3MFGPZlW5uQUq6g9f2XsHWuL3dwVQrRL3xo2qzz2nv5x8hebHNSkN+Bvbn6X61FP6ylP5lZ6ErERvVC34myhxINUS3lUxSRRqT97RSQzRnzjyr50ddphq796qEyboe2MWaaO6BVUGbd/G0ECGo/Z9VHVicc3Yp2c2+16v5Rl9+aCrdMehfytN9BuH6vP008txaQN+L3M8/PX2sRK98Ras4G9DOpuQc7mcSeNzc52RPbOzqzd6pOcmJ+9J59sl/8L0KxZw4t5P4dNPS8cxLiKeH2nHMjrxI+1YRRtW0pb1ZLCXOoAzguX69X/tq6KhqwFSU2HnTti7969lIk4UB4iniBbk0YZVtGEVbVlJR5bTkeXUpRCAfBrzhXRnad3uvFlwBt/THn/3V6anV//YmNgTrCGdLfibqFDZCcTlgieGb6DZxq/pzDd05hs6sYzm/FpmH79yGHlxaTTtfDhpx7lvJ05NZfGKejztSuH3vSnsIxGAOklww5Bi/nlCAYsWFjDvtV3w+zaasIVUttCMX0kjl8PZSALFpe+xjUP4jg7uXHRmCZms5micCfGqZmPqm6pY8DcxzXMyyMkpWxL3Vp/tHM1q2skq0nU9revk0rxkA4fu28ih8VtJ1S3ElxQF/J4lCFtJZQuN+Y2m5JJGLmnkkM5qjmYVbcinCYEG+or4XpkY4y1YwT8hGJkxpib5VgNVVH7ZQQNWpHRh2NQugHubPe6VxQBKfXZwELtIoYDU5ALuHVlEr17uNHFxTlHc/UhKbUAx8SH8zxwVjZtkTDBZ8DdRZ+TIsvX/FfGuQ8/I8LeNsIMG7KABAGsLYegzsP4e//trke6/bSA93flb1TSWgfLMeGZMKFU5qqeIHCEiC0VkpYisEJFhftKIiEwUkTUi8p2IdA5Ndo0JrGTsqTrx1J0HWpquLJ33FIkeKSnOcn/rxF37k5oKSUnltxsypOL9GRNqgQzpXATcpqptga7A9SLSzidNT6C1+5EFPBXUXBrjpaqSsb8AGmhpurJ0ffs6jbH+5r31t27mTKdKassWmD69/HZPPlnx/owJtWo3+IrIm8ATqjrfa9kU4GNVneV+vRo4RVV/rWA31uBr9pu/rp+eRt+Kukv628aX9bQx0SAsvX1EJAP4BDhWVXd4LZ8LPKiqn7lffwgMV9UlPttn4VwZABwL/HAgma8hjYEt4c5EAGIsn40bQfPDITEJ9u2FTRthy7bqbbNjO9RvUME+ouF4RkMewfIZbEer6sEHupOAG3xFpB7wKnCzd+D3rPazSbmziqpOBaa697ckGGevULN8BpflM3iiIY9g+Qw2EQlKlUlA0ziKSCJO4Hep6mt+kuQBR3i9bgFsOvDsGWOMCYVAevsIMA1YqaqPVJDsLeBKd6+frsD2yur7jTHGhFcg1T7dgP7A9yKyzL3sbiANQFUnA+8CvYA1QAFwdQD7nVrt3IaH5TO4LJ/BEw15BMtnsAUln2Eb3sEYY0z4BFTnb4wxpnax4G+MMTEopMFfRC5xDwlRIiIVdqESkR4isto9PMRdXstbishiEflZRF4SkaSK9nGA+WwkIvPd7zNfRA7xk+ZUEVnm9SgUkQvd654TkXVe6zqFK5/udMVeeXnLa3kkHc9OIvKl+/vxnYhc5rUuZMezou+a1/o67mOzxn2sMrzWjXAvXy0iZwcrT/uZz1tF5Ef3sftQRNK91vn9/MOUz6tEJN8rPwO91g1wf0d+FpEBYc7no155/ElE/vBaVyPHU0Smi8hmEfF7/5O7Q43fYXT261gGY0aYih5AW+Bo4GMgs4I08cBaoBWQBCwH2rnXvQz0cT+fDAwJUT7/A9zlfn4X8FAV6RsB24AU9+vngN6hPJbVySfwZwXLI+Z4AkcBrd3PmwO/Ag1DeTwr+655pRkKTHY/7wO85H7ezp2+DtDSvZ/4EB2/QPJ5qtf3b4gnn5V9/mHK51U4IwL4+w394v57iPv5IeHKp0/6G4HpYTie/wA6Az9UsL4X8B7OfVVdgcUHcixDWvJX1ZWqurqKZF2ANar6i6ruBWYDF4iIAKcBc9zpZgAXhiirF7j3H+j79AbeU9UAxpYMqurms1SkHU9V/UlVf3Y/3wRsBpqEKD8efr9rPmm88z4HON197C4AZqvqHlVdh9OzrUu48qmqC72+f4tw7q2paYEcz4qcDcxX1W2q+jswH+gRIfm8HJgVorxUSFU/wSlUVuQC4Hl1LAIaikgz9vNYRkKd/+HABq/Xee5lqcAfqlrkszwUmqr7vgT330OrSN+H8l+ObPel2KMiUicUmSTwfCaLyBIRWeSpmiKCj6eIdMEpka31WhyK41nRd81vGvex2o5z7ALZNliq+17X4pQIPfx9/qEQaD4vdn+Wc0TEczNoRB5Pd/VZS+Ajr8U1dTyrUtH/sV/H8oDH8xeRBcBhflaNVNU3A9mFn2VayfL9Ulk+q7mfZkB74AOvxSOA/+EEsKnAcGBcGPOZpqqbRKQV8JGIfA/4DskBkXM8ZwIDVLXEvThox9P37fws8z0GNfJ9rELA7yUi/YBM4J9ei8t9/qq61t/2NZDPt4FZqrpHRAbjXFWdFuC2wVKd9+oDzFHVYq9lNXU8qxLU7+YBB39VPeMAd1HR0BBbcC5rEtwlsAMaMqKyfIrIbyLSTFV/dQejzZXs6lLgdVXd57Vvz93Me0TkWeD2cObTXY2Cqv4iIh8Dx+EMzxFRx1NE6gPvAPe4L2M9+w7a8fQRyDAknjR5IpIANMC5FK/JIUwCei8ROQPnZPtPVfXMUVbR5x+KYFVlPlV1q9fLp4GHvLY9xWfbj4Oew7/eK9DPrg9wvfeCGjyeVano/9ivYxkJ1T5fA63F6YmShHPw31KnJWMhTv06wAAgkCuJ/fGWe/+BvE+5+kB3gPPUq19I6EYrrTKfInKIp5pERBrj3KH9Y6QdT/dn/TpOHeYrPutCdTz9ftcqyXtv4CP3sXsL6CNOb6CWOHNXfBWkfFU7nyJyHDAFOF9VN3st9/v5hzGfzbxeng+sdD//ADjLnd9DgLMoezVdo/l05/VonAbTL72W1eTxrEpFw+js37EMcev1RThnpT3Ab8AH7uXNgXd9WrF/wjmbjvRa3grnB7YGeAWoE6J8pgIfAj+7/zZyL88EnvFKlwFsBOJ8tv8I+B4nSL0A1AtXPoGT3XlZ7v57bSQeT6AfsA9Y5vXoFOrj6e+7hlOldL77ebL72KxxH6tWXtuOdG+3GugZ4t9OVflc4P5NeY7dW1V9/mHK57+BFe78LATaeG17jfs4rwGuDmc+3a/H4AxN771djR1PnELlr+7fRR5OW85gYLB7vQCT3P/D93j1oNyfY2nDOxhjTAyKhGofY4wxNcyCvzHGxCAL/sYYE4Ms+BtjTAyy4G+MMTHIgr8xxsQgC/7GGBOD/h98Y5ncH0BNowAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2.98927619]] [[2.09818307]]\n"
     ]
    }
   ],
   "source": [
    "plt.plot(x, y_pred,'r-',label='predict')\n",
    "plt.scatter(x, y,color='blue',marker='o',label='true') # true data\n",
    "plt.xlim(-1,1)\n",
    "plt.ylim(2,6)  \n",
    "plt.legend()\n",
    "plt.show()\n",
    "print(w1,b1)# 打印w，b的值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用Tensor及antograd实现机器学习\n",
    "上一节可以说是纯手工完成一个机器学习任务，数据用Numpy表示，梯度及学习是自己定义并构建学习模型。这种方法适合于比较简单的情况，如果稍微复杂一些，代码量将几何级增加。是否有更方便的方法呢？这节我们将使用Pytorch的自动求导的一个包antograd，利用这个包及对应的Tensor，便可利用自动反向传播来求梯度，无需手工计算梯度。以下是具体实现代码。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（1）导入需要的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch as t\n",
    "%matplotlib inline\n",
    " \n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）生成训练数据，并可视化数据分布情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHrpJREFUeJzt3X+QHHd55/H3o/VgDQd4/UMJ0tpCovApF+KcBFuGO1WRIChEDJEVY85KlXN2DkplEpIgLs6twhUhrkpZnKvikErufCLkYkIq3sQYRSA4nYnsyuGKTVbW2sKWZYRJzlr58Bp7HXxaxEp67o/pkXp7u6d7dntm+sfnVaXSbE/P7Fc9rWe+/e3n+3zN3RERkWpZNugGiIhI/hTcRUQqSMFdRKSCFNxFRCpIwV1EpIIU3EVEKkjBXUSkghTcRUQqSMFdRKSCLhjUL77ssst8zZo1g/r1IiKldPDgwRfcfUXafgML7mvWrGFiYmJQv15EpJTM7J+y7KdhGRGRClJwFxGpIAV3EZEKUnAXEakgBXcRkQpScBcRqaCBpUKKiNTFnkNT3LH/KCdmZlk13OTWzevYumGkp79TwV1EpIf2HJpi532HmZ07A8DUzCw77zsM0NMAX/rgPohvRBGRrO7Yf/RcYG+bnTvDHfuPKrgnGdQ3oohIVidmZrvanpdSBvd2b30q5uD04xtRRCSrVcPN2Fi1arjZ09+bKVvGzP7RzA6b2aSZLSgIYy1/aGbHzOxxM3tL/k1taffW4w5WW6+/EUVEsrp18zqajaF525qNIW7dvK6nv7ebnvs73f2FhOd+Drgy+PM24L8Ff+cubvwqqtffiCIiWbVHEcqaLXMt8Hl3d+BhMxs2s5Xu/lxO739OWq+8H9+IIiLd2LphpO9DxVknMTnwv8zsoJltj3l+BHg29PPxYNs8ZrbdzCbMbGJ6err71tK5Vz4y3OT2667SeLuI1F7WnvtGdz9hZj8G3G9mT7n734Wet5jX+IIN7ruB3QCjo6MLns/i1s3r5mXIQKu3rqAuIoNWpNTsTD13dz8R/P088CXg6sgux4ErQj9fDpzIo4FRWzeMcPt1VzEy3MRQb11EiiGc7OG0UrN3jE+yZmwfG3cdYM+hqb62J7Xnbmb/Aljm7j8IHr8HuC2y217go2Z2D60bqS/3Yry9bRDjVyIincQle7SHJwYxBydLz/3HgW+Y2WPAN4F97v4/zewWM7sl2OerwDPAMeCzwK/0pLUiIgWVluzRnoPTL6k9d3d/BvjXMdvvCj124FfzbZqISHkkTVYK6+ccHJX8FRHJQdxkpah+zsEpZfkBEZGiCU9WmpqZxZifMtjvOTgK7iIiOQkneww6LbJSwX3QB1NEpG3QWX2VCe4q/ysicl5lbqh2KogvIlI3lQnugyqILyJSRJUJ7kkpRir/KyJ1VJngHpdjarTG3gdR10FEZJAqc0O1U46pbq6KSN1UpucOrcD90NgmRoabC+oN6+aqiNRJZXruYbq5KiL9UtT5NZXqubcl3UR10Pi7iOQmrob7zvsOFyLGVDK4dyrgU6SDLyLlVuT5NZUM7uHVmuIU5eCLSLkVeQi4ksEdzt9cjVvcFYpx8EWk3Io8v6aywb2tyAdfRMotbgi436V9k1Q+uBf54ItIOe05NMXGXQfYMT7JhRcs4+JXNzBgZLjJ7dddVYhsmUqmQoaFJzedmJnlomYDM9gxPskd+48WJm1JRMohWoF2ZnaOZmOIO29YX6hYUvmeO5wff7/zhvWcOn2Wl07OFS5tSUTKocgZMmG1CO5tZflQRKS4ipwhE1ar4F6WD0VEiqssSRq1Cu5l+VBEpLjKkqSRObib2ZCZHTKzr8Q8d7OZTZvZZPDnw/k2Mx9l+VBEpLjCkySLliET1k22zG8AR4DXJTw/7u4fXXqTeieaOVOkIj8iUh6DXvw6i0zB3cwuB94H/B7w8Z62qMfK8KGIiCxV1p77HwC/Bby2wz4fMLN3AE8DO9z92egOZrYd2A6wevXqLpsqIjI4RS3tmyR1zN3M3g887+4HO+z2ZWCNu/808HXg7rid3H23u4+6++iKFSsW1WARkX4rcmnfJFluqG4EtpjZPwL3AJvM7AvhHdz9++5+Kvjxs8Bbc22liMgAlXGOTGpwd/ed7n65u68BtgEH3P3G8D5mtjL04xZaN15FRCqhjHNkFl1bxsxuAybcfS/w62a2BTgNvAjcnE/zRET6K25sfdVwk6mYQF7kOTLmHl1Kuj9GR0d9YmJiIL9bRCROtCgYtObCfOCtI3zx4NSC7YPIbzezg+4+mrZfrWaoRrXLdq4d26e1VUUkcWz9gaemSzFxKazyJX+TRL+h23e/gUJ/YCLSO53G1ss2R6a2Pfcy3v0Wkd5KGkN3KN3VfW2DexnvfotIb8XVn2orQ257WG2DuypEikhUuChYnDJd3dc2uKtCpIjEaa/cZgnPl+XqvrbBvSxlO0VkMMp+dV/bbBlQhUgRSXbr5nWxOe9lubqvdXAXEUlS9vUfFNwDZSvnKSK9V+arewV3NKFJRKqntjdUwzShSUSqRsEdTWgSkerRsAyUspyniOSnivfc1HMnfkKT0Rp7L1s9CRHpThmX0MtCwZ2FU46NVqEgqM4HLSLxqnrPTcE90J5yPDLcJLp8SRU+aBGJV9V7bhpzj6jqBy0i87XH2ZPWoiv7PTf13CPKXk9CRNKFx9njlKnMQBIF9whVixSpvrhx9raqFBHUsExE2etJiEi6pGFWAx4a29TfxvSIgnuMcD2J9rjcjvFJBXqRiqjD3BYNy3RQ1fxXkbqrw/Br5uBuZkNmdsjMvhLz3IVmNm5mx8zsETNbk2cjB6Wq+a8idVeHxXq6GZb5DeAI8LqY5z4EvOTubzKzbcCngRtyaN9AKS1SpLrKXM43i0w9dzO7HHgf8CcJu1wL3B08vhd4l5klLUFYGkqLFCm/PYem2LjrAGvH9tWqnEjWYZk/AH4LOJvw/AjwLIC7nwZeBi6N7mRm281swswmpqenF9Hc/uo0LlfXE0akTOp83yw1uJvZ+4Hn3f1gp91iti2Y+OXuu9191N1HV6xY0UUzByNpXA6o7QkjUiZJ980+Nj5Z+U5ZljH3jcAWM7sGWA68zsy+4O43hvY5DlwBHDezC4CLgBdzb+0AxKVFxqVQtW+0VnkMT6RsOt0fq/qKa6k9d3ff6e6Xu/saYBtwIBLYAfYCNwWPrw/2SSrZUEpp05VBN1pFiqI9bJoWhKqc/bboSUxmdhsw4e57gc8Bf25mx2j12Lfl1L7C6DRduU03WkUGL7omcpqqdsq6Cu7u/iDwYPD4k6HtPwQ+mGfDiibtBKjaBAiRssrSEQuraqdMM1Qz6nQCDDcbLG8sY0cNbtKIFF2njljVZ6WGKbhnlJQWeePbV3Pq9FleOjmnzBmRAkjqiLWz3ao8KzVMhcMySqoW2alEQVVPGpEiu3XzugVj7u0eetVnpYYpuHch7sTYMT4Zu29Vb9KIFJ3KdrcouC9RHUqHipRNnXroSTTmvkRxY/FGa+xdN1dFZFDUc1+i8CXg1Mwsxvm6C1WfAScixaWeew62bhjhobFNjAw3F8yIq/IMOBEpLgX3HKn+u4gUhYJ7jlT/XUSKQsE9R3VYl1FEykE3VHOk/FoRKQoF95wpv1ZkMNrrLahj1aLgLiKlFy3zqzRkBXcRKZm4HrpqPC2k4N5D4ZPwomYDM5g5OadLRpFFSuqhJ9Vvr3MasoJ7j0RPwpnZuXPP6ZJRZHGSeuhDZpyJWdmzzmnISoXskbTVYDRzVaR7ST3xM+5YZFvd05AV3Hsky+VgnS8ZRRajU0/c4VyAr/pCHFkouPdIlsvBZWasHdun6pEiGcVNFAxzWoH9obFNtQ7soODeM2knIbQuJbU0n0h2WzeMnFsqL4muiFsU3HskfBIarUW0L351AwOGLDo6qDF4kazCVVjj1PkmapiyZXooabbq2rF9sfurxyGSXae1UiVDz93MlpvZN83sMTN7wsx+N2afm81s2swmgz8f7k1zq0HVI0WWLnp1rJuo82XpuZ8CNrn7K2bWAL5hZl9z94cj+427+0fzb2L1qMchkg/VckqWGtzd3YFXgh8bwZ+FswUkM1WPFJFeyzTmbmZDwEHgTcAfu/sjMbt9wMzeATwN7HD3Z2PeZzuwHWD16tWLbnQVqMchIr2UKVvG3c+4+3rgcuBqM/upyC5fBta4+08DXwfuTnif3e4+6u6jK1asWEq7K2XPoSk27jqgnHcRyU1XqZDuPgM8CLw3sv377n4q+PGzwFtzaV0NtGvQTM3MKuddRHKTJVtmhZkNB4+bwLuBpyL7rAz9uAU4kmcjq6xTqVKRutFVbH6yjLmvBO4Oxt2XAX/l7l8xs9uACXffC/y6mW0BTgMvAjf3qsFVk5Tbrpx3qRstuJGvLNkyjwMbYrZ/MvR4J7Az36bVw6rhJlMxgVw571I3aVexyi7rjsoPDFhcDRrlvEsdJV2tTs3MsmN8UveluqTgPmCaZSfSklbON0z3pdKptkwBKOddJH7mdie6L9WZgruIFEJ45nbcfago3ZfqTMMyIlIYaeV823RfKp2Cu4gUTlyigZbQ646GZUSkcFRcb+kU3Atmz6EpndAiKNFgqRTcC0Qz9EQkLxpzLxDVmRGRvCi4F4jqzIhIXhTcCyQpb9dBFfJEpCsK7gUSl/7VpnoaItIN3VAtkLQZeqqQJ1WgjLD+sNb61/03OjrqExMTA/ndZbB2bF/iKuTNxtC8G6/NxpAmdUgpRDPCABrLjNcsv4CZk3MK9hmY2UF3H03bTz33gkqq8z5klphRo/8QUlTt3nrcOT131nnp5Byg9N88acy9oJLqvJ9JuNJSRo0UVXid4CyU/psP9dwLKjr9+qJmAzMSy6GqQp4UVdz8jTTqrCydeu4F1q6Qd+cN6zl1+uy5S9coVciTIltMoFZnZekU3EugU89HFfKk6DoF6uFmg8aQzdumzko+NCxTAkk9HwMeGtt07melmMmgxZ2DcSsshTO8dN72hoJ7CSRlzqwabs7LQjDOrzWprAPpt7jCdzvGJ3FaPfTljWWx6Y6q/tgbGpYpgaTMmXf+xIp5WQhaRFgGKW74sH1OzszO8cO5s9x5w3oeGtukYN4HqcHdzJab2TfN7DEze8LMfjdmnwvNbNzMjpnZI2a2pheNrautG0a4/bqrGBluYpwfZ3/gqenULARlHUi/pJ1r6mz0V5ZhmVPAJnd/xcwawDfM7Gvu/nBonw8BL7n7m8xsG/Bp4IYetLe24i5dd4xPpr5OWQfSL0nDh2HqbPRPas/dW14JfmwEf6IjANcCdweP7wXeZWaG9FRa4FbWgfRTp8J3beps9E+mMXczGzKzSeB54H53fySyywjwLIC7nwZeBi6NeZ/tZjZhZhPT09NLa7loEWEplPDwIZw/F9vU2eivTNky7n4GWG9mw8CXzOyn3P1boV3ieukL5sm7+25gN7QKhy2ivRKiRYSlaMLDh0pxHKyuUiHdfcbMHgTeC4SD+3HgCuC4mV0AXAS8mFcjJVmWNDL9J5NBUIrjYGXJllkR9NgxsybwbuCpyG57gZuCx9cDB3xQtYRlnnDRJkeLfojURZYx95XAA2b2OPAPtMbcv2Jmt5nZlmCfzwGXmtkx4OPAWG+aK93Sotsi9ZQ6LOPujwMbYrZ/MvT4h8AH822aLEWn+tmglDSRqlP5gQqKW+0mSilpItWm8gMVlFY/WylpItWnnnsFdRpyGVG2jEgtqOdeQUlDLu3Afsf+o6wd28fGXQeUNSNSUeq5V1BS/ex2FclwSVaVBZaswvMl2ss+zpycm/dY8yiKQz33CuqmiqTSIiWL6HyJmdk5Xjo5t+Cx5lEUh3ruFdVNFUmlRUqabha5bncY1HsfLPXcayRpLF5pkZKm2w6AOgyDp+BeI0krOiktUtJ02wFQh2HwFNxrJGksXpfPkiZLrfY2dRiKQWPuNRNXknXH+KSyHGouKRMmel4oW6Y8FNxrKm6leqVF1lP0XJiZnTv3XPS80LlRHhqWqSlVi5S2tEwYnRflpOBeU0nZDMpyqJ8sn7nOi/JRcK+pTmmRew5NsXHXAZUoqIksmS0OOhdKRsG9ppLSItslCrRyU31kzYTRuVAuCu41FU2LHG42WN5Yxhce/j8ai6+ZuHPh4lc3YvfVuVAeypapsXb2Q5bFPTTmWm1xmTBrx/YRtxCyzoVyUM9dMtUN0YzD+lG5inJTcJfUnpjRGm/VDbV6UbmKclNwl449MYNzl+a6oVYvKldRbhpzl8TFPZY3lvHSybl5+4ZvqLWnomvKeXVpVmp5qecuiT20mUhgb2v34JUuKVJcqT13M7sC+DzweuAssNvdPxPZ52eBvwG+G2y6z91vy7ep0ktxPbQ79h9lKmY8fsgsMV1SvbzyCBcL09VX9WQZljkN/Ed3f9TMXgscNLP73f3JyH7/293fn38TZVCShmuSMmuUIlccaeudvnRyLvZ+CqhwXFWkDsu4+3Pu/mjw+AfAEUCffg0kDdeMKEWu0LKsdwosyGHXBKVq6eqGqpmtATYAj8Q8/W/M7DHgBPCb7v7EklsnAxdX/31qZnZerw+UIlck3ax3GqWrr+rIHNzN7DXAF4GPufs/R55+FHiDu79iZtcAe4ArY95jO7AdYPXq1YtutPRfdBarcz5NckTjtQMTN/wSzXDqhq6+qiNTcDezBq3A/hfufl/0+XCwd/evmtl/NbPL3P2FyH67gd0Ao6OjcTObpaDieoPtwP7Q2KbBNKrmOi2ysRi6+qqWLNkyBnwOOOLuv5+wz+uB77m7m9nVtMbyv59rS2WgOtV/z5J1ocyM/ISHx5ZKV1/VlaXnvhH4JeCwmU0G234bWA3g7ncB1wMfMbPTwCywzd3VM6+QVcPN2GByUbORulyflvTLT5Yib1HDWu+0lmxQMXh0dNQnJiYG8rule3FBJWkWK8wfrtm460DsF4OGdLqXdCyT6BhXj5kddPfRtP00Q1Uy6XYWa3gYR0v65aebY6Yx9HpTbRnJrJtZrOGsi6QhHWVmdC/pWML84RcNuYiCuyxJ0izWcI8xbp/GMuPkj06zdmyfAlEXko63qjVKlIK7LEk7oHTKhInuc1Gzwf/70elzY/XhG6xp71V3WY63COiGquQsqaZJOAgl3RQcbjY4dfqseqUxlEoqbVlvqKrnLrnpNKkm3DtPuikYNwmnztUmk8o9KJVUslBwl9yk1TRpB+pONwXjVDGrJu0KB1hQ7iGszl96ko2Cu+QmSxA+MTPLnTes7ypnvmpZNVmucJY3lqVOVKril57kR8FdcpOlR75quJl4UxBIzbypgixXOFlmoFbtS0/ypeAuuYlL0wsLB+pOa3NW/cZhHj3uKn7pSb4U3CU3cSmP3U6qiasfv2N8slKBPssVTlzmkIp8STeUCikDF5fmB/FDNLdfdxVQjt59UvpiWvGvdhDXjFOJkzUVUsFdBqrbgmR55sL3Mnc86d/VbmfSIhtxK1wpz1/CFNylFLqtcpik2+qHacF3qRZTCVPVMyULTWKSUsgrna/b94nLWInLHe92IZK0pe6mZmbZuOtA7PuoeqbkScFdBirp5mLS8EvWXPi0oJwlkGZZZGQxS90lzTBV9UzJk+q5y0DdunkdzcbQvG3NxhCf2vLm2Prxv/Pzb16wv3G+R7zn0NS5gDs1M4tzPpjuOTR17jVJAdPh3Pt06t23peWsJ4m+T6djoZRHWQyNucvALWXoI3oDMvpzWDuFMK5eS1SzMdQxaLffa8f4ZOJ7pDHgu7veN2+bCoRJGt1QlcroNqOmk2jQ7hTg02R97ZAZZ2L+n+lGqSyGltmTykgaHuk2sA+ZLXifpXRtsry22RjiF992hYZbpO90Q1UKL6/p+osZG+9W0sSj0TdcouEW6SsFdym8bjJq4kTH2hf7PmkMmPyd98Q+16mWjkgvKLhL4SWtG/qpLW8GSLxBGi1XkLRP9H3iZPkCUMqiFElqcDezK4DPA68HzgK73f0zkX0M+AxwDXASuNndH82/uVJHaeuGRguNdapR4yQX4Eqq+5L1i0Rj6FIkqdkyZrYSWOnuj5rZa4GDwFZ3fzK0zzXAr9EK7m8DPuPub+v0vsqWkX5YzJT+blMzNYYu/ZRb+QF3fw54Lnj8AzM7AowAT4Z2uxb4vLe+KR42s2EzWxm8VmRgFjOlP8v4uMbQpei6SoU0szXABuCRyFMjwLOhn48H20QGKmkcXOPjUnWZg7uZvQb4IvAxd//n6NMxL1kw3mNm281swswmpqenu2upyCJoSr/UVabgbmYNWoH9L9z9vphdjgNXhH6+HDgR3cndd7v7qLuPrlixYjHtFenK1g0jsTVqNKQiVZclW8aAzwFH3P33E3bbC3zUzO6hdUP1ZY23S1FofFzqKEue+0bgl4DDZjYZbPttYDWAu98FfJVWpswxWqmQv5x/U0VEJKss2TLfIH5MPbyPA7+aV6NERGRpVDhMRKSCFNxFRCpIwV1EpIIU3EVEKmhgKzGZ2TTwT0t8m8uAF3JoTp6K2CZQu7pVxHYVsU2gdnUjjza9wd1TJwoNLLjnwcwmshTQ6acitgnUrm4VsV1FbBOoXd3oZ5s0LCMiUkEK7iIiFVT24L570A2IUcQ2gdrVrSK2q4htArWrG31rU6nH3EVEJF7Ze+4iIhKj8MHdzD5oZk+Y2VkzS7zLbGbvNbOjZnbMzMZC29ea2SNm9m0zGzezV+XQpkvM7P7gPe83s4tj9nmnmU2G/vzQzLYGz/2ZmX039Nz6pbYpa7uC/c6Efvfe0Pbcj1XWdpnZejP7++CzftzMbgg9l9vxSjpPQs9fGPzbjwXHYk3ouZ3B9qNmtnmxbVhkuz5uZk8Gx+ZvzewNoediP88+tetmM5sO/f4Ph567KfjMv21mN/WxTXeG2vO0mc2EnuvJsTKzPzWz583sWwnPm5n9YdDmx83sLaHnenKccPdC/wH+FbAOeBAYTdhnCPgO8EbgVcBjwE8Gz/0VsC14fBfwkRza9F+AseDxGPDplP0vAV4EXh38/GfA9T04VpnaBbySsD33Y5W1XcC/BK4MHq+itbTjcJ7Hq9N5EtrnV4C7gsfbgPHg8U8G+18IrA3eZyin45OlXe8MnT8faber0+fZp3bdDPxRzGsvAZ4J/r44eHxxP9oU2f/XgD/tw7F6B/AW4FsJz18DfI1WEca3A4/08ji5e/F77u5+xN2Ppux2NXDM3Z9x9x8B9wDXmpkBm4B7g/3uBrbm0Kxrg/fK+p7XA19z95M5/O5Oum3XOT08Vpna5e5Pu/u3g8cngOeBvFd0iT1POrT1XuBdwbG5FrjH3U+5+3dplbe+ul/tcvcHQufPw7QWxOm1LMcryWbgfnd/0d1fAu4H3juANv0i8Jc5/N6O3P3vaHXgkpxbZ9rdHwaGzWwlvTtOxQ/uGSWt4XopMOPupyPbl+rHPViMJPj7x1L238bCE+z3gsuzO83swhza1E27lltrucOH20NF9O5YddMuAMzsalq9su+ENudxvLKs9Xtun+BYvEzr2PRyneBu3/tDtHqBbXGfZz/b9YHgs7nXzNorsvXqeGV+32Doai1wILS5V8cqTVK7e3ZeZVmso+fM7OvA62Oe+oS7/02Wt4jZ5h22L6lNWV4fep+VwFXA/tDmncD/pRXAdgP/Cbitj+1a7e4nzOyNwAEzOwxE18WFjMcqx3a1j9efAze5+9lg86KPV/TtY7ZF/425n0sZZH5vM7sRGAV+JrR5wefp7t+Je30P2vVl4C/d/ZSZ3ULrqmdTxtf2qk1t24B73f1MaFuvjlWavp9XhQju7v7uJb5F0hquL9C6/Lkg6IXFru3abZvM7HtmttLdnwuC0fMd3urfAV9y97nQe7eXIDxlZv8D+M0sbcqrXcGwB+7+jJk9CGygtUbuoo5VXu0ys9cB+4D/HFy6tt970ccrIstav+19jpvZBcBFtC63M60T3MN2YWbvpvVl+TPufqq9PeHzzCNgpbbL3b8f+vGzwKdDr/3ZyGsf7EebQrYRWUSoh8cqTVK7e3WcKjMs8w/AldbK9ngVrQ91r7fuWDxAa8wb4CYgy5VAmr3Be2V5zwVjfkGAa49zbwVi77D3ol1mdnF7WMPMLqO1jOKTPTxWWdv1KuBLtMYl/zryXF7HK/Y86dDW64EDwbHZC2yzVjbNWuBK4JuLbEfX7TKzDcB/B7a4+/Oh7bGfZx/btTL04xbgSPB4P/CeoH0XA+9h/tVrz9oUtGsdrRuUfx/a1stjlWYv8O+DrJm3c36d6V4dp1Jky/wCrW+3U8D3gP3B9lXAV0P7XQM8Tetb+BOh7W+k9Z/wGPDXwIU5tOlS4G+Bbwd/XxJsHwX+JLTfGmAKWBZ5/QHgMK0g9QXgNTkdq9R2Af82+N2PBX9/qJfHqot23QjMAZOhP+vzPl5x5wmtIZ4twePlwb/9WHAs3hh67SeC1x0Ffi7n8zytXV8Pzv/2sdmb9nn2qV23A08Ev/8B4CdCr/0PwXE8Bvxyv9oU/PwpYFfkdT07VrQ6cM8F5/BxWvdFbgFuCZ434I+DNh8mlPnXq+OkGaoiIhVUlWEZEREJUXAXEakgBXcRkQpScBcRqSAFdxGRClJwFxGpIAV3EZEKUnAXEamg/w+BVoDAu0VnSwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "t.manual_seed(100) \n",
    "dtype = t.float\n",
    "#生成x坐标数据，x为tenor，需要把x的形状转换为100x1\n",
    "x = t.unsqueeze(torch.linspace(-1, 1, 100), dim=1) \n",
    "#生成y坐标数据，y为tenor，形状为100x1，另加上一些噪音\n",
    "y = 3*x.pow(2) +2+ 0.2*torch.rand(x.size())                 \n",
    " \n",
    "# 画图，把tensor数据转换为numpy数据\n",
    "plt.scatter(x.numpy(), y.numpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（3）初始化权重参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 随机初始化参数，参数w，b为需要学习的，故需requires_grad=True\n",
    "w = t.randn(1,1, dtype=dtype,requires_grad=True)\n",
    "b = t.zeros(1,1, dtype=dtype, requires_grad=True) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（4） 训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr =0.001 # 学习率\n",
    " \n",
    "for ii in range(800):\n",
    "    # 前向传播，并定义损失函数loss\n",
    "    y_pred = x.pow(2).mm(w) + b\n",
    "    loss = 0.5 * (y_pred - y) ** 2\n",
    "    loss = loss.sum()\n",
    "    \n",
    "    # 自动计算梯度，梯度存放在grad属性中\n",
    "    loss.backward()\n",
    "    \n",
    "    # 手动更新参数，需要用torch.no_grad()，使上下文环境中切断自动求导的计算\n",
    "    with t.no_grad():\n",
    "        w -= lr * w.grad\n",
    "        b -= lr * b.grad\n",
    "    \n",
    "    # 梯度清零\n",
    "        w.grad.zero_()\n",
    "        b.grad.zero_()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（5）可视化显示"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4U1X6wPHvW1ooZacgotAWFcUBBAERBRdEGVlGHAVFi4qA/YEr4gKKooOiuCOiIoiOSl0QdVQGF1QYFwQtCooiCtqWCgIWAaFSoH1/f9ykpGnSpiVpkub9PE+eJPee3JzepO+9Oefc94iqYowxJrbEhbsCxhhjqp8Ff2OMiUEW/I0xJgZZ8DfGmBhkwd8YY2KQBX9jjIlBAQV/EWksIvNF5AcRWSMiJ3mtFxGZLiLrROQbEekSmuoaY4wJhvgAyz0KvKuqg0WkNpDktb4f0NZ1OxF40nVvjDEmAlV45i8iDYFTgTkAqrpXVbd7FRsEPK+OZUBjEWkZ9NoaY4wJikDO/I8AtgLPikgnYAVwnaru9ihzOLDB43mea9kmzw2JSAaQAVCvXr2u7dq1O4iqG2NM7FmxYsXvqtr8YLcTSPCPB7oA16jqchF5FJgA3O5RRny8rkzeCFWdBcwC6Natm2ZlZVW+xsYYE8NEJCcY2wmkwzcPyFPV5a7n83EOBt5lWns8bwVsPPjqGWOMCYUKg7+q/gZsEJFjXIv6AN97FXsLuNQ16qcHsENVN2GMMSYiBTra5xog0zXS52fgchEZDaCqM4GFQH9gHVAAXB6CuhpjjAmSgIK/qq4EunktnumxXoGrglgvY0yM2LdvH3l5eezZsyfcVYkoiYmJtGrVioSEhJBsP9Azf2OMCYm8vDwaNGhAWloaIr7GjsQeVSU/P5+8vDzatGkTkvew9A7GmLDas2cPycnJFvg9iAjJyckh/TVkwd8YE3YW+MsK9T6x4G+MMTHIgr8xxgRZ/fr1Adi4cSODBw8ut+y0adMoKCiojmqVYsHfGGMCUFRUVOnXHHbYYcyfP7/cMhb8jTEmTLKzs2nXrh2XXXYZxx13HIMHD6agoIC0tDQmT55Mr169ePXVV1m/fj1nn302Xbt25ZRTTuGHH34A4JdffuGkk07ihBNO4Pbbby+13Q4dOgDOwePGG2+kY8eOHHfccTz22GNMnz6djRs30rt3b3r37l2tf7MN9TTGRI6xY2HlyuBus3NnmDatwmJr165lzpw59OzZkxEjRvDEE08Aznj7Tz/9FIA+ffowc+ZM2rZty/Lly7nyyiv56KOPuO666xgzZgyXXnopjz/+uM/tz5o1i19++YWvv/6a+Ph4tm3bRtOmTXn44YdZvHgxzZo1C97fHAAL/sYYA7Ru3ZqePXsCMGzYMKZPnw7AhRdeCMCuXbtYunQpQ4YMKXlNYWEhAJ999hmvvfYaAJdccgnjx48vs/0PPviA0aNHEx/vhN2mTZuG7o8JgAV/Y0zkCOAMPVS8h1a6n9erVw+A4uJiGjduzEo/v0wqGpqpqhE1pNXa/I0xBsjNzeXzzz8H4KWXXqJXr16l1jds2JA2bdrw6quvAk4wX7VqFQA9e/bk5ZdfBiAzM9Pn9vv27cvMmTPZv38/ANu2bQOgQYMG/Pnnn8H/gypgwd8YY4Bjjz2W5557juOOO45t27YxZsyYMmUyMzOZM2cOnTp1on379rz55psAPProozz++OOccMIJ7Nixw+f2R40aRUpKCscddxydOnXixRdfBCAjI4N+/fpVe4evODnZqp9N5mKMAVizZg3HHntsWOuQnZ3NwIEDWb16dVjr4c3XvhGRFarqnWiz0uzM3xhjYpAFf2NMzEtLS4u4s/5Qs+BvjDExyIK/McbEIAv+xhgTgwK6yEtEsoE/gSJgv3dPs4icDrwJ/OJa9LqqTg5eNY0xxgRTZc78e6tq53KGGH3iWt/ZAr8xJlps3769JI9PLLFmH2NMTPMX/KuSwjmaBBr8FXhfRFaISIafMieJyCoReUdE2gepfsYYU0pmJqSlQVycc+8nm0LAJkyYwPr16+ncuTMnnHACvXv35uKLL6Zjx46lUjIDPPjgg9x5550AftM7R4tAE7v1VNWNInIIsEhEflDVjz3WfwWkquouEekP/Ado670R14EjAyAlJeUgq26MiTWZmZCRAe65T3JynOcA6elV2+bUqVNZvXo1K1euZMmSJQwYMIDVq1fTpk0bsrOz/b4uIyPDZ3rnaBFQ8FfVja77LSLyBtAd+Nhj/U6PxwtF5AkRaaaqv3ttZxYwC5z0DkGovzEmhkyceCDwuxUUOMurGvy9de/enTZt2pRbprz0ztGiwuAvIvWAOFX90/W4LzDZq8yhwGZVVRHpjtOclB+KChtjYldubuWWV4U7hTNAfHw8xcXFJc/37NkDVJzeORoE0ubfAvhURFYBXwD/VdV3RWS0iIx2lRkMrHaVmQ4M1XBljDPG1Fj+WosPphW5vJTKLVq0YMuWLeTn51NYWMiCBQuA8tM7R4sKz/xV9Wegk4/lMz0ezwBmBLdqxhhT2pQppdv8AZKSnOVVlZycTM+ePenQoQN169alRYsWJesSEhKYNGkSJ554Im3atKFdu3Yl6zIzMxkzZgx33303+/btY+jQoXTqVCZURixL6WyMCavKpnTOzHTa+HNznTP+KVOC194faUKZ0tmmcTTGRJX09Job7KuTXeRljDExyIK/MSbsbHxIWaHeJxb8jTFhlZiYSH5+vh0APKgq+fn5JCYmhuw9rM3fGBNWrVq1Ii8vj61bt4a7KhElMTGRVq1ahWz7FvyNMWGVkJBQ4RW1Jvis2ccYY2KQBX9jjIlBFvyNMSYGWfA3xpgYZMHfGGNikAV/Y4yJQRb8jTEmBlnwN8aYKOCeuxi6dg3G9uwiL2OMiXDecxcHQ/jO/P/4AzhwNIuLc+4zM8NWI2OMiUjuuYt7E7wJ4sM3mUtcnN72yC+k35paZlaeWbMsX7cxxrjFxUG87mUtx3AEyahmyUFvMxgVqwpV2Hb95DI/YwoKnKOcMcYYR0oKXMFs2pAdtG0GdOYvItnAn0ARsN97CjEREeBRoD9QAAxX1a/K22ZraaHZ/E57vmMt7UqtE4Hi4sr8GcYYU3O9PGc3vUcdyQ8cw+nsrvYz/96q2tnP3JH9gLauWwbwZEUb+42W/EVdJjOpzLqUlErUyhhjarihv8+gBZuZ3uKeoG0zWM0+g4Dn1bEMaCwiLct7wX7ieZhxXMCrHM+BHwlJSc6EzMYYY4Dt2+G++2DAAF77rSewYkUwNhto8FfgfRFZISIZPtYfDmzweJ7nWlaKiGSISJaIZMFWHuIGttGEe5iICKSmWmevMSZ2+Rz9+OCDzujIu+8O6nsFOs6/p6puFJFDgEUi8oOqfuyx3lf7U5nOBFWdBcwCEOmmO2nEwwkTuHvfeIqXfAynnlrpP8AYY2oC77H8OTkwfvhmBu2fxgIuZMK5nYPaKhLQmb+qbnTdbwHeALp7FckDWns8bwVsrGi7qanQYebVcNhhMGGCMwTIGGNikHssv6fx++8mkT1MYjI5Oc7BAZo1Dcb7VRj8RaSeiDRwPwb6Aqu9ir0FXCqOHsAOVd1U3na7doXsbBg6IgnuuAM+/xzefLNqf4UxxkS53NzSz49gPaOZydOM4ieOBtwHh8PKNKlXRSBn/i2AT0VkFfAF8F9VfVdERovIaFeZhcDPwDpgNnBlpWoxYgQcfTTceivs31+plxpjTE3gPcrxLm5nL7X5F3d4lUyoHYz3qzD4q+rPqtrJdWuvqlNcy2eq6kzXY1XVq1T1SFXtqKpZlapFfDzccw+sWQPPP1+lP8QYY6LZlCnOaEeAznzNxbzENMbyG94DJ/ftDcb7RU5Wz/POg+7dnSagv/4Kd22MMaZapac7ox1TU+FebmGbNGVaws2lyjgHh42/BuP9Iif4i/DBmVMhL4+bkh63JG/GmJiTng7Zzy7mbN6j6QO3Mu3ZRqSmUmooPPy+LRjvFb7Ebt26aVbWgdYh9zCn+QX96MEyjmQ9hUlNbdy/MSZ2FBfDiSfC5s3w44+QmFimiIis8JNpoVIi5szfPcxpPPfRiB3cwr2W5M0YE1vmzYOsLKcDwEfgD6aIOfOPizswzP9ZhjOUlzmGtWyQVEvyZoyp+QoL4dhjoWFD+OorJyj6UOPO/D2HOd3OXYAz1EnVJnkxxsSAJ5+EX36B++/3G/iDKWKCv+cwpzxaM42xDGMunfm65Mo2OwAYY2qk7dvhrrvgrLOgb99qecuICf6ew5wApjKBP2jC/ThDnaz93xhTY02d6iRvu+++anvLiAn+4BrmlO0Ma9pBY+7ids7iA/7Ou0DZy5+NMSba/efRHArvm8bzOoy0fx5fbS0cERX83dzt/09wJes4kge5kVrsJy7OJno3xtQcmZlQeMOtFCNMZEq1NnFHZPB3t//vozY3cz8d+I4RPENRkTMiyPoAjDE1wbwbv+DCohd5iBvIcyVGrq4m7ogZ6uktM9PZAbk5yv84jaNZy1GsYxcNSsqkpjrNRMYYE3VU+STuVNryE235qVRsK28e8xo31NObu/2/WIUbeIgWbGECU0uVsT4AY0zUev11TuFTJjG5VOCH6pnHPGKDv6ctqSeQycWM42FaecwWaRO9G2OiTWYmtE3dy7rB41lTqz1zE0aUWl9d85hHRfCfMgUmJ94LOENAwflZlJNjnb/GmOjhzmH2j9wZHMV6ri96kP3Ek5xMtc9jHrFt/t4yM2Hblbdxzc4pnMTnLKNHybqkJJv43RgT+dLSYHfOVn6iLUs5mQEsBCrXfxmsNv+oCf4A7NrF5sZH80tRCiezFPX44WKdv8aYSBcXB4/rGK5gNh35lh84Fii/g9dbje/w9al+fcYX3UsPlnMRL5VaZZ2/xphId9ah35LBLJ7gypLAD+Hpvww4+ItILRH5WkQW+Fg3XES2ishK121UcKt5wP9SLiGLrtzHeJLYXbLcOn+NMRFNlX83vZ4dNCo1L291dfB6q8yZ/3XAmnLWv6KqnV23pw+yXn7dfU8cE+pMoxW/chMPAOHbecYYE7AFC2j53Yesv/RfNEhNrvYOXm8BBX8RaQUMAEIW1AOVng6Xz+nF20kXcjP3c9LhudbZa4yJaC/9u5Ds865nDe24aMlopkxx2vizs8MXuwI9858G3AyU1yVxvoh8IyLzRaS1rwIikiEiWSKStXXr1srWtUR6Ovxjzf0k1YWlPW+ywG+MiViZmfB9xjTS9q9nLNNYn5sQEelpKgz+IjIQ2KKqK8op9jaQpqrHAR8Az/kqpKqzVLWbqnZr3rx5lSpcIiUFJkxwpj1bsuTgtmWMMSHy6PiNjN93F29yDu/zdyAyUtRXONRTRO4FLgH2A4lAQ+B1VR3mp3wtYJuqNipvu1Ua6untr7+cac8aNYIVKyA+/uC2Z4wxQfa8XMqFvMLf+J6fObJkeWWGd3qqtqGeqnqLqrZS1TRgKPCRd+AXkZYeT8+h/I7h4KlbFx58EL75xuk1McaYMMnMdC7iKpV2fulSLuUFHuKGUoEfwj9Cscrj/EVksoic43p6rYh8JyKrgGuB4cGoXCAy95zP0jq92XbVbXRunR/2djRjTOxxp23IyTmQdn70FUXkD7uWgiaHMa3uraXKR8QIRVUNy61r1656sObOVU1KUm3Pt7qPWvok/6dJSc5yY4ypLqmpqk7YP3C7gqecB3Pn6ty5ThkR5/5gYhSQpUGIwdGV3sFLWppzhAV4hLFcy3S68wW/p3azVA/GmGoTF+eEfLem5PMjR7OaDvRmCSmpwpQpwRnWGZvpHbx4pnS4g3+xhUN4nKvYkFOFXhRjjKki7/b7KUykETu4mhkoEpGzD0Z18Pfc4TtpxE08wIl8wY1NnwlfpYwxMcc99SxAV7LIYBaPcQ2r6VhSJhKGd3qK6uDvucMB5jKMz+J68a+9E2DbtvBVzBgTU9LTnQGHaSnFzOBqtnAId3JnmXKRlIAyqoO/e4enpronQhC23zWDxII/IusQa4yp8dLT4ZdJz9KD5dyX/AA7KXupU7iHd3qK6uAPHnP9uvJkDLi1E1xzDTz1FHzxRbirZ4yJFb//DjffDKecQrdpw0q1SkCEDO/0EPXB31tmJnR4fTIb9VC+PWUML75QFO4qGWNiwfjxsHMnPPkk6cPEq1Ui8mYbrFHB332hxXcbGnI9j9Bx71dkjXwyonrYjTE10GefwTPPwLhx0L49ULZVIpICP0TbNI4V8Bz3D8p7/J0TWc6Zrdby5YZDg/pexhgDwL590LUrbN8Oa9ZAvXohfTsb5+9D6Z504WpmkMgexubdEK4qGWNquunT4dtvnfsQB/5gqlHB37sn/SeOZioTSOdFhrVYZM0/xpigeuPRXHbfdAdvM5C06wZFVYypUcHfe9w/wL3cwlqO5s4tY7j2ir+i6sMxxkSuzLlKnXFXgSpXM4OcXIm4q3jLU6OCv+e4f7dCEhnNTI5iPeP+utuG/xtjguKT61+nf/ECbucucnGCTqRdxVueGtXh68k70dKzDCedTLrwNd9qh5C9rzGmZsvMhKm37OC9DceymRacwJcUcWAiqapO0hIo6/CtgHf7/408yA4a8Wyd/wvtJ2OMqbHcw8n/b8NEWrCZDGaVCvwQWVfxlqfGBn/v9v98mnFr7YfoVrjUZv0yxlTJxInQsWAZV/IEM7iaLE4otT7SruItT40N/mXz/kD8iEv5NLEPO8fczImtfo2ajhljTGTYlLOXpxlFHq24jbtLrYvEq3jLU6NnPE9PP/BBOD/XhPf2PMW3dOTWX6/k4iv+A0jUfFjGmPC6t9FUOuz4jgEsYBcNSpanphJ1E0gFfOYvIrVE5GsRWeBjXR0ReUVE1onIchFJC2Ylg2HiRKcn/meOZBKTGcRb9P9rftT0zBtjwuz777lu193Mq3URCxlQsjiamno8VabZ5zpgjZ91I4E/VPUo4BHgvoOtWLB5Xv07jbFk0ZUZXM2fOZb33xhTgeJiGDWKWo0bEvfYoxGdsC1QAQV/EWkFDACe9lNkEPCc6/F8oI+IyMFXL3g8e+CLiGckc0gmn6fqjSMz08kLFBfn3FtfgDGxzTsmfHn5E/D55zBtGoPHNI/ohG2BCvTMfxpwM+BvjOThwAYAVd0P7ACSvQuJSIaIZIlI1tatW6tQ3arzHv3zDZ14OH48g3c/x6sj3yUnx7kuIBLn2jTGVB/3cE53TJCcXzj2+Qm8w9mkTUyvMbGhwuAvIgOBLaq6orxiPpaVuXpMVWepajdV7da8efNKVPPgeY/+SU6GaQ0n8T3H8ljhFTRgZ0nZaLpKzxgTXO7+QYfyNKMoJo4MZkVdCofyBHLm3xM4R0SygZeBM0RkrleZPKA1gIjEA42AiGtMd+fXfuEF+Osv2LStDpfzLIexkQe4qVTZSJpr0xgTWp7NPAfSwsMVzKYPH3EjD5LnhLgac3JYYfBX1VtUtZWqpgFDgY9UdZhXsbeAy1yPB7vKhCdvRAA8j+xfcCIPM47/YxZn8GFJmWi5Ss8Yc3C8m3ncWpPLg9zIB/RhNleUek1NODms8kVeIjJZRM5xPZ0DJIvIOmAcMCEYlQsV7w9uEpP5kbY8zSjqsYuEBNi1yzqAjYkFpZt53JRZZBBHMVcwG++W7Zpwclip4K+qS1R1oOvxJFV9y/V4j6oOUdWjVLW7qv4cisoGi/cHt4e6jOAZUsnhsTo3IQL5+dYBbEws8HUWP4qnOZv3mMBUcqRNqXXROq7fW41N71AeX3n/v07qxdp+13N54UxO3buo1Lqa0sZnjCnL+2QwlWweZhxL6/RmRtGVvPBCZE/EXlUxGfx95f2ZNQuOfe1u1tCOZxhBQ3aUek1NaOMzxpTleTIoFPMMI1CELVOfgbi4iJ+IvapiMvgDvj/QunWZcOhzHMZGHuH6UuVrQhufMaYsz5PBq3iCM1jM9yMf5tyxaeGuWkjFbPD354IHu/NQ/ARG8CwDcNIYiTht/9b5a0zNlJ4O2R+s47Gk8dCvHz1mjwx3lULOgr+X9HRoPXsSaxKO42lG0ZytJcO/rPPXmBpq/3645BKoXRtmz3bO+Go4C/4+XDS8DseumEsT/uApMvC8WNk6f42pgaZOhWXLYOZMOPzwcNemWljw96djR25jCv/kP1xWkrPOYZ2/xtQgWVnwr3/BRRfBhReGuzbVxoJ/OV5LuZ4lnMZ0riWNX0qWW+evMTVEQYHT3NOiBTz+eLhrU60s+JfjrntqMTrROet/jsuIo6jGXOBhjAHGj4cffoB//xuaNAl3baqVBf9ypKfD7U+nckfy45zKJ0xtPLXGXOBhTKxbfNNCmDGDR7mOtFFnxtxADgv+FUhPh0e2DoOhQ7npzztIP2p5uKtkjAmQv4maXntiM+0fvJxVHMd4psbkSD4L/oEQgSefhFat+POcdNqn/ElcHDRr5twsAZwxkcc7W2dJgJ+rNL3hchqwk4t5kUISgdgbyWfBP1CNG/P+pXNJ2vILN264FlUn+ZslgDMmMvnK1llQAD9eO4Pee97hRh7ke9qXWh9LI/ks+FdCxvO9mMJELuffXMjLZdbH2pmDMZHMVyDvyDfc8sdNLGAAT3BlmfWxNJLPgn8l5ObCZCbxGScziwzaUDZztaWBMCYyeAfyJHbzCheyjaaM4Bm8c/TH2kg+C/6VkJICRcRzMS9SRC1eZigJ7C1TzpqAjAk/79Tt07mWY1jLMOaylUNKla1JqZoDZcG/EtxfplxSGckcuvMlU/DdzmNNQMaEl2e2zot5kZE8wxQmspgzSpUTqVmpmgMl4Zpqt1u3bpqVlRWW9z4YmZlOUM/NhadrX8mIwifpx0LepV+ZsiJOymhjTBitWwdduvDl3uM4qXAJRcSXWp2a6gT/aCEiK1S128Fup8IzfxFJFJEvRGSViHwnIv/yUWa4iGwVkZWu26iDrVik8pwHYMQfD8Fxx5EZdwmHk1embCx1HhkTkfbsgQsugPh48u57kTpJpQN/rLXzewqk2acQOENVOwGdgbNFpIePcq+oamfX7emg1jJS1a0L8+bRoHYh8+KGEs++klWx/KUyJmKMGwdffw3PP88/r0vxOYNfrDX3uFUY/NWxy/U0wXULT1tRJDrmGBKenc3JxZ/xWMOJ9qUyJlK89JJzceZNN8HAgYCfGfxiVEAdviJSS0RWAluARarqK8fB+SLyjYjMF5HWfraTISJZIpK1devWg6h2hBk6FMaMYfTOByh+8+2S9kNfl5UbY6rB2rXOkLuePe0nuB+V6vAVkcbAG8A1qrraY3kysEtVC0VkNHCBqp7hbzsQvR2+fu3Z43zRfv6Z/0z6ivTb2pS6ujApyX4NGFMtdu+GHj3gt9+cJp9WrcJdo6Cqtg5fT6q6HVgCnO21PF9VC11PZwNdD7ZiUScxEV59FYCjJpxPccFfpVbb0E9jgsdfwjZU4f/+D777zllYwwJ/MAUy2qe564wfEakLnAn84FWmpcfTc4A1waxk1DjiCHjhBTrs/ZrHuQrvrpFYyhtiTKj4TdiWCTzxhPNg8mTo2zfcVY1ogZz5twQWi8g3wJc4bf4LRGSyiJzjKnOtaxjoKuBaYHhoqhsFBg5keqPbGMGzjKL0oCcb+mnMwfOXsO21Gz+H668n7/iBtJl9q/W3VcAu8gqBF18o4pDh/TmleAm9+JQsTrA2f2OCJC7OOeP3dAibWUFXCqnDCWTxBwdm5app/3thafM3gbn4klr88fiLbK3Vktc5j66tNteoL58x4eT9CzqBvcxnME3Zxvm8Virwg/W3+WPBP0SGjE6m1Zdv0LpuPllthpA+pGwCOGNM5XknbHuE6zmFTxnJHFbR2edrrL+tLAv+oXT88TBnDnzyiXOloTHmoHkmbBvJHK7iCe7nJl7mIr+vsf62suIrLmIOykUXOWONH3jAORiMHBnuGhkT9dLTcebTPvVKPo7ryy177vVb1lKt+GZn/tXgxY738nFiXwpHXcn5h35mow+MOVh5eXDuudCqFZunvURiUq1Sq8U1T4ulWvHPzvxDLDMTMkbXovael1nOiTy5+Z+cOupLINW+kMZURUEBDBrkXMn7wQcMad+UvfUPpFpPSXHO9O3/q3w21DPE0tKci1AAjmYtyzmRHFK5qPVnfJ9bP6x1MybqFBc7ubTmz4e334YBA8Jdo2pnQz2jhOcogx85hguYR3u+454Nw2ymF2Mq6667nDQq998fk4E/mCz4h5j3KINF9OV6HuFc3mRmkwnW/m9MoF56Ce68Ey67DG64Idy1iXoW/EPMe0wywAyu5nGuZPTOB/h8xCw7ABhTkU8/heHD4dRT4amnDvTomiqz4B9inmOSDxCu41EW0o9pe6/k3XHvA+VkKjQmBrn/H9rKOraddi6b66bS+efXiatbx/4/gsA6fKuRd06SBuzkE04hjWw+ufczLryrg80BYGJaZqYzaicnxzm5b6zb+JyTSCafHixjPUeVlI3V/w/r8I1C3u3/f9KQgSxgT616dLmtH40Lfi213nKSmFjimaoZoLbu4U0GkUY25/KfUoEf7P/jYFnwr0a+2v83J7TmwvoLaVC0nYX0pyE7Sq23nCQmVnimahaKmcswTuFTLuV5PqOXz9fY/0fVWfCvRp7t/yKQnOzc/29HZ87jdf7G97zG+SRwIAmc5SQxseJAIFceZhyDeY1xPMQ8LvT7Gvv/qDoL/tUsPR2ys50h/vXrw15XnP+AsxjJHM7kQ57lcoTiUjlJrDPY1BT+vsvuQD6OhxnLozzCWB7Bf0JEy9lzkFQ1LLeuXbtqrBNRdbqAD9wmcI8q6NMNxuqY0cWamuos9y6blKQ6d264/wJjKmfuXOe76/ldTkhQTU52Hl/Kc6qgrzBEhaKS731qquqYMc69iHMfq99/IEuDEIMt+IeRO7CXvhXrnAbXqYJOSpjiY/2BW2pquP8CYyrH93feuQ3kLd1HLX2fM7U2e2I6wJcnWME/kAncE0XkCxFZ5Zqn918+ytQRkVdEZJ2ILBeRtOD/Rql5fHUAJyUJdR5/mNfrDeNf+yaSwVN+X2+dXSba+PvOnsLHzOMCvqIL5/E6LVPrkJ0de8PP5iSdAAAWhElEQVQ4q1Mgbf6FwBmq2gnoDJwtIj28yowE/lDVo4BHgPuCW82aybsDuCT97CVxDN39DP+lP08yhiHM8/l66+wy0cbXd7YzX/MW55BDKv1ZyC4a2IlNNagw+Lt+aexyPU1w3byvDBsEPOd6PB/oI2LXXwfCswPY80znsNQEhvAqn9GTTNIZwIJSr7POLhONvH/tHsv3vE9fdtKQvrxPPs0AO7GpDgGN9hGRWiKyEtgCLFLV5V5FDgc2AKjqfmAHkOxjOxkikiUiWVu3bj24mtdwU6aAJCUxkAWspDPzGUwfPgRsggoTvTx/7R7Jej6SMymiFn34kA04Ed9ObKpHQMFfVYtUtTPQCuguIh28ivg6yy+TN0JVZ6lqN1Xt1rx588rXNoa4/0mapDaiH++SndCWd+oMQj9bam2hJqqlp0P2p3msS+3DoU0Kybr3A/alti3d9Gnf75Cr1Dh/Vd0OLAHO9lqVB7QGEJF4oBGwLQj1i2nuJqHfNZl2uYtISD0c+vWDL74oVc6uATBR5ddfoXdv+OMPeP99Bk7o4LPp04RWIKN9motIY9fjusCZwA9exd4CLnM9Hgx85BqSZILl0EPhww+heXPo2xdcSfE886GoOvcZGXYAMBFq0yY44wz47Td4913o2jXcNYpZgZz5twQWi8g3wJc4bf4LRGSyiJzjKjMHSBaRdcA4YEJoqhvjWrWCxYv5s3ZTdpx4Fl3lKy67jFKZQMESXpkI9dtvzhn/r786gf+kk8Jdo5hW4QTuqvoNcLyP5ZM8Hu8BhgS3asaXzI9bc9efi3mn+HQWcSZ9i95nBWWzu9pQORNRNm2CPn0gLw/eeQd69gx3jWKe5faJMhMnwto9qfRmMTtoxIf04USWlSlnQ+VMxMjLg9NOc85IFi6EU04Jd40MFvyjjvuMPoc0TuN/bOEQFnEWvfikpIyI0/Zvnb8m7LKznakXN2+G9993HpuIYME/ynie0W8ghdP4H3m04l3O5gw+ROTAbGHW+Wuqk+eos2bNoHuTn8hpcxo7cv7gvrM+IO3ik21EWgSx4B9lvK+Q3MRh9K+7hL2tj2Qh/Rmkb5Qqb52/pjp4jzo7PH8Vb20/hSQKOL34Iya8doKNSIswFvyjjK98QHfPbkGTlUv4ii7MZzCXlmTacFjnrwk1z1m4TmIpSzidfSRwCp+wsux4ETspiQAW/KOQz3xATZsysvUiPuIMnmM41zGtpLx1/ppQc59g9OU9FnEWW2lOLz5lLe0qfI0JDwv+NcjEe+tzQd0FvMZ5TON6pjKeenWLLU+KCbmUFBjGCyxgID/RllP4hFxSK3yNCR8L/jVIejrMmF2Hm1Lm8SRjGM/9fNj6Mu68da91tJmg8JlKRJU3etzHC1zKx5zqGoXWotztWPK2CBCMGWGqcrOZvEKsuFi/HnK3Kui79NUG7LDpH02VzJ2rfqcTrV93v67pe40q6C89hmrblD0q4kzLmJx8YMpFm4IxeAjSTF6iYUrB061bN81y5acxoZGWBr1znmU2V/A9f2MA/yWP1qSmOn0FxlTEPYrHO4UIQD12kUk6g3gLxo2DBx5wfhKYkBKRFapa9rL+SrJPqgbLzYV/czn9eIdUcljOiXRhBbm5lgnUBMZzFI+nlmzkY05lIAu4mhnw0EMW+KOMfVo1mLtD7QPO4mSWspfafMypDKv3hmUCNQHxNSKnEytZzom05Sf+wdvMrHWVnUREIQv+NZjnBWHf054TWc5qOvD8rvMYV3AXnvPt2Lhr44v3iJzBvMpSTgagF5/yDv0pKrKTiGhkwb8G87wgDGCrtOA0/sfzXMJdTGIeF5DE7pLyNu7aeHOfQAjFTOZ2XuUCvuZ4TuBLvqvVqUx5O4mIHhb8azj3BWGpqc7ZWSGJXMZz3MCDnMfrLOVkjmA9YOOuTVnp6fDstB28V/ef3M7dvFx/JLnPfsRveijFxb5fYycR0cGCf4wo/Q8pPMwNDOC/tGYDK+jKoFoL2LXLOoCNl2++4YL7u3HWvoUwfTpDd87mouF1AP8nC3YSER0s+McIX/+Q73E2XVlBbq0j+E/RPxibfxuiRdZ2axxz50KPHrB7NyxZAtdc4ySUcvFOMgh28VY0seAfI/z9o949tw1DDl/K04zkNqawiLM4lE0lbbc2JDQGFRTAFVfAJZdA9+7w1Vc+Z97ylWRw1iybgD1qVHQVGNAaWAysAb4DrvNR5nRgB7DSdZtU0XbtCt/q575S0/sqS/dVm8N5RneRpJtprn15t+SKYM8rOu0K4Rpu9WrV9u21WERnNLxF49lnV+RGGIJ0hW8gwb8l0MX1uAHwI/A3rzKnAwsq88YW/COH+9J9UD2W7/QbOqiCPiA3aW32lAr+4JQ30cf74O+ZciG5abFeX+8p3U1d3SyHaP/49+ygH6GqLfiXeQG8CZzltcyCfxSbO7f0GX4iBTqT/1MF/ZpO+jdWlwoEIuGusaks78/Y89aczfom/1AFfZ8ztQWbfJazg35kCFbwr1Sbv4ikAccDy32sPklEVonIOyLSvjLbNeHlfT1AodRlNDM5hzc5jI2soCvXMQ3BGdtnozmij780DQN5m2/pSF/eZyyP8HfeYzOH+tyGDeGsWQIO/iJSH3gNGKuqO71WfwWkqmon4DHgP362kSEiWSKStXXr1qrW2YSA9/UAAG9zDh35lkWcxTSuZzG96ZC4zkZzRDjvuXSbNXOuvvXUhG08x6W8zTlsoiXdyOJRxlLe+aAd9GuYQH4eAAnAe8C4AMtnA83KK2PNPpHJO2WvcyvW4TyjO6SRFsbX1TubTNM4isrtCPTXuWxCq7zmHfftH7ypGzlU91FLJ3Obz34d75u1+UcOqrHDV4DngWnllDkUStJDdwdy3c/93Sz4RybPzl/v9t7XpufpwrgBqqBL6aEdWeUzKPgKQBY8QsfzQFurlv8A3pJf9RWGqLsvpzNf+S2bkFA6H799dpGjOoN/L5wMYN9wYChnf2A0MNpV5mqcYaCrgGXAyRVt14J/ZCovcDsHhmK9mLm6mea6j1p6Pzdqu9a7Sm2jvAOICa5AzvTj2K9XMkO301D/oo5O5C49KqWw9GifZAv20aLagn+obhb8I1dF1wOAahPydRajVEFzaK36yiuqxcVlytkoodDyd6B133qwVL+gmyroe5ylR/KTHYSjXLCCv13ha8pwd/4WFzv37is2PTv8/qApGcymF5+wK6EpXHgh9O4Nq1ZZzpdq5G8ETks28jyX8DkncxgbuYgX+TvvsSnpKOuwN4CldzCV4CtFxNdJvVg5ZwU8+SR8+y106cIHaaM4MvHXUuVEnBEnliIiuLwPqPXYxR3cyY8czQXM4+G6t3Jy07W8IheRmiqWfsEcEIyfD1W5WbNPdCpvFM+8mfk6p8F1WkiC7qauPlh3ojZgR5lmIM/OXxsVVDXek6rHs1dH84RuooUqaHb3warr1oW7miYEsDZ/E26egTs5WbV2becb1Yb1mslFqqD50lRvYYrWZ2eZzl8bFRQ4f/u6Fvv0Mp7VdRyhCrqszqn6zp3Lwl1dE0IW/E1YBTLKpAtZ+jbO0NCtJOvNTNUG7Cg5W7VRQeXn23EfIH3t63j26qX8W3/kKFXQLLro5c3fLul0NzWXBX8TVhWNMvG8ncBy/S/9VEH/oJHewwTt1mpTzI8KCuQAmpTknOmXPGeXXss0Z4QV6Fd01n/wpkJxzOy3WBes4G8dvqZKAs3zkpwM3yV1ZwAL6UoW79OX8dzHss1pvJQ0kk6sLPOaWBkV5C/fjqeCAsjPh9bkcg+3kEsKjzKWbNLoz3/pwle8zTmAxMx+M8Fhwd9USSCBJikJHn30QNK4r6UrN6fOY8GDa6k1Yjjn73uJlRzP/ziVIcyjNoUxNSqoogOoUMzpLOZVBvMzR3Az9/Mxp3Iyn3EaH/MO/XEuwLcZtEwVBOPnQ1Vu1uwT3Xw1WQSaEsDdzt2UfJ2U9IBmx6Wpgm6hmT7E9SUppGvSqCBf9ffXdNaSX/UWpuhPHKkKmk8TvY+bNYVsS79grM3fhF9VArKvg0b9uvt1SIN3dB6DtZAEVdAVHK838ICeePiGkI0Kqq4Dir/6jxlzYHkj/tDLmaOL6KP7iVMF/YjTNZ0XNJGCkr4Qz5FSJjZZ8DdRqaKO4mZs0et4RJfRXRW0CNEv65ysN3K/tmVt0EYFVecwU39/c7dWm3TZiKf0w7r9dQ/O2M3NjY7SaY1u1yP5KeZHQhnfLPibqORvhI+v25H8pA80ukuz6FKycA3H6DSu1f4s0Pr8WeV6VGaYaaC/ELzH4rubZUqaaijUU/ifTuY2XUZ3LcJZufOQI1THjVP98kvLj2QqZMHfRCV/QTc5ufxsoq3J0auZru/wd91NXVXQQhJ0KT10ZsMbdcnYN1Q3bQo4UJd3EPJ8XaC/EHyVa8h27cMivYM79H3O1D+ppwq6j1r6KSfrbUzW9nyrSXWLy2zProEw/ljwN1GpvGDqL3B7v6YOf+kZfKD3MEE/pldJk4mC/kpLfZsBehcT9SIy9eTEFfry03+WbCeQvPfeB54KDxKFhdqn5Xf6T17TW7lbX2FISWetu+nqKzrrY1ylg3hDG7K9wqBuVz8bf4IV/N0TsFS7bt26aVZWVlje24RXZqYzxj031xkyOmWK72RjnuWaNnWWbdvmTE9YVHSgXB320I0supFFF76iC19xLGuo5ZpzGOCvhofw/a4UsotT+JXD+Z1m/E4z8klmF/X5i7oUkMQ+EkpeU4sikiigLn9Rj900ZZvrVb/Tkk2kkEsKuRzGxlLv9TNtXLXowgq6sowe7KRRuftExMmiWpX9ZGKLiKxQ1W4HvR0L/iYSZWZCRkbpi6CSkpxrBi65xDkXLk9tCjmKdbTjB9qxlvb1s2myawMp5HI4v9KYHVWu25/UZzMtXKE/hRxS+ZGj+YF2/MjR7KKBz9elpjr33vPputdlZ1e5SiaGWPA3NVpamv8gCb7X+ZOa6pw9e37V49nnOpPP58hDd7PjtwKSKCCe/SVlionDWercttGUfJIpJLHSf4/7wAX+D2p2Vm8CEazgHx+MyhgTbP6ufs3NhRdeKBtA/UlIgF27yv5S2E8CW2hB3dQWXDgl8O0FQqR0M5WvJhtrzjHhZukdTEQqbzaw9PQDKSPACbae3M+Tk53H+fm+t+VOieC9PV9SU2Hu3LKT2fgqV1wMv//u3LxnQwP/M6UZU50qDP4i0lpEFovIGhH5TkSu81FGRGS6iKwTkW9EpEtoqmtiha9Zwzzz17gDqKrzSyA11Qn0qanOc1WoXx/27vW9/dTU0k0t7u35CvD+DhLeBx3Lr2OiSkXDgYCWQBfX4wbAj8DfvMr0B97ByTLVA1he0XZtqKepyMGmX6jqhVJVuajLUi6Y6kK4hnqKyJvADFVd5LHsKWCJqr7ker4WOF1VN/nbjnX4mlArr9PYRtaYaBWWDl8RSQOOB5Z7rToc2ODxPM+1rFTwF5EMIMP1tFBEVlfm/cOkGfB7uCsRAKtn2bdqCimpIB7Nm1qck5ObI/L7topeTOTvz2ioI1g9g+2YYGwk4OAvIvWB14CxqrrTe7WPl5T5SaGqs4BZru1lBePoFWpWz+CyegZPNNQRrJ7BJiJBaTIJaLSPiCTgBP5MVX3dR5E8oLXH81bAxoOvnjHGmFAIZLSPAHOANar6sJ9ibwGXukb99AB2lNfeb4wxJrwCafbpCVwCfCsi7glXbwVSAFR1JrAQZ8TPOqAAuDyA7c6qdG3Dw+oZXFbP4ImGOoLVM9iCUs+wpXcwxhgTPnaFrzHGxCAL/sYYE4NCGvxFZIgrJUSxiPgdQiUiZ4vIWld6iAkey9uIyHIR+UlEXhGR2iGqZ1MRWeR6n0Ui0sRHmd4istLjtkdEznWt+7eI/OKxrnO46ukqV+RRl7c8lkfS/uwsIp+7vh/fiMiFHutCtj/9fdc81tdx7Zt1rn2V5rHuFtfytSLy92DVqYr1HCci37v23YcikuqxzufnH6Z6DheRrR71GeWx7jLXd+QnEbkszPV8xKOOP4rIdo911bI/ReQZEdkifq5/cg2o8ZlGp0r7MhiXCfu7AcfiXJCwBOjmp0wtYD1wBFAbWIUrfQQwDxjqejwTGBOiet4PTHA9ngDcV0H5psA2IMn1/N/A4FDuy8rUE9jlZ3nE7E/gaKCt6/FhOBcENg7l/izvu+ZR5kpgpuvxUOAV1+O/ucrXAdq4tlMrRPsvkHr29vj+jXHXs7zPP0z1HI6TEcDX/9DPrvsmrsdNwlVPr/LXAM+EYX+eCnQBVvtZ7zONTlX3ZUjP/FV1jaquraBYd2Cdqv6sqnuBl4FBIiLAGcB8V7nngHNDVNVBru0H+j6DgXdUNUhJgANW2XqWiLT9qao/qupPrscbgS1A8xDVx83nd82rjGfd5wN9XPtuEPCyqhaq6i84I9u6h6ueqrrY4/u3DOfamuoWyP705+/AIlXdpqp/AIuAsyOknhcBL4WoLn6p6sc4J5X+DAKeV8cyoLGItKSK+zIS2vz9pYZIBrar6n6v5aHQQl3XJbjuD6mg/FDKfjmmuH6KPSIidUJRSQKvZ6KIZInIMnfTFBG8P0WkO84Z2XqPxaHYn/6+az7LuPbVDpx9F8hrg6Wy7zUS54zQzdfnHwqB1vN812c5X0TcF4NG5P50NZ+1AT7yWFxd+7Mi/v6OKu3Lg57MRUQ+AA71sWqiqr4ZyCZ8LNNylldJefWs5HZaAh2B9zwW3wL8hhPAZgHjgclhrGeKqm4UkSOAj0TkW8A7JQdEzv58AbhMVd2z2AZtf3q/nY9l3vugWr6PFQj4vURkGNANOM1jcZnPX1XX+3p9NdTzbeAlVS0UkdE4v6rOCPC1wVKZ9xoKzFdVj1miq21/ViSo382DDv6qeuZBbsJfaojfcX7WxLvOwA4qZUR59RSRzSLSUlU3uYLRlnI2dQHwhqru89i2+2rmQhF5FrgxnPV0NaOgqj+LyBKcZHyvEWH7U0QaAv8FbnP9jHVvO2j700sgaUjcZfJEJB5ohPNTvDpTmAT0XiJyJs7B9jRVLXQv9/P5hyJYVVhPVfWcSmc2cJ/Ha0/3eu2SoNfwwHsF+tkNBa7yXFCN+7Mi/v6OKu3LSGj2+RJoK85IlNo4O/8tdXoyFuO0rwNcBgTyS6Iq3nJtP5D3KdMe6Apw7nb1c4FQZSutsJ4i0sTdTCIizXCu0P4+0van67N+A6cN81WvdaHanz6/a+XUfTDwkWvfvQUMFWc0UBugLfBFkOpV6XqKyPHAU8A5qrrFY7nPzz+M9Wzp8fQcYI3r8XtAX1d9mwB9Kf1rulrr6arrMTgdpp97LKvO/VkRf2l0qrYvQ9x7/U+co1IhsBl4z7X8MGChVy/2jzhH04key4/A+QdbB7wK1AlRPZOBD4GfXPdNXcu7AU97lEsDfgXivF7/EfAtTpCaC9QPVz2Bk111WeW6HxmJ+xMYBuwDVnrcOod6f/r6ruE0KZ3jepzo2jfrXPvqCI/XTnS9bi3QL8T/OxXV8wPX/5R7371V0ecfpnreC3znqs9ioJ3Ha0e49vM64PJw1tP1/E5gqtfrqm1/4pxUbnL9X+Th9OWMBka71gvwuOtv+BaPEZRV2ZeW3sEYY2JQJDT7GGOMqWYW/I0xJgZZ8DfGmBhkwd8YY2KQBX9jjIlBFvyNMSYGWfA3xpgY9P9sEVNoxv/r7wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[2.9645]], requires_grad=True) tensor([[2.1146]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "plt.plot(x.numpy(), y_pred.detach().numpy(),'r-',label='predict')#predict\n",
    "plt.scatter(x.numpy(), y.numpy(),color='blue',marker='o',label='true') # true data\n",
    "plt.xlim(-1,1)\n",
    "plt.ylim(2,6)  \n",
    "plt.legend()\n",
    "plt.show()\n",
    "        \n",
    "print(w, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用TensorFlow架构\n",
    "上一节用Pytorch的autograd及Tensor实现了这个任务。这节我们用深度学习的另一个框架TensorFlow实现该回归分析任务，大家可比较一下，使用不同架构之间的一些区别。为便于比较，这里使用TensorFlow的静态图（TensorFlow2.0 新增核心功能Eager Execution，并把Eager Execution变为 TensorFlow 默认的执行模式。这意味着 TensorFlow 如同 PyTorch 那样，由编写静态计算图全面转向了动态计算图）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（1）导入需要的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# -*- coding: utf-8 -*-\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    " \n",
    "#生成训练数据\n",
    "np.random.seed(100) \n",
    "x = np.linspace(-1, 1, 100).reshape(100,1) \n",
    "y = 3*np.power(x, 2) +2+ 0.2*np.random.rand(x.size).reshape(100,1)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）初始化参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建两个占位符，分别用来存放输入数据x和目标值y\n",
    "#运行计算图时，导入数据.\n",
    "x1 = tf.placeholder(tf.float32, shape=(None, 1))\n",
    "y1 = tf.placeholder(tf.float32, shape=(None, 1))\n",
    " \n",
    "# 创建权重变量w和b，并用随机值初始化.\n",
    "# TensorFlow 的变量在整个计算图保存其值.\n",
    "w = tf.Variable(tf.random_uniform([1], 0, 1.0))\n",
    "b = tf.Variable(tf.zeros([1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（3）实现前向传播及损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 前向传播，计算预测值.\n",
    "y_pred = np.power(x,2)*w + b\n",
    " \n",
    "# 计算损失值\n",
    "loss=tf.reduce_mean(tf.square(y-y_pred)) \n",
    " \n",
    "# 计算有关参数w、b关于损失函数的梯度.\n",
    "grad_w, grad_b = tf.gradients(loss, [w, b])\n",
    " \n",
    "#用梯度下降法更新参数. \n",
    "# 执行计算图时给 new_w1 和new_w2 赋值\n",
    "# 对TensorFlow 来说，更新参数是计算图的一部分内容\n",
    "# 而PyTorch，这部分是属于计算图之外.\n",
    "learning_rate = 0.01\n",
    "new_w = w.assign(w - learning_rate * grad_w)\n",
    "new_b = b.assign(b - learning_rate * grad_b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（4）训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "损失值、权重、偏移量分别为8.3873,[0.88361335],[0.05644109]\n",
      "损失值、权重、偏移量分别为0.0728,[2.1262996],[2.388259]\n",
      "损失值、权重、偏移量分别为0.0389,[2.3755665],[2.3252509]\n",
      "损失值、权重、偏移量分别为0.0217,[2.5482006],[2.2615983]\n",
      "损失值、权重、偏移量分别为0.0129,[2.6723857],[2.2155912]\n",
      "损失值、权重、偏移量分别为0.0083,[2.761768],[2.182473]\n",
      "损失值、权重、偏移量分别为0.0059,[2.8261025],[2.1586385]\n",
      "损失值、权重、偏移量分别为0.0047,[2.8724084],[2.1414826]\n",
      "损失值、权重、偏移量分别为0.0040,[2.9057407],[2.129134]\n",
      "损失值、权重、偏移量分别为0.0037,[2.9297287],[2.1202452]\n"
     ]
    }
   ],
   "source": [
    "# 已构建计算图, 接下来创建TensorFlow session，准备执行计算图.\n",
    "with tf.Session() as sess:\n",
    "    # 执行之前需要初始化变量w、b\n",
    "    sess.run(tf.global_variables_initializer())\n",
    " \n",
    " \n",
    "    for step in range(2000):\n",
    "        # 循环执行计算图. 每次需要把x1,y1赋给x和y.\n",
    "        # 每次执行计算图时，需要计算关于new_w和new_b的损失值,\n",
    "        # 返回numpy多维数组\n",
    "        loss_value, v_w, v_b = sess.run([loss, new_w, new_b],\n",
    "                                    feed_dict={x1: x, y1: y})\n",
    "        if  step%200==0:  #每200次打印一次训练结果\n",
    "            print(\"损失值、权重、偏移量分别为{:.4f},{},{}\".format(loss_value,v_w,v_b))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（5）可视化操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x2bf094ece80>]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XlYlGX3wPHvERDBDTXccM1cckkpKs0sl3JrkXYrzbSyXbOyNMvKN8t+vmlvi5mlZVlpuZCaSy5ZaS6B4J7mkgu44IJLIiLevz9mBodhgAFnmIXzuS4uh+e5Z+b4MBzu517FGINSSqnAUsrbASillHI/Te5KKRWANLkrpVQA0uSulFIBSJO7UkoFIE3uSikVgDS5K6VUANLkrpRSAUiTu1JKBaBgb73xJZdcYurVq+ett1dKKb+UkJBw2BgTWVA5ryX3evXqER8f7623V0opvyQiu10pp80ySikVgDS5K6VUANLkrpRSAUiTu1JKBSBN7kopFYA0uSulVADy2lBIpZQqKeISkxm9cCspaenUjAhjcJfGxEZHefQ9NbkrpZQHxSUmM3TmBtIzswBITktn6MwNAB5N8H6Z3L3xV1AppYpi9MKt2YndJj0zi9ELt2pyt+etv4JKKVUUKWnphTruLn7XoZrfX0GllPI1NSPCCnXcXfwuuXvrr6BSShXF4C6NCQsJynEsLCSIwV0ae/R9/a5ZpmZEGMlp6USeOkpWqSCOhlfMPq6UUr7G1lw89qeNBO/exZkGjYqln9Dvau6DuzSmalY6v3/6KI/+OQsonr+CSilVVLHRUfza+ARLPn+SFe1Ci6V/0KXkLiL/iMgGEUkSkVzr9IrFByKyXUTWi8iV7g/VIjY6ilceaMOaRjHcu34xdcsF886dLbQzVSnl28aPh7p1oXXrYnm7wjTLdDDGHM7jXDegofXrWuAT678eERsdBe+9Bt268Wuj46CJXSnly7Ztg6VLYeRICAoquLwbuKtZpgfwlbFYBUSISA03vbZznTtDvXqWv4ZKKeXLJkyA4GDo16/Y3tLV5G6An0UkQUT6OzkfBey1+36f9ZjnlCoFjz8Ov/wCf/3l0bdSSqkiO3MGvvgCYmOhevVie1tXk3tbY8yVWJpfnhaRGxzOi5PnGMcDItJfROJFJD41NbWQoTrRt6/lr+GECRf/Wkop5QnTp8PRo/DEE8X6ti4ld2NMivXfQ8As4BqHIvuA2nbf1wJSnLzOBGNMjDEmJjKywP1dC1atGvs6dOPE+M9p8sJM2o5aSlxi8sW/rlJKucv48dCwIXToUKxvW2ByF5GyIlLe9hjoDGx0KDYbeMg6aqY1cNwYs9/t0TqIS0xmWLW2VEg/yS1/Lc9eikATvFLK2+ISk+n93OewYgUfNuxI3DqPp8QcXBktUw2YJSK28t8aYxaIyBMAxpjxwDygO7AdOA309Uy4OY1euJXkms3YUbkWDybNY0aLTsWyII9SSjljW9QwOS0dAd78LY6MoBAmNbiBM8W8BlaByd0YsxNo6eT4eLvHBnjavaEVLCUtHUSYEt2N15d8RrMD29lU/TJdikApVewcFzUMzzjNnZuWMvfydhwLrwjFXPH0uxmq9mxLDsxo3on04FB6Jc7LcVwppYqL46KGd2xeRrmz6XwdfUv2seKsePp1crctyHOiTDnimt5I7OZfqXDmFMlp6dq5qpQqVjkStzH0WvsTG6o1IKlGo+zDxVnx9OvkHhsdxTt3tiAqIowpV95C2LkM7tq4BEA7V5VSxco+cV+9bxNNDu+21Not/ZXFvgaWXyd3sCT4FUM6kta4OQk1m1iaZoxliH16ZhbPTUvSWrxSyuPsl/btnTiP46FlmdPUMiUoKiKs2NfA8rslf/OSkpbO11fewvtz3+O63ev4o16r7HO6W5NSytNsuWXi9JV03foHM9v04J1erb2Wc/y+5m5TMyKM+Y3bciSsAn3Wzs11XndrUkp5Wmx0FHPC/qL0+XP0nPS2VyuTAZPcB3dpTKmwMKa17MxN29cQdfxQrjI6RFIp5VGZmfDJJ5aFDRs1Kri8BwVMcrd1ri6+8U4AHkyal6uMDpFUSnlUXBykpMAzz3g7ksBJ7mBJ8DNH3c/B9p25f93PhGZmZJ/T3ZqUUh734YdQvz507+7tSAIrudvUfHUwldJP8NCeVQje6alWSpUw69bB77/DU08V24Yc+RFjcq3MWyxiYmJMfHyuHfvcwxho3hzKlIH4+Oxxpkop5TH9+8OUKbBvH1Su7LG3EZEEY0xMQeUCsuaOiKXNa+1aWLXK29EopQJYXGIyXYfHkf7FV8xu3oG43b4xcCMwkztA795QoYKlDUwppTzAtljY9cvnEHYug/HNuvrMzPjATe7lyln2K/zhB0vvtVJKuUlcYjJtRy3luWlJZGScpc/an1hdqxmbq13qM3NqAje5g6VpJivLMu5UKaXcwFZbT7bOm7l5+2pqHz/IFzG3Z5fxhTk1gZ3cGzSAW2+FTz+1bFKrlFIXyXFp34cT5rCvQiSLGrbOPuYLc2oCO7kDDBwIqakwdaq3I1FKBQD7WnmTQ7tos2cDX195C1mlLMMffWVOTeAn944doVkz+N//sleLVEqporKvlfeNn016cChTr+gC+NacmsBP7iIwYAAkJcHy5d6ORinl52xL+1Y6fZzYzcuY2bwDZytE8P59rVgxpKNPJHYoCckdoFcvqFTJUntXSqmLYFvH6omtSwjNymR++3t8prZuL2DWc89XeLhl9tjo0bBrl2XtB6WUKqLYZpGwYT7cfDNTxvbzdjhOlYiae1xiMrESTSbC1N6DfWKCgVLKj33/PezfD4MGeTuSPAV8zd02JjXdlOOnJtdzy5/z6PDd/YD3dkhRSvmfuMRkRi/cSsqx08yfMoKa9RtSoUsXb4eVJ5dr7iISJCKJIpJrmyMReVhEUkUkyfr1qHvDLDr7MakTY2IpfzadHgkLfGIGmVLKP9hPXLp63yaapPzNfy/vSty6/d4OLU+FaZYZCGzJ5/w0Y0wr69fnFxmX29iPSd1QoyFrajXl4YQ5HDh6yotRKaX8iX0l8ZE/4zgaVoFpTdr7dCXRpeQuIrWAWwCfSdqucpwpNjEmltrHD9IzZa2XIlJK+RtbJbHOsf3c/PdqvmnVjYyQUJ9YZiAvrtbc3wdeAs7nU+YuEVkvItNFpLazAiLSX0TiRSQ+NTW1sLEWiW1Mqs2ihteyN6I6z2/8KXvxn/pDfqLtqKXa0aqUcspWSeybMJtzpYL46spbchz3RQUmdxG5FThkjEnIp9gcoJ4x5gpgMTDZWSFjzARjTIwxJiYyMrJIAReWbUxqVEQYAlQoW4bv2txBlXXxfDXmO5LT0jFAclq6zyzVqZTyLYO7NKb6udPcu34Rs5veSGq5yj6zzEBeXKm5twVuF5F/gKlARxGZYl/AGHPEGGPbsPQz4Cq3RnmRYqOjWDGkI2Pva0XGufN82aQjx0PL8uiaWTnK+cpSnUop3xIbHcXEjLWUzTzD59fc4VPLDOSlwKGQxpihwFAAEWkPvGiM6WVfRkRqGGNs3ca3k3/Hq9dkd4qUDmNKdHeeXDWdOsf2s6dSjewyvtyGppTykowMmk3/Erp0YcHEp70djUuKPIlJREaIiG0B4wEisklE1gEDgIfdEZy72SfuyVfeyrlSQfSL/zFHGV9uQ1NKecm338KBA/DCC96OxGWFSu7GmGXGmFutj4cbY2ZbHw81xjQzxrQ0xnQwxvzliWAvln3iPlS+Cj82bc+9GxYRkX4CAMHS9q6dq0opsO649M4Str70Bn9Xv5S4Kpd7OySXlYjlB2wcR858dk0s4ZkZPJg4HwFsCwJr56pSyjZx6bLEFTQ+vIdxV8UydNZGv8kLJSq5O46c+bfh5Ry8rj39EudS+tzZHGW1c1Wpks3WR9d/zQz2l6vC3Mvb+VVeCPi1ZRzFRkfl7OG+VqBjR+7cuJTvWnXNUVY7V5UquVLS0mmx/2/a7l7PyPb9yAwKyT7uD0pUzd2p9u3ZEtWYx9bMpNT5rByntHNVqZKrZkQYj6+ewYnQsjkqfv6SFzS5i3BywCAuPZZC579XZR/29QkKSinPeuPy0nTb9gdTortxKjQc8K+8oMkduOaFRzlVqy4D4mcixvjFBAWllGfdvOAbCA5mQYd7EXxrf1RXlLg2d6eCgig3bAhNn3ySXd3Lw403ejsipZQ3HTwIX3xB0MN9mP32Pd6Opki05m7Tpw9UrQrvvuvtSJRS3vbhh3D2rF9NWnKkyd0mLAwGDoT58yEpydvRKKW8ZO7vf3HyvfdZ0LA1bWcl+824dkea3O099RSULw+jRnk7EqWUF8QlJvPXG6Mpf+ZfPm59r19PaNTkbi8iwpLgf/gB/v7b29EopYrZ/+Zu4KHVs/i9bis21GgI+O+ERk3ujp57DkJC4P/+z9uRKKWK2XXL51D132OMa5OzE9VfJi7Z0+TuqHp16NcPJk+GZP+7FVNKFV5cYjI3jFzE46tnkFijMSvrXJHjvL9MXLKnyd2ZwYPh/HkYM8bbkSilPMy2QFir1T9T5/hBS61dJPu8P01csqfJ3Zn69aFnTxg/Hg4f9nY0SikPGr1wK2fOZvLUyh/YVqUOiy+7Jvucv01csqeTmJyIS0zmh8iOfHP6GybcNZBPOvUh7XQmNSPCGNylsV/+oJVSzqWkpdP571U0ObybAbe9iBFLnVeAFUM6eje4i6A1dwe2W7QVodWY1+g6eq6KI+voMd1EW6kAE5eYTNtRSzHG8Mwf09hVqQY/NWmXfd4f29ntaXJ3kL3PKvDxdfdR4expHlo7N/u8vw6LUkpdYKvEJael035nAi0O7mBc63vIKmXZzMdf29ntaXJ3YD/kaVO1BixpcDWP/Pkj4WcvHNet+JTyb9mVOGN49o+p7KsQSVyzDoB/t7Pb0+TuwPFW7KM291HpzEkeTJyf47g20Sjlv2yVuDZ71nNVyl+Mb30PmUEh2e3s/p7YQZN7Lo77rCZGNeH3uq3o/+dMymSeyVFWm2iU8k+2StyAP6ZysFxlfmhxU47jgUCTuwPHfVYjwkL4smMvIv9N44GkhbnK++PMNaVKKlsnanJaOtfu3UibPRv49Jq7yAguHRDt7PZ0KKQTufZZpTMJv07hidXT+aZVVzJCQrPPBNJfeqUCma0T1TZgYsCKbzlUthLftOpKVAAOc3a55i4iQSKSKCJznZwLFZFpIrJdRFaLSD13BukLTg8dRtV/j/HAugXZxwLtL71Sgcx+JNzVezfSdvd6Pr32Li6JjAiYdnZ7hWmWGQhsyePcI8AxY8xlwFgg4Ha8aPfo3Ry+qjVPr5lJ6LmzAdOjrlRJYd+EOnDFd6SWjeCbVl0DtmnVpeQuIrWAW4DP8yjSA5hsfTwd6CRitzhDgLhk9NtccvIIW+vuC8i/9EoFMlsT6lX7NnP97nV8es2dnAkpE7BNq67W3N8HXgLO53E+CtgLYIw5BxwHqjgWEpH+IhIvIvGpqalFCNfL2reHG26wbOZx5kyBxZVSvsM2Eu655d+SGh7BN626B3TTaoHJXURuBQ4ZYxLyK+bkmMl1wJgJxpgYY0xMZGRkIcL0ESLwxhuQkgKffurtaJRShRAbHcWEeqdptzuJCdfeSeWqlQK6aVWMyZWDcxYQeQfoDZwDygAVgJnGmF52ZRYCbxhjVopIMHAAiDT5vHhMTIyJj493w3/BCzp2hM2bYedOCA/3djRKKVd16AB//QU7dvjt766IJBhjYgoqV2DN3Rgz1BhTyxhTD+gJLLVP7FazgT7Wx3dby+T/V8OfjRgBBw/CuHHejkQp5aqlS2HZMhg61G8Te2EUeRKTiIwQkdut304EqojIduB5YIg7gvNZ118PnTvDu+/CqVPejkYpVRBjYPhwiIqC/v29HU2xKNQkJmPMMmCZ9fFwu+NngHucPytAjRgBrVvDhx9aagJKKZ8Rl5jM6IVbSUlLp2JYCNftiGfcihWM7jGQhluOBGw7uz2doVpEcaVrEdn4WpqNeJt7M5rxVI+rSsQHRilf5zgTNe30Wfovnsy+ClWZ0LA9wTM3AAT876uuLVMEtg/P220eIOLMKW5ZNFVXiFTKR9jPRAW4eftqWu3fxgfX9SQzKKTELPinyb0IbB+eTdUaMLfx9TwSH0fY8aMl4gOjlK+zn3Fa6nwWL/z2NTsqRzGjRSenZQKVJvcisP9gjG33IGGZGTy56ocS8YFRytfZzzi9bctvNDm8mzHX98reZcmxTKDS5F4E9h+MHVVqM6N5Rx5a+xMtRUfOKOUt9sv5ChCcdY7nl3/DpqqXMq9J2+xygTwr1Z4m9yJw3NDjf20fQIxh7LY5XoxKqZLLfk9UsEyPv3fDIuqmHeCTm/oSUTYUIXC20HOFjpYpAtsHwzbUirp12Xtvbxr88BVsGw6NGnk5QqVKFsdO1DKZZxiw4jvW123GR1OGWZYOKWG05l5EsdFRrBjSkV2jbmFwl8YMvLQ7/5YKYcnd/XXUjFLFzLG/q2/CHKqfOspbbXuXyMQOmtwvmu12cGNWGJ9ffQedNvzKNx9N1wSvVDGy7wermH6SJ1dNZ3GDq0lucbUXo/IuTe4Xyf528LNr7uBweEUGLZ7E6AV/eTkypUoO+36wp1b9QLmM03zQqV+J6DjNiyb3i2R/O3gqNJyP2tzHdXvWc1niCi9GpVTJYtvY/kpO8HDCHBZceTP9nritRHSc5kWT+0VyHC/7batu7KlYjWHLv4bzee1topRyt9joKGYeWkRocCm6z/qsRCd20OR+0RyHRZ4NDuGDDn1otH87fPutFyNTqoRZvx6+/BKefhrq1PF2NF6nyf0i2W4HoyLCECAiLISlLTuwoVoDDjzzArNXbgcuTLCoP+Qn2o5aqh2uSrnbSy9BxYowbJi3I/EJmtzdwDYscux9rcg4d56jZ7J4u8MjVD9+iL9ffZtX4zZkT7AwQHJaui40ppQ7LVoECxfCa69B5crejsYnaHJ3I/uRMyvrXsHiBlfz2PJpLPhlY44JFkCJWZlOKY/LyoIXX4T69S1NMgrQ5O5WjhMpRrXvS9nMMzy94juXyiulCicuMZmRPYfC+vUMv/YB4jYf9nZIPkOTuxs5jpzZfkkdprbsTK/EedQ/mrsJpiSsTKeUp8QlJjNi6p/0WzCJpBqN+KpOa23utKPJ3Y0cR84AvH/9g5wJLs0rv0zKcbykrEynlDs4G5AweuFWeq/4gRqnjvBWx0dARJs77WhydyP7kTMAAqSWrcS4Nvdy8/bVtP0nCShZK9MpdbHsV3y0H5CQtWcvj6+ZwdzG1xNfq1l2eW3utNDk7ma2kTNREWEY67FJMT3YW7Eary79nNoVSrNiSEdN7Eq5yHHFR7AMSHj5t68IOp/FqPYP5zinzZ0Wmtw9xL72kBFcmnfa9+Xy1H+4/rfZgI57V8pVzmriV+zfxh2bljLx6lj2RVTPPq7NnRcUmNxFpIyIrBGRdSKySUTedFLmYRFJFZEk69ejngnXfzjWHuY1bsuaWk15afkU5v62Rce9K+WiXDVxY3htyeekhkcwrvW92Bb01ebOnFypuWcAHY0xLYFWQFcRae2k3DRjTCvr1+dujdIP5epcFeH/Oj9OxOnjnHj1DR33rpSLHH+Xbt/yG1cnb+a/N/TmVGg4Bkti1+bOnApM7sbCtjloiPXL5PMURe5lCaIiwuj1zF1Iv37cs2IGDY7szfUc7QhSKjf736Xws+kM/WUS66tfxg8tbsouo787ubm0zZ6IBAEJwGXAx8aY1U6K3SUiNwDbgEHGmNzZq4SJjY7KXZN4+20ypkxl+JLP6HPPmzl2idGOIKWcs/0uTe7wIDVOHeHpHkM4X+pCbV5/d3JzqUPVGJNljGkF1AKuEZHmDkXmAPWMMVcAi4HJzl5HRPqLSLyIxKempl5M3P6ralV2PTOYG3etpdOONdmHtSNIqQLs2EGvFdP5sUUn1ta6PPuw/u44V6jRMsaYNGAZ0NXh+BFjTIb128+Aq/J4/gRjTIwxJiYyMrII4QaGFu+8won6DRnxy+eEnjurHUFKueL55wkKLU3of9/N0dypvzvOFdgsIyKRQKYxJk1EwoCbgHcdytQwxuy3fns7sMXtkQaSkBAqTBhHhZtvZmulzTBElyhVKl/z5sHs2TBqFF07X0XXzt4OyPe50uZeA5hsbXcvBXxvjJkrIiOAeGPMbGCAiNwOnAOOAg97KuCAcdNNcPfdMHIkPPgg1Kvn7YiU8im2JQYOHz7Oki+eoWK9BpQfNMjbYfkNV0bLrDfGRBtjrjDGNDfGjLAeH25N7BhjhhpjmhljWhpjOhhjdHdoV4wZY+lQ1Q+sUjnYLznQf/UMah1NYeB1jxC3qYT21RWBzlD1ptq1YfhwiIuz3HaiM1eVggtLDtRKO8DTq35gbpN2LK19hc4FKQRN7t42aBA0bgwDBjB71Q6duaoUF8atv77kM7KkFG91eCTHcVUwTe7eVro0fPwx7NjB4WFv6sxVpbCMW7/p79XcvH01/2t7PwcqXJJ9XLlGk7sPiKvchJ9bduLBX7/j0iP7cp3X2ooqaYa0q8WIxeP565K6TIrpAeh49sLS5O5lto6jV9o9zJngUEb+/DGYnKs7aG1FlTS3zZpAzROpjL37BbKCgnU8exG4tPyA8hxbx1F62Uq82/5h3l74MXduWsrM5p0Ara2oEigpCd5/Hx57jE8/edbb0fgtrbl7mX2Ty3ctu5BQswnDlk4kIv2E1lZUyZOVBY8/DlWqwKhR3o7Gr2ly9zL7JhcjpXil6zNUyPiXkSsm6xKmquQZNw7WrLHMAalc2dvR+DVN7l7muFb11sh6TGxzN7ckLITFi70YmVLFbM8eGDoUunSBBx7wdjR+T5O7lzlb973mf0dCo0bQvz+cPu3tEJXyqLjEZNq+s4SlHe4i/ew5ul/+APWHztNJfBdJO1R9gNN13ydMgPbt4fXXYfRor8SllKfZRovdvG4pHXfGM6LjY2wOrQJcmMQHaPNkEWjN3VfdeKOl5j5mDMTHezsapTxi9MKtlDl+lNcXf0pSjUZ8edWtOc7rJL6i0+Tuw356cCCp5SrxV7e7ufGthdm3qLr+jAoUKWnpvLF4AuUzTvNytwE5dleyL6MKT5tlfFRcYjJDF+2hdedn+GL6m9w5/0uGZvQhfvdRZiQkZy9ToLeuyp/1TE6gx5Zfee/6B9kaWc9pGZ3EVzSa3H2UbXLTLw2uZkbzjjy98nt+btSGKQ5rz8CFW1dN7sof2NZpP73/IItmvc/mapfySet7nJbVSXxFp80yPsr+VnREx8c4Gl6R0fPeJyQrs8DySvkq+3XaX1vyGRXTT/DyLYMoX94yWiwiLIRK4SG6hZ4baM3dR9WMCCPZmrCPh5XnlS7P8PnM//DMH98ztt2DTssr5etsd6Q3/72KOzf9wvtt72dDZH2iSgeTOFz3znMnrbn7KMfJTYsbXsuMZh14euU0rti/LUdZvXVV/iIlLZ0q/6bxzoIP2VT1Uj5uc2/2ceVemtx9lP3kJps3b3qcQ+UqM3buGEIzMwDLbWyZkFIMmpakI2eUz6tZsQwjf/6Y8hn/MujW58kMCrEc1ztPt9Pk7sNio6NYMaQj79/XirCQIE6UKcfg7s/R4Og+hi3/il6t65Bx7jzHTmfqzk3KL3xwbiNdt63kvXa92WYdHaN3np6hyd0P2Nfi/6jXiultYnlozY8c/+ln3blJ+Y+9e7nqvdc53Ooa5t98v3aaepgYh40hiktMTIyJ15mXRXP6NERHsz/lCF37fcTxsPI5Tguwa9Qt3olNKWeysqBTJ0hIsKzX3qCBtyPyWyKSYIyJKaic1tz9UXg4fPstkafTGLlQd25SfuC//4Vff4UPPtDEXkwKTO4iUkZE1ojIOhHZJCJvOikTKiLTRGS7iKwWkXqeCFbZueoqtj49mFu3LueujUuzD2v7pfI5CQmcf/VVlja/gfp/RWrHfzFxZZx7BtDRGHNKREKA5SIy3xizyq7MI8AxY8xlItITeBe4zwPxKjvNxr7F4T+W8Z/F44mv1ZRz9S9lcJfG2n6pvMY2+zQlLZ2KYSGEZZ5hyidPEV6mIoM6PokR0SUzikmBNXdjccr6bYj1y7Ghvgcw2fp4OtBJRMRtUSrngoK4ZNb3hIeH8mviBFY8f73+siivsZ99aoC09Eyem/0R9Y+m8Pytz+foG9KOf89zqc1dRIJEJAk4BCwyxqx2KBIF7AUwxpwDjgNV3BmoykPt2vD55/DnnzBsmLejUSWYbfapze2bl3HfhkWMa3MPK+u2zFVeJy55lkvJ3RiTZYxpBdQCrhGR5g5FnNXScw3DEZH+IhIvIvGpqamFj1Y5d9dd8OSTlk6r+fO9HY0qoeyTdd1jKby98GPioy5n7PW5l8sA7fj3tEKNljHGpAHLgK4Op/YBtQFEJBioCBx18vwJxpgYY0xMZGRkkQJWeRgzBq64Ah56CJK1s0oVP1uyLn0uk49+fJcsKcWA2weT5WSNdu3497wCO1RFJBLINMakiUgYcBOWDlN7s4E+wErgbmCp8dYA+pKqTBkWv/4BbXt2ZX2brgx+fAzPd2+qbfDKIxw7TkXg2OlMBBi6bBItDu7g8TteIaVC1eznCJbb+aiIMO34LwaujJapAUwWkSAsNf3vjTFzRWQEEG+MmQ1MBL4Wke1Yauw9PRaxciouMZmhiel07fwUY38aw/1zJjD0zCOAjkhQ7mXrOLW1r6elX1iG+pYtv9E3YQ4TY3qwuuWNVBJIO51JTU3oxa7A5G6MWQ9EOzk+3O7xGcD5avuqWNg6s2Y170hM8maeXD2dhKjLGV22tP5CKbdy7Di1ufTIPkYt+JCEmk34qseTJL2qS/h6k85QDRA5Nvfo1J8N1Rrw3k9jKPXPLi9GpQKJbe/eZCejXMpknmFc3DtkBIXwTI+X2XPqnBciVPY0uQcI+5EHGcGleTJ2KACfzxkF6TrkTF0c+zHsuRjDOws+otHhPTx324vsrxCpI2F8gCb3AOG4uce+iOq8HPsSjfbvgP79c60/o1Rh5NUUA9A3YTZ3bF7Ge+168Xv9K3UkjI/Q5B4g7JcFtu1FuboTT5EAAAAWoUlEQVRpa8a2fQCmTGH9y//xdojKj+U14aj1nvUMWzqRJZe35ZM29+gSvj5E91ANILHRUcRGR+UYzfDhdffR4sB2Ovz3Te7bV5p9V1yjoxZUodnv6WtT40Qqn8z+P4IbN6LTqnnsrFDBS9EpZ7TmHoDsb6GNlOL5W59nd6WajIt7B9n9j+7WpArNsdkv7OwZJs56i3Kcg1mzQBO7z9HkHoAcb6FPhpbl0bteI/h8Fp/N+A+lTp3URZtUodg3+5Uy5xm36H9cfnAnId9PgyZNvB2eckKTewByNlJhV+Uonu4xhIaH9/D+3PfYf+xfL0Sm/JltT9+dYQl02Pg7Mno0dO/u7bBUHjS5ByDHW2ib5fWjGdHpMW7evpo3V3/rhciU35s2Dd58E/r2heef93Y0Kh/aoRqAbJ2loxduJTktPXtND4CvrryVpkf30vvXqfBZR3jsMSDnWiE6VVw5tWIF9OkD118Pn3wCumWDT9PkHqBsI2fAIXFXCids/DgY/oRlmeC6dYmLbJZjrRDdKUflsn079OgBdepAXByEhno7IlUA8dbijTExMSY+Pt4r762AEyegXTvYtYve/cbwe5kauYpERYSxYkhHLwSnfMqRI9CmDRw9CitXQsOG3o6oRBORBGNMTEHltM29hLGtD1L/7d+J7f4K6WXC+b+JQ6hxIvfmKbpTjuL0abj9dti921Jj18TuNzS5lyCOe1wmmXLceeurlD2bzlffD6di+skc5XV9kBLu3Dno2dNSW58yxdLWrvyGJvcSxNn6IFuq1uexu16jTtoBJk1/kzKZZwDdKafEMwYefxzmzOG/tz1L/YRw2o5aqpPf/Igm9xIkr2aW1XVaMPC2F4lO2crHP75LnfIhuj5ISTdsGEyaxCfX389Hl3fGcKGjXRO8f9DkXoLk18yyoHFbXu3yFJ12/MlvG78g9orqxRiZ8iWbBgyFd97h25Zdefe6B3KcS8/M0tnNfkKTewmS1+Qmm29bdePjbv1h6lR44gldJrgEWjdkJM0+HEVc0xt5tfOTTseya0e7f9Bx7iVIfpObwNLOHjVyOFxVFd56C8qWJe6hFxn98zad3FQSTJ5My3dfZdFl1/Ji90GcL+W8IqAd7f5Bk3sJk+fkJvvE3WoEnDoF77/P0dV7Sb6hL4jo5KYAlvCf/xH9+iBW1G3FMz1e5lyQ89SgHe3+Q5N7CWaf6HMQgTFjmLFqF/1WzSTzPLzT3pLgbW2umtwDR/xbH3Dl8EGsrNuCx+56lYzg0k7LRemdm1/R5K6cE+HFGx7l34xzPL5mJgYY1f5CDb7tqKX6ix4IvvmG6OGDWFWnBY/cNZwzIWVyFQkLCdLRU35Ik7vKU81K4Qy/+QkEwxNrZhKalcmITo9hpJQ20QSCiRPhscdYU7t5nolda+v+q8DkLiK1ga+A6sB5YIIx5n8OZdoDPwK7rIdmGmNGuDdUVVxsbfHJaemICK/d/CQZQSE8Gv8jYZkZvNLlac6XCtImGn/2wQcwcCArG11N31tfzjOx69pC/suVmvs54AVjzFoRKQ8kiMgiY8xmh3K/G2NudX+IqjjZ778KltE0IsJbHR/l39JhDPxjKuGZZ3jhlkFkBoXosDg/E7d2HweHvs7jP09iQaM2DLjtJc4Gh+Qqpx2n/q/A5G6M2Q/stz4+KSJbgCjAMbmrAOBsiQIDBJUqxdh2vThdugxDl31JRPpJnrjjFSpVreydQFUuBa3JH5ewlxNPPsvjf/7IzGYdGNz9ObKcDHfUppjAUKg2dxGpB0QDq52cbiMi64AU4EVjzKaLjk4Vu7xq4lnGEBYSxKfX3s3RsAq8s+Ajpk4bxr6vfyjmCJUzjndcufpEzp6lbL8+xK7/hc9jejCy4yMYyT2HUUCbYgKEyzNURaQcMAN4zhhzwuH0WqCuMaYl8CEQl8dr9BeReBGJT03NvcSs8r68JqhERYRlb5A8/YrOvNL7TZod2U33J+62bOSgvMK2hPNz05Jy3XGlZ2bx3LQk2g2ZyerLr+Xm9b/wdvu+vNXxUaeJHXSCUiBxabMOEQkB5gILjTFjXCj/DxBjjDmcVxndrMM3OdYAIZ+hcH/8YVnrGyxrfTssCatb93mWs5+Vo1ppB5g0fQT1jqUwpNuzzGzeKc+yOuTRP7i6WUeByV1EBJgMHDXGPJdHmerAQWOMEZFrgOlYavJ5vrgmd99VqKS8fTsnb+pC6L69vNRtAH+27Z7dEeeYeGzLHWibrnu0HbWU5Hw6tKOT/+Kzmf8h+HwWT9zxCqvqXJGrjP5M/I+ryd2VNve2QG9gg4gkWY+9AtQBMMaMB+4GnhSRc0A60DO/xK58W54zV+3YD5eM6DGST+Le5v257/FJ6m6GnXyY0qEhTjtmQfdodZf8Rirds34Rb/08jv3lL6Hf3a+zs0qtXGU0oQc23UNVFZqz5oCQrEyGL/mM3onz+LX+lTx7+0ucKFMu39fRcdQXx1nNPTjrHMN+mUjfhDksr9uSZ3q8TFpYhVzP1Wvvv3QPVeUxzoZLZgaF8FrnpxjS5Rna7F7P7MmDaHpwZ76vo2PkL47jEs6Rp47y3fev0jdhDpNa30mfe0c4Tew6hr1k0OSuCi2/pDy1VVd63v8O4VlnmfX1CzyQND/PdeF1ZMbFiY2Oyh7B1PafJBZOHkD0oR3w9ddUHvcB1SuXQ4CIsBAqhYcgXBj1pE0xgU/XllGFVjMiLN+OvC31mxMft5Srhg3g7YUfc+2ejbza5WlOhoZnl7GvPeqomqKLbVGN2LgJ8P1/oEkTmD4dmjYlFu3PKOm05q4KzdmOTrb9emw1w+4dr6DaiqXwn//QY+vvrPr+Bbqlbc9Ve7S13yenpes+nfmwjWevP+SnCxtV79gBN9wAI0ZA797w55/QtKm3Q1U+QjtUVZHkVdt2evzMHujVC/75B4YOheHDobRlzfC8hvNph98FuTqwjeGBzUt5c+kEQkKC4ZNP4P77vRukKjbuHAqpVC7OhkvmOQX+zhbEJiXBwIEwciTMng2TJkFMTJ7t99rZeoF9B3bNE4d4e8HHtN+VwNr6Lbly2WyoU8fLESpfpM0yym2cjaKxLQtM+fKWhD57Nhw5AtdeCy+9RP3w3Bswg3a2woWmmOS0dMScp1fiPH6e+DRX79vE6zc9zj13j9DErvKkNXflNi7Vwm+7Ddq1g8GDYfRo5tb4jsHX9+On+ldbtvfjQmdrSe5otb8Lan5gO2/9PI5W+7fxe91WDO32LPsqViNK/wCqfGibu3KbQref//YbPPUUbNrEisbX8toN/TgaVQ8ROHY6M3tqvE1JWvuk7ailpKccYNDyb3gwcT5HylbkrQ6P8GPT9iCiywaUYDqJSRU7Z6No8p0wc8MNkJgIY8bQNmUzi794msFzP0IOW9abc6x2ZDfxBLr0dG5f8BXLPn2M+5MW8OVVt9HxsU/5sVmHHIkddHSRypvW3JVbFbkp5eBBZvboz+1r5pIeEsrEq2OZFNPD6RIGgVZbtV2z1CMn6P3XMh79/VtqHE9l0WXXMqr9w+yoUju7bJAIWU5+Z3V0UcnhtlUhPUWTe8nm7I/AoGlJXHpkL4N/+4qu21ZyPLQsE6+O5curbsuV5D3ZRFOcbf1xickM/z6B7omLeGbl99Q6cYjEGo35vxv7sLJuzlUcw0KC8lzeV4Bdo27xSIzKt2hyVz4rrzXjy4SU4tjpTACaHdzBgBXf0eXvVfwbUoZpV3Rm0tU92FexWo7Xcnct3llsHmvfPnqU8Q+8xJ3LZ1L132Mk1WjE2Osf5Nf6V2Z3LtvY3tu2EqcjrbmXHJrclc/Kq+M1IiyEjHPncyTWpod28siaWdy+5TdKGcOSy67h25Zd+a1+NOet+38WVIt3pSZuv4RxftxyxxAfD59+Ct9+C6dP82v9K/ns6jtYXq9VrqQOOWvlhdpMRQUkncSkfFZeQyaPp2cy9r5WORJx//vuZPTCZozes5c+a+dyz4ZFdP57FfsqVGVG80782PRGdlapxeiFW50mtwL3FnVSJj+2Tt1CJ9IDB2DaNPjqK1i7FsLDoWdPHirXmt/Caub7VPsx/7b3LalDRJXrNLmrYpfXwmM1I8Ly3Chk6MyzvNv+Yca0e5DO21Zx/7oFPPvHVAb+8R3rq1/G/MZt4bZqlrVVRPKtiTsmaGeTr/KTkpbuWrv8vn0wZw7MmgVLlsD58xAdDR99ZFmOoWJF7kxM5s98/rA4G23kymYqSmlyV8VucJfGTpsW8hoyaZ+Ek9Pgp8vb8dPl7ah68gi3/fU7t235lZd/nQzNJ0P9+uyMacevWVFk1GwKZSs5fU37u4fCLnVQMSzE6d1A8KmT3HpqFyxbBosWWYZ5Alx2GVv7Pcub5VuysnRVap4MY/DOU8RGV8xVE68YFoIIpJ3O1Fq5uija5q68oqgjUvJqcx57/SV0/ScB5szh9KKlhJ+1JOx/ImqwofplbKh+GX9F1mNnlVokV4ikZqWy2R2Q+e1F6mwiVZlgITg1lUuP7qPR4d20OLCdFge20/DwHoLNeTKDgkms0Zg/m19H40cf4FT9yxg6a2POnatKCeXKBGsSV4WmHaoqYBX0h+Gyl2bT7MB2Wu/dQKuUbbQ4sJ1aJw5lnz8TXJrMWrUpX78O1KjBDlOGRXtOkxZchozgEAxCSFApejS9hOB/T5G0eS/Bx9OonXGcJuYksmcvFc6ezn69I2EV2FC9IetqNGR17easjWrCmZAyQO4/DnnRTlHlKk3uqsTJr5298unjNDy8h+gzqdxV7l8anjkKKSmWryNHMKdOIefPO3/hcuUgIoKjEZew8Xw4O8tUZlflKHZWjmJHlVqklI90OsqlsHQ4o3KFjpZRJYJ9Qs+vlpxesTL3970RgIdt7dvN7Nq3K5ZhyI11uK1JlQtPCgmBsmWhVKlCjagpKl3mWLmTJnfltxwTbl6J3TYBCMhRPi09M7tM8vEzvDR/B1lh4U6bRgo7oqYodJlj5U66cJjyW64kXAFWDOlIbHRUgeXzW5gsv1q142JpBYkICyEkKGczTr4LrClVBAUmdxGpLSK/iMgWEdkkIgOdlBER+UBEtovIehG50jPhKnWBK80Y9rVhV8rnVSavWrVtP9ioiDAES+KuFB4CXNhX1iYsJIj372tF0uudGX13y+zn2O8pq5S7uNIscw54wRizVkTKAwkissgYs9muTDegofXrWuAT679KeUxek6FsHGvDBZW3lXEmv7H5eU0qym9Uj05EUp5WYHI3xuwH9lsfnxSRLUAUYJ/cewBfGcvQm1UiEiEiNazPVcojnCXc/Bb5clbenqsTqVwdm68JXHlToTpURaQeEA2sdjgVBey1+36f9Zgmd+UxhU24FzsbVJO18icuJ3cRKQfMAJ4zxpxwPO3kKbkGL4hIf6A/QB3d2Fe5QWETriZoVVK4NFpGREKwJPZvjDEznRTZB9S2+74WkOJYyBgzwRgTY4yJiYyMLEq8SimlXODKaBkBJgJbjDFj8ig2G3jIOmqmNXBc29uVUsp7XGmWaQv0BjaISJL12CtAHQBjzHhgHtAd2A6cBvq6P1SllFKucmW0zHKct6nblzHA0+4KSiml1MXRGapKKRWANLkrpVQA8tqSvyKSCuy+yJe5BDjshnDcyRdjAo2rsHwxLl+MCTSuwnBHTHWNMQUON/RacncHEYl3ZV3j4uSLMYHGVVi+GJcvxgQaV2EUZ0zaLKOUUgFIk7tSSgUgf0/uE7wdgBO+GBNoXIXli3H5YkygcRVGscXk123uSimlnPP3mrtSSiknfD65i8g91h2gzotInr3MItJVRLZad4MaYne8voisFpG/RWSaiJR2Q0yVRWSR9TUXiUglJ2U6iEiS3dcZEYm1nvtSRHbZnWt1sTG5Gpe1XJbde8+2O+72a+VqXCLSSkRWWn/W60XkPrtzbrteeX1O7M6HWv/v263Xop7duaHW41tFpEtRYyhiXM+LyGbrtVkiInXtzjn9eRZTXA+LSKrd+z9qd66P9Wf+t4j0KcaYxtrFs01E0uzOeeRaicgkETkkIhvzOC+Sx251nrpOGGN8+gu4HGgMLANi8igTBOwALgVKA+uAptZz3wM9rY/HA0+6Iab/A4ZYHw8B3i2gfGXgKBBu/f5L4G4PXCuX4gJO5XHc7dfK1biARkBD6+OaWPYCiHDn9crvc2JX5ilgvPVxT2Ca9XFTa/lQoL71dYLcdH1ciauD3efnSVtc+f08iymuh4GPnDy3MrDT+m8l6+NKxRGTQ/lngUnFcK1uAK4ENuZxvjswH8tSLq2B1Z68TsYY36+5G2O2GGOc71p8wTXAdmPMTmPMWWAq0ENEBOgITLeWmwzEuiGsHtbXcvU17wbmG2NOu+G981PYuLJ58Fq5FJcxZpsx5m/r4xTgEODudaGdfk7yiXU60Ml6bXoAU40xGcaYXVgWybumuOIyxvxi9/lZhWVZbU9z5XrlpQuwyBhz1BhzDFgEdPVCTPcD37nhffNljPkNSwUuL9m71RljVgERIlIDz10n30/uLsprJ6gqQJox5pzD8YtVzViXNLb+W7WA8j3J/QEbab09GysioW6IqTBxlRGReBFZZWsqwnPXqjBxASAi12Cple2wO+yO65XX58RpGeu1OI7l2rjy3KIq7Gs/gqUWaOPs51mccd1l/dlMFxHbvg6eul4uv6616ao+sNTusKeuVUHyittjn6tCbbPnKSKyGKju5NQwY8yPrryEk2Mmn+MXFZMrz7d7nRpAC2Ch3eGhwAEsCWwC8DIwohjjqmOMSRGRS4GlIrIBcNxdC1y8Vm6My3a9vgb6GGPOWw8X+Xo5vryTY47/R7d/llzg8muLSC8gBrjR7nCun6cxZoez53sgrjnAd8aYDBF5AstdT0cXn+upmGx6AtONMfab5nrqWhWk2D9XPpHcjTE3XeRL5LUT1GEstz/B1lqY0x2iChuTiBwU6wbg1mR0KJ+XuheYZYzJtHtt20YmGSLyBfCiKzG5Ky5rswfGmJ0isgzLvrgzKOK1cldcIlIB+Al41XrranvtIl8vB67sGGYrs09EgoGKWG63XdptzINxISI3YfljeaMxJsN2PI+fpzsSVoFxGWOO2H37GfCu3XPbOzx3WXHEZKcnDkuRe/BaFSSvuD11nQKmWeZPoKFYRnuUxvJDnW0sPRa/YGnzBugDuHInUJDZ1tdy5TVztflZE5ytnTsWcNrD7om4RKSSrVlDRC7BshnLZg9eK1fjKg3MwtIu+YPDOXddL6efk3xivRtYar02s4GeYhlNUx9oCKwpYhyFjktEooFPgduNMYfsjjv9eRZjXDXsvr0d2GJ9vBDobI2vEtCZnHevHovJGldjLB2UK+2OefJaFSSv3eo8dZ38YrTMHVj+umUAB4GF1uM1gXl25boD27D8FR5md/xSLL+E24EfgFA3xFQFWAL8bf23svV4DPC5Xbl6QDJQyuH5S4ENWJLUFKCcm65VgXEB11nfe53130c8ea0KEVcvIBNIsvtq5e7r5exzgqWJ53br4zLW//t267W41O65w6zP2wp0c/PnvKC4Fls//7ZrM7ugn2cxxfUOsMn6/r8ATeye2896HbcDfYsrJuv3bwCjHJ7nsWuFpQK33/oZ3oelX+QJ4AnreQE+tsa8AbuRf566TjpDVSmlAlCgNMsopZSyo8ldKaUCkCZ3pZQKQJrclVIqAGlyV0qpAKTJXSmlApAmd6WUCkCa3JVSKgD9P6Kt8eFEPOeFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化结果            \n",
    "plt.figure() \n",
    "plt.scatter(x,y)\n",
    "plt.plot (x, v_b + v_w*x**2,'r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow使用静态图，其特点是先构造图形(如果不显式说明，TensorFlow会自动构建一个缺省图形)，然后启动session，开始执行相关程序，这个时候程序才开始运行，前面都是铺垫，所以也没有运行结果。而Pytorch的动态图，动态最关键一点就是它是交互式的，而且执行每个命令马上就可看到结果，这对训练、发现问题、纠正问题非常方便，其构图是一个叠加过程或动态过程，期间我们可以随时添加内容。这些特征对于训练和调式过程无疑是非常有帮助的，这或许也是Pytorch为何在高校、科研院所深得大家喜爱的重要原因。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 课后感悟\n",
    "\n",
    "这是我第一次比较系统的了解机器学习的整个过程，这个本书讲的比较详细，对tensor的操作也了解了比较多，对使用pytorch进行机器学习的兴趣也有提升，终于找到了一定的门路\n",
    "\n",
    "对最后几个例子的比较感觉：pytorch实现模型相对比较简单，便于理解，只是可能刚刚开始时候，觉得tensor的操作还不是特别明朗，继续加油"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}