exercise_110.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensors in Pytorch\n",
    "\n",
    "Tensors are just a generalization of vectors and matrices.  Vectors are ordered arrangements of data in 1 dimension.  Matrices are ordered arrangements of data in 2 dimensions.  If you generalize ordered arrangements of data in space to any number of dimensions, you get tensors.  Vectors and matrices are also tensors.\n",
    "\n",
    "Tensors are a very useful tool for neural networks because neural networks and their data processing pipelines can be built entirely out of tensors.\n",
    "\n",
    "The data that is fed to neural networks can be represented by tensors.  The parameters of the neural networks (the weights and the biases) can be represented by tensors.  So can the outputs.\n",
    "\n",
    "So, once you get the hang of tensors, you can start wiring up neural networks in no time at all.\n",
    "\n",
    "This set of exercises shows you how to create and manipulate tensors in Pytorch.\n",
    "\n",
    "Let's start by importing Pytorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construction of Tensors\n",
    "\n",
    "First, let's create a 1-dimensional tensor (just a fancy name for a vector) with 2 elements!\n",
    "\n",
    "The constructor takes one parameter for each dimension of the tensor, specifying the tensor's length in that dimension.\n",
    "\n",
    "This being a 1-dimensional tensor, its constructor needs just one parameter, specifying the length of the vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0., -2.])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor = torch.Tensor(2)\n",
    "a1DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The vector, as you can see, is uninitialized (well, initialized with garbage).  If you'd rather have it filled with zeros, you can do the following instead. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.,  0.])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor = torch.zeros(2)\n",
    "a1DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to initialize a tensor with more than one dimension, pass more parameters to the constructor or factory method.\n",
    "\n",
    "Other common factory methods are 'ones' and 'rand' and 'randn'.\n",
    "\n",
    "We create a 2-dimensional tensor filled with ones, a 3-dimensional tensor filled with random numbers between 0 and 1, and a 4-dimensional tensor filled with random numbers with a normal distribution with a mean at zero and a variance of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.,  1.,  1.],\n",
       "        [ 1.,  1.,  1.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor = torch.ones(2, 3)\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So you can create tensors of arbitary dimensions.\n",
    "\n",
    "The 2-dimensional tensors, as you know, are called matrics.\n",
    "\n",
    "Tensors are merely generalization of vectors and matrices to higher dimensions.\n",
    "\n",
    "You can also create tensors from matrices.  This is particularly useful when you're converting input data and output data into tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.,  2.,  3.],\n",
       "        [ 4.,  5.,  6.]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor = torch.Tensor([[1, 2, 3],[4, 5, 6]])\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing Tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.9006,  0.0140,  0.5068,  0.1320],\n",
       "         [ 0.4104,  0.6199,  0.3985,  0.8844],\n",
       "         [ 0.2050,  0.1709,  0.3295,  0.8669]],\n",
       "\n",
       "        [[ 0.9818,  0.0751,  0.0092,  0.2305],\n",
       "         [ 0.4660,  0.5046,  0.1751,  0.7514],\n",
       "         [ 0.6547,  0.5198,  0.5565,  0.4588]]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a3DTensor = torch.rand(2, 3, 4)\n",
    "a3DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that when Pytorch prints a tensor, the outermost square brackets enclose the first dimension.\n",
    "\n",
    "The next inner nesting of square brackets enclose the second dimension (here, the matrix-row dimension).\n",
    "\n",
    "The innermost square brackets point in the third dimension (here, the matrix-column dimension).\n",
    "\n",
    "In the Pytorch representation, the last two dimensions are represented as matrix-row and matrix-column.  This out-in ordering of dimensions is good to get a hang of for debugging data flows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[[-1.1389,  0.2702, -1.4285, -0.9025, -1.0754],\n",
       "          [-0.1063, -0.0091,  1.2139, -0.3160,  1.6785],\n",
       "          [ 0.9278,  1.7833, -0.3822, -1.5971,  0.4693],\n",
       "          [-0.2132, -0.0054,  1.7708,  0.7218,  0.2263]],\n",
       "\n",
       "         [[ 0.1724,  0.2538,  0.3255,  0.3690,  1.0615],\n",
       "          [-0.9741, -0.5404, -0.4065,  0.2456,  1.5072],\n",
       "          [-0.8681, -0.3479, -0.2729, -0.5055, -1.1438],\n",
       "          [-0.5925,  0.4438,  1.4026, -1.3871, -0.2138]],\n",
       "\n",
       "         [[-0.4431, -0.6146,  0.2168, -0.8108, -1.0003],\n",
       "          [ 0.1847, -0.9210,  1.1536, -0.2260,  0.1937],\n",
       "          [-0.6403, -1.5614, -0.5467,  0.9163,  0.4388],\n",
       "          [-0.0931,  2.0600, -1.6304,  0.0111,  0.5999]]],\n",
       "\n",
       "\n",
       "        [[[ 0.1407,  0.3031, -0.0901, -0.4847, -0.7330],\n",
       "          [-0.7387,  1.2340,  0.9686, -2.0939,  1.5344],\n",
       "          [-0.3066,  0.1749, -0.5112,  1.4151,  0.3104],\n",
       "          [-0.4935, -0.1328,  0.3981, -0.5200, -1.6364]],\n",
       "\n",
       "         [[-0.6043,  0.6915, -0.3539, -0.0924, -1.5290],\n",
       "          [ 0.4126,  1.3571,  2.1300,  1.6577,  0.0819],\n",
       "          [-0.4045, -0.0112,  0.3560, -0.4958,  2.3963],\n",
       "          [-1.2857, -0.1174,  1.0342,  0.1440, -0.2464]],\n",
       "\n",
       "         [[ 0.6675,  0.2634, -1.3504, -0.3679, -0.5598],\n",
       "          [-0.7374, -1.3172,  0.4503,  0.7979, -1.0447],\n",
       "          [-1.0169, -1.0769, -0.4469,  0.4486, -0.9237],\n",
       "          [-1.1366,  1.9009, -0.3650,  0.6777, -0.7895]]]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a4DTensor = torch.randn(2, 3, 4, 5)\n",
    "a4DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Editing Tensors\n",
    "\n",
    "You can slice and dice tensors.\n",
    "\n",
    "The following code assigns the value 4 to all elements in the second column of the 2D tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.,  4.,  3.],\n",
       "        [ 4.,  4.,  6.]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[:,1] = 4\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also edit individual elements in the tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.,  2.,  3.],\n",
       "        [ 4.,  5.,  6.]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[0,1] = 2\n",
    "a2DTensor[1,1] = 5\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can fill the tensor with the value 2.2 (any member function ending in an underscore modifies the object it was called on)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 2.2000,  2.2000,  2.2000],\n",
       "        [ 2.2000,  2.2000,  2.2000]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.fill_(2.2)\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can slice the matrix and return the second row as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 2.2000,  2.2000,  2.2000])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or the first column (the indices start at 0):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 2.2000,  2.2000])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[:,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also get back the dimensions of a tensor using the size() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or a specific dimension (and this returns an integer):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.size(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There you go.  If you have a 1D tensor, and look up an element in it, you get a float or an int.  Similarly if you have e 2D tensor and look up an element at a row and column, you get a float or an int."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(2.2000)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[0,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tensors are by default assumed to be meant to hold real numbers.\n",
    "\n",
    "You can also explicitly specify in the constructor whether the tensor should hold real numbers or be limited to integers.\n",
    "\n",
    "The following will hold only integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.0000e+00, -1.0737e+09]], dtype=torch.int32)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anIntTensor = torch.IntTensor(1,2)\n",
    "anIntTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It appears to hold floating point values because it hasn't been initialized.\n",
    "\n",
    "Fill it with something.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 5,  5]], dtype=torch.int32)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anIntTensor.fill_(5)\n",
    "anIntTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following tensor is intended to hold real numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0., -2.]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aFloatTensor = torch.FloatTensor(1,2)\n",
    "aFloatTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike an int tensor, you can can fill a float tensor with real numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 5.5000,  5.5000]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aFloatTensor.fill_(5.5)\n",
    "aFloatTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise:  Operations on Tensors\n",
    "\n",
    "Create and add two tensors.  This tutorial on Pytorch explains how: http://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}