exercise_110.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensors in Pytorch\n",
    "\n",
    "Tensors are just a generalization of vectors and matrices.  Vectors are ordered arrangements of data in 1 dimension.  Matrices are ordered arrangements of data in 2 dimensions.  If you generalize ordered arrangements of data in space to any number of dimensions, you get tensors.  Vectors and matrices are also tensors.\n",
    "\n",
    "Tensors are a very useful tool for neural networks because neural networks and their data processing pipelines can be built entirely out of tensors.\n",
    "\n",
    "The data that is fed to neural networks can be represented by tensors.  The parameters of the neural networks (the weights and the biases) can be represented by tensors.  So can the outputs.\n",
    "\n",
    "So, once you get the hang of tensors, you can start wiring up neural networks in no time at all.\n",
    "\n",
    "This set of exercises shows you how to create and manipulate tensors in Pytorch.\n",
    "\n",
    "Let's start by importing Pytorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construction of Tensors\n",
    "\n",
    "First, let's create a 1-dimensional tensor (just a fancy name for a vector) with 2 elements!\n",
    "\n",
    "The constructor takes one parameter for each dimension of the tensor, specifying the tensor's length in that dimension.\n",
    "\n",
    "This being a 1-dimensional tensor, its constructor needs just one parameter, specifying the length of the vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "-9.8505e-32\n",
       " 4.5789e-41\n",
       "[torch.FloatTensor of size 2]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor = torch.Tensor(2)\n",
    "a1DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The vector, as you can see, is uninitialized (well, initialized with garbage).  If you'd rather have it filled with zeros, you can do the following instead. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 0\n",
       " 0\n",
       "[torch.FloatTensor of size 2]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor = torch.zeros(2)\n",
    "a1DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to initialize a tensor with more than one dimension, pass more parameters to the constructor or factory method.\n",
    "\n",
    "Other common factory methods are 'ones' and 'rand' and 'randn'.\n",
    "\n",
    "We create a 2-dimensional tensor filled with ones, a 3-dimensional tensor filled with random numbers between 0 and 1, and a 4-dimensional tensor filled with random numbers with a normal distribution with a mean at zero and a variance of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1  1  1\n",
       " 1  1  1\n",
       "[torch.FloatTensor of size 2x3]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor = torch.ones(2, 3)\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "(0 ,.,.) = \n",
       "  0.6118  0.5193  0.9150  0.9306\n",
       "  0.9906  0.1572  0.4360  0.6906\n",
       "  0.3010  0.4468  0.5174  0.5219\n",
       "\n",
       "(1 ,.,.) = \n",
       "  0.1513  0.9197  0.0548  0.7084\n",
       "  0.0903  0.7841  0.1023  0.1698\n",
       "  0.4516  0.0837  0.2382  0.4214\n",
       "[torch.FloatTensor of size 2x3x4]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a3DTensor = torch.rand(2, 3, 4)\n",
    "a3DTensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "(0 ,0 ,.,.) = \n",
       " -0.8499 -1.7045  0.7730  0.7749  0.4734\n",
       " -0.6560 -1.3752 -0.8713  0.7747  0.6232\n",
       "  0.4945  0.9998 -0.5692  1.6949 -0.7417\n",
       "  0.8606  2.3180 -0.8494 -0.4825 -1.1582\n",
       "\n",
       "(0 ,1 ,.,.) = \n",
       "  0.1876 -1.1114  1.1169  2.0289 -0.3846\n",
       "  0.1451 -0.4179  0.1244 -1.6353 -0.9711\n",
       "  0.7424 -2.7805  1.0816 -0.0072 -0.1059\n",
       " -0.5275 -0.5290 -0.2082  0.4367 -0.1160\n",
       "\n",
       "(0 ,2 ,.,.) = \n",
       " -1.7193  1.1614 -0.0332  0.0330 -0.0558\n",
       " -0.7614 -0.8246  1.5288 -0.5309 -0.3108\n",
       " -0.5180 -0.0520  1.1208 -0.2534  1.5658\n",
       " -0.8679  0.6346 -0.7800  0.5492  1.6903\n",
       "\n",
       "(1 ,0 ,.,.) = \n",
       "  0.4532 -1.9499 -0.3475  0.6006 -1.1247\n",
       "  0.0827 -1.0662 -0.4450 -0.4823  0.8346\n",
       "  0.7627  2.5110 -1.9230  1.0761  1.8737\n",
       " -1.4201  1.3169 -0.7594 -1.0689 -0.1131\n",
       "\n",
       "(1 ,1 ,.,.) = \n",
       "  0.8274  0.2676  0.5355 -2.3884  1.7401\n",
       " -0.8509  1.5411  0.1777  0.7324 -0.0365\n",
       "  0.1070  1.1835 -0.0473 -0.0699  0.2342\n",
       "  2.4218  1.6472 -0.0680 -0.3118  0.3072\n",
       "\n",
       "(1 ,2 ,.,.) = \n",
       " -0.3922 -0.4971  0.1229 -0.4838 -1.7769\n",
       " -0.4688  0.4796  0.3979 -0.5539 -0.3340\n",
       " -0.2009  1.2470 -0.9213 -0.7476  0.4924\n",
       "  1.6058 -2.1083  0.4165  1.2652  1.1148\n",
       "[torch.FloatTensor of size 2x3x4x5]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a4DTensor = torch.randn(2, 3, 4, 5)\n",
    "a4DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So you can create tensors of arbitary tensors.\n",
    "\n",
    "The 2-dimensional tensors, as you know, are called matrics.\n",
    "\n",
    "Tensors are merely generalization of vectors and matrices to higher dimensions.\n",
    "\n",
    "You can also create tensors from matrices.  This is particularly useful when you're converting input data and output data into tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1  2  3\n",
       " 4  5  6\n",
       "[torch.FloatTensor of size 2x3]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor = torch.Tensor([[1, 2, 3],[4, 5, 6]])\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Editing Tensors\n",
    "\n",
    "You can slice and dice tensors.\n",
    "\n",
    "The following code assigns the value 4 to all elements in the second column of the 2D tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1  4  3\n",
       " 4  4  6\n",
       "[torch.FloatTensor of size 2x3]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[:,1] = 4\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also edit individual elements in the tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 1  2  3\n",
       " 4  5  6\n",
       "[torch.FloatTensor of size 2x3]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[0,1] = 2\n",
    "a2DTensor[1,1] = 5\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can fill the tensor with the value 2.2 (any member function ending in an underscore modifies the object it was called on)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 2.2000  2.2000  2.2000\n",
       " 2.2000  2.2000  2.2000\n",
       "[torch.FloatTensor of size 2x3]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.fill_(2.2)\n",
    "a2DTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can slice the matrix and return the second row as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 2.2000\n",
       " 2.2000\n",
       " 2.2000\n",
       "[torch.FloatTensor of size 3]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or the first column (the indices start at 0):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 2.2000\n",
       " 2.2000\n",
       "[torch.FloatTensor of size 2]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[:,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also get back the dimensions of a tensor using the size() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or a specific dimension (and this returns an integer):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor.size(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There you go.  If you have a 1D tensor, and look up an element in it, you get a float or an int.  Similarly if you have e 2D tensor and look up an element at a row and column, you get a float or an int."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a1DTensor[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.200000047683716"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a2DTensor[0,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tensors are by default assumed to be meant to hold real numbers.\n",
    "\n",
    "You can also explicitly specify in the constructor whether the tensor should hold real numbers or be limited to integers.\n",
    "\n",
    "The following will hold only integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "-1.9462e+09  3.2676e+04\n",
       "[torch.IntTensor of size 1x2]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anIntTensor = torch.IntTensor(1,2)\n",
    "anIntTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It appears to hold floating point values because it hasn't been initialized.\n",
    "\n",
    "Fill it with something.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 5  5\n",
       "[torch.IntTensor of size 1x2]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anIntTensor.fill_(5)\n",
    "anIntTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following tensor is intended to hold real numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "-9.8504e-32  4.5789e-41\n",
       "[torch.FloatTensor of size 1x2]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aFloatTensor = torch.FloatTensor(1,2)\n",
    "aFloatTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike an int tensor, you can can fill a float tensor with real numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       " 5.5000  5.5000\n",
       "[torch.FloatTensor of size 1x2]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aFloatTensor.fill_(5.5)\n",
    "aFloatTensor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise:  Operations on Tensors\n",
    "\n",
    "Create and add two tensors.  This tutorial on Pytorch explains how: http://pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html#operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}