coursera_deep_neural_network/1.1_2 Two-Dimensional Tensors_v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Two-Dimensional Tensors</h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Objective</h2><ul><li> How to perform tensor operations on 2D tensors.</li></ul> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Table of Contents</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In this lab, you will learn the basics of tensor operations on 2D tensors.</p>\n",
    "<ul>\n",
    "    <li><a href=\"#Types_Shape\">Types and Shape </a></li>\n",
    "    <li><a href=\"#Index_Slice\">Indexing and Slicing</a></li>\n",
    "    <li><a href=\"#Tensor_Op\">Tensor Operations</a></li>\n",
    "</ul>\n",
    "\n",
    "<p>Estimated Time Needed: <b>10 min</b></p>\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Preparation</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following are the libraries we are going to use for this lab.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:19:09.438383Z",
     "start_time": "2021-07-07T17:19:08.768552Z"
    }
   },
   "outputs": [],
   "source": [
    "# These are the libraries will be used for this lab.\n",
    "\n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Types_Shape\">Types and Shape</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The methods and types for 2D tensors is similar to the methods and types for 1D tensors which has been introduced in <i>Previous Lab</i>.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us see how to convert a 2D list to a 2D tensor. First, let us create a 3X3 2D tensor. Then let us try to use <code>torch.tensor()</code> which we used for converting a 1D list to 1D tensor. Is it going to work?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:19:23.878764Z",
     "start_time": "2021-07-07T17:19:23.875302Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The New 2D Tensor:  tensor([[11, 12, 13],\n",
      "        [21, 22, 23],\n",
      "        [31, 32, 33]])\n"
     ]
    }
   ],
   "source": [
    "# Convert 2D List to 2D Tensor\n",
    "\n",
    "twoD_list = [[11, 12, 13], [21, 22, 23], [31, 32, 33]]\n",
    "twoD_tensor = torch.tensor(twoD_list)\n",
    "print(\"The New 2D Tensor: \", twoD_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bravo! The method <code>torch.tensor()</code> works perfectly.Now, let us try other functions we studied in the <i>Previous Lab</i>.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us try <code><i>tensor_obj</i>.ndimension()</code> (<code>tensor_obj</code>: This can be any tensor object), <code><i>tensor_obj</i>.shape</code>, and <code><i>tensor_obj</i>.size()</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:19:34.767403Z",
     "start_time": "2021-07-07T17:19:34.763513Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The dimension of twoD_tensor:  2\n",
      "The shape of twoD_tensor:  torch.Size([3, 3])\n",
      "The shape of twoD_tensor:  torch.Size([3, 3])\n",
      "The number of elements in twoD_tensor:  9\n"
     ]
    }
   ],
   "source": [
    "# Try tensor_obj.ndimension(), tensor_obj.shape, tensor_obj.size()\n",
    "\n",
    "print(\"The dimension of twoD_tensor: \", twoD_tensor.ndimension())\n",
    "print(\"The shape of twoD_tensor: \", twoD_tensor.shape)\n",
    "print(\"The shape of twoD_tensor: \", twoD_tensor.size())\n",
    "print(\"The number of elements in twoD_tensor: \", twoD_tensor.numel())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because it is a 2D 3X3 tensor,  the outputs are correct.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us try converting the tensor to a numpy array and convert the numpy array back to a tensor.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:19:52.759367Z",
     "start_time": "2021-07-07T17:19:52.748166Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor -> Numpy Array:\n",
      "The numpy array after converting:  [[11 12 13]\n",
      " [21 22 23]\n",
      " [31 32 33]]\n",
      "Type after converting:  int64\n",
      "================================================\n",
      "Numpy Array -> Tensor:\n",
      "The tensor after converting: tensor([[11, 12, 13],\n",
      "        [21, 22, 23],\n",
      "        [31, 32, 33]])\n",
      "Type after converting:  torch.int64\n"
     ]
    }
   ],
   "source": [
    "# Convert tensor to numpy array; Convert numpy array to tensor\n",
    "\n",
    "twoD_numpy = twoD_tensor.numpy()\n",
    "print(\"Tensor -> Numpy Array:\")\n",
    "print(\"The numpy array after converting: \", twoD_numpy)\n",
    "print(\"Type after converting: \", twoD_numpy.dtype)\n",
    "\n",
    "print(\"================================================\")\n",
    "\n",
    "new_twoD_tensor = torch.from_numpy(twoD_numpy)\n",
    "print(\"Numpy Array -> Tensor:\")\n",
    "print(\"The tensor after converting:\", new_twoD_tensor)\n",
    "print(\"Type after converting: \", new_twoD_tensor.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result shows the tensor has successfully been converted to a numpy array and then converted back to a tensor.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us try to convert a Pandas Dataframe to a tensor. The process is the  Same as the 1D conversion, we can obtain the numpy array via the attribute <code>values</code>. Then, we can use <code>torch.from_numpy()</code> to convert the value of the Pandas Series to a tensor.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:20:22.971396Z",
     "start_time": "2021-07-07T17:20:22.966621Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pandas Dataframe to numpy:  [[ 11  12]\n",
      " [ 21  22]\n",
      " [ 31 312]]\n",
      "Type BEFORE converting:  int64\n",
      "================================================\n",
      "Tensor AFTER converting:  tensor([[ 11,  12],\n",
      "        [ 21,  22],\n",
      "        [ 31, 312]])\n",
      "Type AFTER converting:  torch.int64\n"
     ]
    }
   ],
   "source": [
    "# Try to convert the Panda Dataframe to tensor\n",
    "\n",
    "df = pd.DataFrame({'a':[11,21,31],'b':[12,22,312]})\n",
    "\n",
    "print(\"Pandas Dataframe to numpy: \", df.values)\n",
    "print(\"Type BEFORE converting: \", df.values.dtype)\n",
    "\n",
    "print(\"================================================\")\n",
    "\n",
    "new_tensor = torch.from_numpy(df.values)\n",
    "print(\"Tensor AFTER converting: \", new_tensor)\n",
    "print(\"Type AFTER converting: \", new_tensor.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Practice</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to convert the following Pandas Dataframe  to a tensor\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:21:06.805858Z",
     "start_time": "2021-07-07T17:21:06.798119Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[11,  3],\n",
       "        [33,  3],\n",
       "        [22,  2]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Practice: try to convert Pandas Series to tensor\n",
    "\n",
    "df = pd.DataFrame({'A':[11, 33, 22],'B':[3, 3, 2]})\n",
    "tensor_df = torch.from_numpy(df.values)\n",
    "tensor_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!--\n",
    "converted_tensor = torch.tensor(df.values)\n",
    "print (\"Tensor: \", converted_tensor)\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Index_Slice\">Indexing and Slicing</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use rectangular brackets to access the different elements of the tensor. The correspondence between the rectangular brackets and the list and the rectangular representation is shown in the following figure for a 3X3 tensor:  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2index1.png\" width=500 alt=\"Matrix Structure Introduce\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can access the 2nd-row 3rd-column as shown in the following figure:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2index.png\" width=\"500\" alt=\"Example of Matrix Index\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You simply use the square brackets and the indices corresponding to the element that you want.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us try to access the value on position 2nd-row 3rd-column. Remember that the index is always 1 less than how we count rows and columns. There are two ways to access the certain value of a tensor. The example in code will be the same as the example picture above.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:21:37.458696Z",
     "start_time": "2021-07-07T17:21:37.452688Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the value on 2nd-row 3rd-column?  tensor(23)\n",
      "What is the value on 2nd-row 3rd-column?  tensor(23)\n"
     ]
    }
   ],
   "source": [
    "# Use tensor_obj[row, column] and tensor_obj[row][column] to access certain position\n",
    "\n",
    "tensor_example = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "print(\"What is the value on 2nd-row 3rd-column? \", tensor_example[1, 2])\n",
    "print(\"What is the value on 2nd-row 3rd-column? \", tensor_example[1][2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, both methods return the true value (the same value as the picture above). Therefore, both of the methods work.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the elements shown in the following figure: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2_index2.png\" width=\"500\" alt=\"Example of Matrix Index\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the method above, we can access the 1st-row 1st-column by <code>tensor_example[0][0]</code>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:21:43.726071Z",
     "start_time": "2021-07-07T17:21:43.722691Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(11)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor_example[0][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what if we want to get the value on both 1st-row 1st-column and 1st-row 2nd-column?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use slicing in a tensor. Consider the following figure. You want to obtain the 1st two columns in the 1st row:  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2sliceing.png\" width=\"500\" alt=\"Example of Matrix Index and Slicing\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let us see how  we use slicing with 2D tensors to get the values in the above picture.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:21:46.530438Z",
     "start_time": "2021-07-07T17:21:46.523508Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the value on 1st-row first two columns?  tensor([11, 12])\n",
      "What is the value on 1st-row first two columns?  tensor([11, 12])\n"
     ]
    }
   ],
   "source": [
    "# Use tensor_obj[begin_row_number: end_row_number, begin_column_number: end_column number] \n",
    "# and tensor_obj[row][begin_column_number: end_column number] to do the slicing\n",
    "\n",
    "tensor_example = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "print(\"What is the value on 1st-row first two columns? \", tensor_example[0, 0:2])\n",
    "print(\"What is the value on 1st-row first two columns? \", tensor_example[0][0:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get the result as <code>tensor([11, 12])</code> successfully.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But we <b>can't</b> combine using slicing on row and pick one column by using the code <code>tensor_obj[begin_row_number: end_row_number][begin_column_number: end_column number]</code>. The reason is that the slicing will be applied on the tensor first. The result type will be a two dimension again. The second bracket will no longer represent the index of the column it will be the index of the row at that time. Let us see an example. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:21:54.721468Z",
     "start_time": "2021-07-07T17:21:54.711306Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. Slicing step on tensor_example: \n",
      "Result after tensor_example[1:3]:  tensor([[21, 22, 23],\n",
      "        [31, 32, 33]])\n",
      "Dimension after tensor_example[1:3]:  2\n",
      "================================================\n",
      "2. Pick an index on sliced_tensor_example: \n",
      "Result after sliced_tensor_example[1]:  tensor([31, 32, 33])\n",
      "Dimension after sliced_tensor_example[1]:  1\n",
      "================================================\n",
      "3. Combine these step together:\n",
      "Result:  tensor([31, 32, 33])\n",
      "Dimension:  1\n"
     ]
    }
   ],
   "source": [
    "# Give an idea on tensor_obj[number: number][number]\n",
    "\n",
    "tensor_example = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "sliced_tensor_example = tensor_example[1:3]\n",
    "print(\"1. Slicing step on tensor_example: \")\n",
    "print(\"Result after tensor_example[1:3]: \", sliced_tensor_example)\n",
    "print(\"Dimension after tensor_example[1:3]: \", sliced_tensor_example.ndimension())\n",
    "print(\"================================================\")\n",
    "print(\"2. Pick an index on sliced_tensor_example: \")\n",
    "print(\"Result after sliced_tensor_example[1]: \", sliced_tensor_example[1])\n",
    "print(\"Dimension after sliced_tensor_example[1]: \", sliced_tensor_example[1].ndimension())\n",
    "print(\"================================================\")\n",
    "print(\"3. Combine these step together:\")\n",
    "print(\"Result: \", tensor_example[1:3][1])\n",
    "print(\"Dimension: \", tensor_example[1:3][1].ndimension())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See the results and dimensions in 2 and 3 are the same. Both of them contains the 3rd row in the <code>tensor_example</code>, but not the last two values in the 3rd column.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So how can we get the elements in the 3rd column with the last two rows? As the below picture.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2slicing2.png\" width=\"500\" alt=\"Example of Matrix Index and Slicing\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the code below.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:25:42.149995Z",
     "start_time": "2021-07-07T17:25:42.146720Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the value on 3rd-column last two rows?  tensor([23, 33])\n"
     ]
    }
   ],
   "source": [
    "# Use tensor_obj[begin_row_number: end_row_number, begin_column_number: end_column number] \n",
    "\n",
    "tensor_example = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "print(\"What is the value on 3rd-column last two rows? \", tensor_example[1:3, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fortunately, the code <code>tensor_obj[begin_row_number: end_row_number, begin_column_number: end_column number]</code> is still works.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Practice</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to change the values on the second column and the last two rows to 0. Basically, change the values on <code>tensor_ques[1][1]</code> and <code>tensor_ques[2][1]</code> to 0.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:26:36.275445Z",
     "start_time": "2021-07-07T17:26:36.271180Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[11, 12, 13],\n",
       "        [21,  0, 23],\n",
       "        [31,  0, 33]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Practice: Use slice and index to change the values on the matrix tensor_ques.\n",
    "\n",
    "tensor_ques = torch.tensor([[11, 12, 13], [21, 22, 23], [31, 32, 33]])\n",
    "tensor_ques[1:3, 1] =0\n",
    "tensor_ques"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!--\n",
    "tensor_ques[1:3, 1] = 0\n",
    "print(\"The result: \", tensor_ques)\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 id=\"Tensor_Op\">Tensor Operations</h2> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also do some calculations on 2D tensors.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Tensor Addition</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also add tensors; the process is identical to matrix addition. Matrix addition of <b>X</b> and <b>Y</b> is shown in the following figure:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2add.png\" width=\"500\" alt=\"Tensor Addition in 2D\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us see how tensor addition works with <code>X</code> and <code>Y</code>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:26:48.543564Z",
     "start_time": "2021-07-07T17:26:48.531317Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result of X + Y:  tensor([[3, 1],\n",
      "        [1, 3]])\n"
     ]
    }
   ],
   "source": [
    "# Calculate [[1, 0], [0, 1]] + [[2, 1], [1, 2]]\n",
    "\n",
    "X = torch.tensor([[1, 0],[0, 1]]) \n",
    "Y = torch.tensor([[2, 1],[1, 2]])\n",
    "X_plus_Y = X + Y\n",
    "print(\"The result of X + Y: \", X_plus_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like the result shown in the picture above. The result is <code>[[3, 1], [1, 3]]</code>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3> Scalar Multiplication </h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiplying a tensor by a scalar is identical to multiplying a matrix by a scaler. If you multiply the matrix <b>Y</b> by the scalar 2, you simply multiply every element in the matrix by 2 as shown in the figure:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2scaller_mult.png\" width=\"500\" alt=\"The product of tensor and scalar\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us try to calculate the product of <b>2Y</b>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:26:53.688409Z",
     "start_time": "2021-07-07T17:26:53.685804Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result of 2Y:  tensor([[4, 2],\n",
      "        [2, 4]])\n"
     ]
    }
   ],
   "source": [
    "# Calculate 2 * [[2, 1], [1, 2]]\n",
    "\n",
    "Y = torch.tensor([[2, 1], [1, 2]]) \n",
    "two_Y = 2 * Y\n",
    "print(\"The result of 2Y: \", two_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Element-wise Product/Hadamard Product</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiplication of two tensors corresponds to an element-wise product or Hadamard product.  Consider matrix the <b>X</b> and <b>Y</b> with the same size. The Hadamard product corresponds to multiplying each of the elements at the same position, that is, multiplying elements with the same color together. The result is a new matrix that is the same size as matrix <b>X</b> and <b>Y</b> as shown in the following figure:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " <a><img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/chapter%201/1.2tensor_pruduct.png\" width=500 align=\"center\"> </a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code below calculates the element-wise product of the tensor <strong>X</strong> and <strong>Y</strong>:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:26:59.580588Z",
     "start_time": "2021-07-07T17:26:59.571013Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result of X * Y:  tensor([[2, 0],\n",
      "        [0, 2]])\n"
     ]
    }
   ],
   "source": [
    "# Calculate [[1, 0], [0, 1]] * [[2, 1], [1, 2]]\n",
    "\n",
    "X = torch.tensor([[1, 0], [0, 1]])\n",
    "Y = torch.tensor([[2, 1], [1, 2]]) \n",
    "X_times_Y = X * Y\n",
    "print(\"The result of X * Y: \", X_times_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a simple calculation. The result from the code matches the result shown in the picture.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Matrix Multiplication </h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also apply matrix multiplication to two tensors, if you have learned linear algebra, you should know that in the multiplication of two matrices order matters. This means if <i>X _ Y</i> is valid, it does not mean <i>Y _ X</i> is valid. The number of columns of the matrix on the left side of the multiplication sign must equal to the number of rows of the matrix on the right side.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let us create a tensor <code>X</code> with size 2X3. Then, let us create another tensor <code>Y</code> with size 3X2. Since the number of columns of <code>X</code> is equal to the number of rows of <code>Y</code>. We are able to perform the multiplication.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use <code>torch.mm()</code> for calculating the multiplication between tensors with different sizes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:27:09.421862Z",
     "start_time": "2021-07-07T17:27:09.411340Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result of A * B:  tensor([[0, 2],\n",
      "        [0, 2]])\n"
     ]
    }
   ],
   "source": [
    "# Calculate [[0, 1, 1], [1, 0, 1]] * [[1, 1], [1, 1], [-1, 1]]\n",
    "\n",
    "A = torch.tensor([[0, 1, 1], [1, 0, 1]])\n",
    "B = torch.tensor([[1, 1], [1, 1], [-1, 1]])\n",
    "A_times_B = torch.mm(A,B)\n",
    "print(\"The result of A * B: \", A_times_B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Practice</h3>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to create your own two tensors (<code>X</code> and <code>Y</code>) with different sizes, and multiply them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-07-07T17:28:40.053293Z",
     "start_time": "2021-07-07T17:28:40.046283Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[13, 20, 24, 28],\n",
       "        [23, 36, 44, 52],\n",
       "        [32, 50, 61, 72]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Practice: Calculate the product of two tensors (X and Y) with different sizes \n",
    "\n",
    "# Type your code here\n",
    "X = torch.tensor([[1, 3], [3,5], [4,7]])\n",
    "Y = torch.tensor([[1,2,3,4], [4, 6, 7, 8]])\n",
    "torch.mm(X, Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!--\n",
    "X = torch.tensor([[0, 1], [1, 2]])\n",
    "Y = torch.tensor([[-1, -2, 0], [2, 1, 2]])\n",
    "X_times_Y = torch.mm(X, Y)\n",
    "print(\"The result of X * Y: \", X_times_Y)\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--Empty Space for separating topics-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>About the Authors:</h2> \n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a> \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                                          |\n",
    "| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n",
    "| 2020-09-21        | 2.0     | Shubham    | Migrated Lab to Markdown and added to course repo in GitLab |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}