11_introduction-to-tensorflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What is Tensor in TensorFlow?\n",
    "\n",
    "## What is a Tensor?\n",
    "\n",
    "Tensorflow’s name is directly derived from its core framework:\n",
    "Tensor. In Tensorflow, all the computations involve tensors. A\n",
    "tensor is a vector or matrix of n-dimensions that represents all\n",
    "types of data. All values in a tensor hold identical data type with a\n",
    "known (or partially known) shape. The shape of the data is the\n",
    "dimensionality of the matrix or array.\n",
    "\n",
    "A tensor can be originated from the input data or the result of a\n",
    "computation. In TensorFlow, all the operations are conducted inside a\n",
    "graph. The graph is a set of computation that takes place\n",
    "successively. Each operation is called an op node and are connected\n",
    "to each other.\n",
    "\n",
    "The graph outlines the ops and connections between the nodes. However,\n",
    "it does not display the values. The edge of the nodes is the tensor,\n",
    "i.e., a way to populate the operation with data.\n",
    "\n",
    "Machine learning involves learning the underlying pattern from the data.\n",
    "The learning phase requires selecting a numerous amount of vectors. Each\n",
    "vector is a simplification of the data or a way to measure the observed\n",
    "phenomena. These feature vectors will be used to train a model and from\n",
    "this model, you can predict the future values.\n",
    "\n",
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/11_introduction-to-tensorflow_files/image51_1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Machine Learning, models are feed with a list of objects called\n",
    "**feature vectors**. A feature vector can be of any data type. The\n",
    "feature vector will usually be the primary input to populate a tensor.\n",
    "These values will flow into an op node through the tensor and the result\n",
    "of this operation/computation will create a new tensor which in turn\n",
    "will be used in a new operation. All these operations can be viewed in\n",
    "the graph.\n",
    "\n",
    "\n",
    "\n",
    "## Representation of a Tensor\n",
    "\n",
    "\n",
    "In TensorFlow, a tensor is a collection of feature vectors (i.e., array)\n",
    "of n-dimensions. For instance, if we have a 2x3 matrix with values from\n",
    "1 to 6, we write:\n",
    "\n",
    "$\n",
    "\\begin{bmatrix} \n",
    "1 & 2 & 3  \\\\\\\\\\\\\n",
    "4 & 5 & 6\n",
    "\\end{bmatrix} \n",
    "$\n",
    "\n",
    "TensorFlow represents this matrix as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "[[1, 2, 3],\n",
    " [4, 5, 6]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we create a three-dimensional matrix with values from 1 to 8, we\n",
    "have:\n",
    "\n",
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/11_introduction-to-tensorflow_files/image052.png)\n",
    "\n",
    "\n",
    "\n",
    "TensorFlow represents this matrix as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[[1, 2], [3, 4]], [[5, 6], [7, 8]]]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[ [[1, 2],\n",
    "  [3, 4]],\n",
    "  [[5, 6],\n",
    "  [7,8]] ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** A tensor can be represented with a scalar or can have a shape\n",
    "of more than three dimensions. It is just more complicated to visualize\n",
    "higher dimension level.\n",
    "\n",
    "## Types of Tensor\n",
    "\n",
    "\n",
    "In TensorFlow, all the computations pass through one or more tensors. A\n",
    "tensor is an object with three properties:\n",
    "\n",
    "-   A unique label (`name`)\n",
    "\n",
    "-   A dimension (`shape`)\n",
    "\n",
    "-   A data type (`dtype`)\n",
    "\n",
    "Each operation you will do with TensorFlow involves the manipulation of\n",
    "a tensor. There are four main tensors you can create:\n",
    "\n",
    "-   `tf.Variable`\n",
    "\n",
    "-   `tf.constant`\n",
    "\n",
    "-   `tf.placeholder`\n",
    "\n",
    "-   `tf.SparseTensor`\n",
    "\n",
    "In this tutorial, you will learn how to create a `tf.constant` and a\n",
    "`tf.Variable`.\n",
    "\n",
    "Before we go through the tutorial, make sure you activate the conda\n",
    "environment with TensorFlow. We named this environment `hello-tf`.\n",
    "\n",
    "**For MacOS user:**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "source activate hello-tf\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**For Windows user:**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "activate hello-tf\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After you have done that, you are ready to import `tensorflow`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/Thomas/anaconda3/envs/hello-tf/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "# Import tf\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a tensor of n-dimension\n",
    "\n",
    "You begin with the creation of a tensor with one dimension, namely a\n",
    "scalar.\n",
    "\n",
    "To create a tensor, you can use `tf.constant()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "tf.constant(value, dtype, name = \"\")\n",
    "arguments\n",
    "\n",
    "- `value`: Value of n dimension to define the tensor. Optional\n",
    "- `dtype`: Define the type of data:\n",
    "    - `tf.string`: String variable\n",
    "    - `tf.float32`: Flot variable\n",
    "    - `tf.int16`: Integer variable\n",
    "- \"name\": Name of the tensor. Optional. By default, Const_1:0   \n",
    "```        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a tensor of dimension 0, run the following code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const:0\", shape=(), dtype=int16)\n"
     ]
    }
   ],
   "source": [
    "## rank 0\n",
    "# Default name\n",
    "r1 = tf.constant(1, tf.int16) \n",
    "print(r1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/11_introduction-to-tensorflow_files/image053.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"my_scalar:0\", shape=(), dtype=int16)\n"
     ]
    }
   ],
   "source": [
    "# Named my_scalar\n",
    "r2 = tf.constant(1, tf.int16, name = \"my_scalar\") \n",
    "print(r2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each tensor is displayed by the tensor name. Each tensor object is\n",
    "defined with a unique label (`name`), a dimension (`shape`) and a data\n",
    "type (`dtype`).\n",
    "\n",
    "You can define a tensor with decimal values or with a string by changing\n",
    "the type of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_1:0\", shape=(), dtype=float32)\n",
      "Tensor(\"Const_2:0\", shape=(), dtype=string)\n"
     ]
    }
   ],
   "source": [
    "# Decimal\n",
    "r1_decimal =  tf.constant(1.12345, tf.float32)\n",
    "print(r1_decimal)\n",
    "# String\n",
    "r1_string = tf.constant(\"Guru99\", tf.string)\n",
    "print(r1_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A tensor of dimension 1 can be created as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_3:0\", shape=(3,), dtype=int16)\n",
      "Tensor(\"Const_4:0\", shape=(3,), dtype=bool)\n"
     ]
    }
   ],
   "source": [
    "## Rank 1\n",
    "r1_vector = tf.constant([1,3,5], tf.int16)\n",
    "print(r1_vector)\n",
    "r2_boolean = tf.constant([True, True, False], tf.bool)\n",
    "print(r2_boolean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can notice the shape is only composed of 1 column.\n",
    "\n",
    "To create an array of 2 dimensions, you need to close the brackets after\n",
    "each row. Check the examples below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_5:0\", shape=(2, 2), dtype=int16)\n"
     ]
    }
   ],
   "source": [
    "## Rank 2\n",
    "r2_matrix = tf.constant([ [1, 2],\n",
    "                          [3, 4] ],tf.int16)\n",
    "print(r2_matrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix has 2 rows and 2 columns filled with values 1, 2, 3, 4.\n",
    "\n",
    "A matrix with 3 dimensions is constructed by adding another level with\n",
    "the brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const_6:0\", shape=(1, 3, 2), dtype=int16)\n"
     ]
    }
   ],
   "source": [
    "## Rank 3\n",
    "r3_matrix = tf.constant([ [[1, 2],\n",
    "                           [3, 4], \n",
    "                           [5, 6]] ], tf.int16)\n",
    "print(r3_matrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix looks like the picture two.\n",
    "\n",
    "## Shape of tensor\n",
    "\n",
    "\n",
    "When you print the tensor, TensorFlow guesses the shape. However, you\n",
    "can get the shape of the tensor with the `shape` property.\n",
    "\n",
    "Below, you construct a matrix filled with a number from 10 to 15 and you\n",
    "check the shape of `m_shape`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([Dimension(3), Dimension(2)])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Shape of tensor\n",
    "m_shape = tf.constant([ [10, 11],\n",
    "                        [12, 13],\n",
    "                        [14, 15] ]\n",
    "                      ) \n",
    "m_shape.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The matrix has 3 rows and 2 columns.\n",
    "\n",
    "TensorFlow has useful commands to create a vector or a matrix filled\n",
    "with 0 or 1. For instance, if you want to create a 1-D tensor with a\n",
    "specific shape of 10, filled with 0, you can run the code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"zeros:0\", shape=(10,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Create a vector of 0\n",
    "print(tf.zeros(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The property works for matrix as well. Here, you create a 10x10 matrix\n",
    "filled with 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"ones:0\", shape=(10, 10), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Create a vector of 1\n",
    "print(tf.ones([10, 10]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the shape of a given matrix to make a vector of ones. The\n",
    "matrix `m_shape` is a 3x2 dimensions. You can create a tensor with 3\n",
    "rows filled by one’s with the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"ones_1:0\", shape=(3,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Create a vector of ones with the same number of rows as m_shape\n",
    "print(tf.ones(m_shape.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you pass the value 1 into the bracket, you can construct a vector of\n",
    "ones equals to the number of columns in the matrix `m_shape`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"ones_2:0\", shape=(2,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Create a vector of ones with the same number of column as m_shape\n",
    "print(tf.ones(m_shape.shape[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, you can create a matrix 3x2 with only one’s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"ones_3:0\", shape=(3, 2), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "print(tf.ones(m_shape.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Type of data\n",
    "\n",
    "\n",
    "The second property of a tensor is the type of data. A tensor can only\n",
    "have one type of data at a time. A tensor can only have one type of\n",
    "data. You can return the type with the property `dtype`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<dtype: 'int32'>\n"
     ]
    }
   ],
   "source": [
    "print(m_shape.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In some occasions, you want to change the type of data. In TensorFlow,\n",
    "it is possible with `tf.cast` method.\n",
    "\n",
    "**Example**\n",
    "\n",
    "Below, a float tensor is converted to integer using you use the method\n",
    "`cast`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<dtype: 'float32'>\n",
      "<dtype: 'int32'>\n"
     ]
    }
   ],
   "source": [
    "# Change type of data\n",
    "type_float = tf.constant(3.123456789, tf.float32)\n",
    "type_int = tf.cast(type_float, dtype=tf.int32)\n",
    "print(type_float.dtype)\n",
    "print(type_int.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow chooses the type of data automatically when the argument is\n",
    "not specified during the creation of the tensor. TensorFlow will guess\n",
    "what is the most likely types of data. For instance, if you pass a text,\n",
    "it will guess it is a string and convert it to `string`.\n",
    "\n",
    "## Creating operator\n",
    "\n",
    "\n",
    "### Some Useful TensorFlow operators\n",
    "\n",
    "You know how to create a tensor with TensorFlow. It is time to learn how\n",
    "to perform mathematical operations.\n",
    "\n",
    "TensorFlow contains all the basic operations. You can begin with a\n",
    "simple one. You will use TensorFlow method to compute the square of a\n",
    "number. This operation is straightforward because only one argument is\n",
    "required to construct the tensor.\n",
    "\n",
    "The square of a number is constructed with `tf.sqrt``(x)` with `x` as a\n",
    "floating number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Sqrt:0\", shape=(1,), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "x = tf.constant([2.0], dtype = tf.float32)\n",
    "print(tf.sqrt(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** The output returned a tensor object and not the result of the\n",
    "square of 2. In the example, you print the definition of the tensor and\n",
    "not the actual evaluation of the operation. In the next section, you\n",
    "will learn how TensorFlow works to execute the operations.\n",
    "\n",
    "Following is a list of commonly used operations. The idea is the same.\n",
    "Each operation requires one or more arguments.\n",
    "\n",
    "-   `tf.add(a, b)`\n",
    "\n",
    "-   `tf.substract(a, b)`\n",
    "\n",
    "-   `tf.multiply(a, b)`\n",
    "\n",
    "-   `tf.div(a, b)`\n",
    "\n",
    "-   `tf.pow(a, b)`\n",
    "\n",
    "-   `tf.exp(a)`\n",
    "\n",
    "-   `tf.sqrt(a)`\n",
    "\n",
    "**Example**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Add:0\", shape=(1, 2), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "# Add\n",
    "tensor_a = tf.constant([[1,2]], dtype = tf.int32)\n",
    "tensor_b = tf.constant([[3, 4]], dtype = tf.int32)\n",
    "\n",
    "tensor_add = tf.add(tensor_a, tensor_b)\n",
    "print(tensor_add)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Code Explanation*\n",
    "\n",
    "Create two tensors:\n",
    "\n",
    "-   one tensor with 1 and 2\n",
    "\n",
    "-   one tensor with 3 and 4\n",
    "\n",
    "You add up both tensors.\n",
    "\n",
    "**Notice**: that both tensors need to have the same shape. You can\n",
    "execute a multiplication over the two tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Mul:0\", shape=(1, 2), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "# Multiply\n",
    "tensor_multiply = tf.multiply(tensor_a, tensor_b)\n",
    "print(tensor_multiply)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variables\n",
    "\n",
    "\n",
    "So far, you have only created constant tensors. It is not of great use.\n",
    "Data always arrive with different values, to capture this, you can use\n",
    "the `Variable` class. It will represent a node where the values always\n",
    "change.\n",
    "\n",
    "To create a variable, you can use `tf.get_variable()` method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "tf.get_variable(name = \"\", values, dtype, initializer)\n",
    "argument\n",
    "\n",
    "- `name = \"\"`: Name of the variable\n",
    "- `values`: Dimension of the tensor\n",
    "- `dtype`: Type of data. Optional\n",
    "- `initializer`: How to initialize the tensor. Optional\n",
    "```    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If initializer is specified, there is no need to include the `values` as the shape of `initializer` is used.\n",
    "\n",
    "For instance, the code below creates a two-dimensional variable with two\n",
    "random values. By default, TensorFlow returns a random value. You name\n",
    "the variable `var`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n"
     ]
    }
   ],
   "source": [
    "# Create a Variable\n",
    "## Create 2 Randomized values\n",
    "var = tf.get_variable(\"var\", [1, 2])\n",
    "print(var.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the second example, you create a variable with one row and two\n",
    "columns. You need to use `[1,2]` to create the dimension of the variable\n",
    "\n",
    "The initials values of this tensor are zero. For instance, when you\n",
    "train a model, you need to have initial values to compute the weight of\n",
    "the features. Below, you set these initial value to zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n"
     ]
    }
   ],
   "source": [
    "var_init_1 = tf.get_variable(\"var_init_1\", [1, 2], dtype=tf.int32,\n",
    "  initializer=tf.zeros_initializer)\n",
    "print(var_init_1.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can pass the values of a constant tensor in a variable. You create a\n",
    "constant tensor with the method `tf.constant()`. You use this tensor\n",
    "to initialize the variable.\n",
    "\n",
    "The first values of the variable are 10, 20, 30 and 40. The new tensor\n",
    "will have a shape of 2x2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2)\n"
     ]
    }
   ],
   "source": [
    "# Create a 2x2 matrix\n",
    "tensor_const = tf.constant([[10, 20], \n",
    "                            [30, 40]])\n",
    "\n",
    "# Initialize the first value of the tensor equals to tensor_const\n",
    "var_init_2 = tf.get_variable(\"var_init_2\", dtype=tf.int32,\n",
    "  initializer=tensor_const)\n",
    "print(var_init_2.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Placeholder\n",
    "\n",
    "\n",
    "A placeholder has the purpose of feeding the tensor. Placeholder is used\n",
    "to initialize the data to flow inside the tensors. To supply a\n",
    "placeholder, you need to use the method `feed_dict`. The placeholder\n",
    "will be fed only within a session.\n",
    "\n",
    "In the next example, you will see how to create a placeholder with the\n",
    "method `tf.placeholder`. In the next session, you will learn to fed a\n",
    "placeholder with actual value.\n",
    "\n",
    "The syntax is:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "tf.placeholder(dtype,shape=None,name=None )\n",
    "arguments:\n",
    "\n",
    "- `dtype`: Type of data\n",
    "- `shape`: dimension of the placeholder. Optional. By default, shape of the data\n",
    "- `name`: Name of the placeholder. Optional\n",
    "```    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"data_placeholder_a:0\", dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "data_placeholder_a = tf.placeholder(tf.float32, name = \"data_placeholder_a\")\n",
    "print(data_placeholder_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Session\n",
    "\n",
    "\n",
    "TensorFlow works around 3 main components:\n",
    "\n",
    "-   Graph\n",
    "-   Tensor\n",
    "-   Session\n",
    "\n",
    "\n",
    "| Components | Descritption                                                 |\n",
    "| ---------- | ------------------------------------------------------------ |\n",
    "| Graph      | The graph is fundamental in TensorFlow. All of the mathematical operations (ops) are performed inside a graph. You can imagine a graph as a project where every operations are done. The nodes represent these ops, they can absorb or create new tensors. |\n",
    "| Tensor     | A tensor represents the data that progress between operations. You saw previously how to initialize a tensor. The difference between a constant and variable is the initial values of a variable will change over time. |\n",
    "| Session    | A session will execute the operation from the graph. To feed the graph with the values of a tensor, you need to open a session. Inside a session, you must run an operator to create an output. |\n",
    "\n",
    "Graphs and sessions are independent. You can run a session and get the\n",
    "values to use later for further computations.\n",
    "\n",
    "In the example below, you will:\n",
    "\n",
    "-   Create two tensors\n",
    "\n",
    "-   Create an operation\n",
    "\n",
    "-   Open a session\n",
    "\n",
    "-   Print the result\n",
    "\n",
    "**Step 1)** You create two tensors `x` and `y`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Create, run  and evaluate a session\n",
    "x = tf.constant([2])\n",
    "y = tf.constant([4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 2)** You create the operator by multiplying `x` and `y`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Create operator\n",
    "multiply = tf.multiply(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 3)** You open a session. All the computations will happen within\n",
    "the session. When you are done, you need to close the session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8]\n"
     ]
    }
   ],
   "source": [
    "## Create a session to run the code\n",
    "sess = tf.Session()\n",
    "result_1 = sess.run(multiply)\n",
    "print(result_1)\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Code explanation*\n",
    "\n",
    "-   `tf.Session()`: Open a session. All the operations will flow\n",
    "    within the sessions\n",
    "\n",
    "-   `run(multiply)`: execute the operation created in step 2.\n",
    "\n",
    "-   `print(result_1)`: Finally, you can print the result\n",
    "\n",
    "-   `close()`: Close the session\n",
    "\n",
    "The result shows 8, which is the multiplication of `x` and `y`.\n",
    "\n",
    "Another way to create a session is inside a block. The advantage is it\n",
    "automatically closes the session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8]\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    result_2 = multiply.eval()\n",
    "print(result_2) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In a context of the session, you can use the `eval()` method to\n",
    "execute the operation. It is equivalent to `run()`. It makes the code\n",
    "more readable.\n",
    "\n",
    "You can create a session and see the values inside the tensors you\n",
    "created so far."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "[[1 2]\n",
      " [3 4]]\n",
      "[[[1 2]\n",
      "  [3 4]\n",
      "  [5 6]]]\n"
     ]
    }
   ],
   "source": [
    "## Check the tensors created before\n",
    "sess = tf.Session()\n",
    "print(sess.run(r1))\n",
    "print(sess.run(r2_matrix))\n",
    "print(sess.run(r3_matrix))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variables are empty by default, even after you create a tensor. You need\n",
    "to initialize the variable if you want to use the variable. The object\n",
    "`tf.global_variables_initializer()` needs to be called to initialize\n",
    "the values of a variable. This object will explicitly initialize all the\n",
    "variables. This is helpful before you train a model.\n",
    "\n",
    "You can check the values of the variables you created before. Note that\n",
    "you need to use `run` to evaluate the tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.2510476  1.2630793]]\n",
      "[[0 0]]\n",
      "[[10 20]\n",
      " [30 40]]\n"
     ]
    }
   ],
   "source": [
    "sess.run(tf.global_variables_initializer())\n",
    "print(sess.run(var))\n",
    "print(sess.run(var_init_1))\n",
    "print(sess.run(var_init_2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use the placeholder you created before and feed it with actual\n",
    "value. You need to pass the data into the method `feed_dict`.\n",
    "\n",
    "For example, you will take the power of 2 of the placeholder\n",
    "`data_placeholder_a`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.47110406 0.22262007 0.06629956 0.09401076 0.9245911  0.22435442\n",
      "  0.04688335 0.30758923 0.14803676 0.90118146]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "power_a = tf.pow(data_placeholder_a, 2)\n",
    "with tf.Session() as sess:\n",
    "  data = np.random.rand(1, 10)\n",
    "  print(sess.run(power_a, feed_dict={data_placeholder_a: data}))  # Will succeed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Code Explanation*\n",
    "\n",
    "-   `import `numpy`` as np`: Import numpy library to create the data\n",
    "\n",
    "-   `tf.pow(data_placeholder_a, 2)`: Create the ops\n",
    "\n",
    "-   `np.random.rand(1, 10)`: Create a random array of data\n",
    "\n",
    "-   `feed_dict={data_placeholder_a: data}`: Feed the placeholder\n",
    "    with `data`\n",
    "\n",
    "\n",
    "## Graph\n",
    "\n",
    "\n",
    "TensorFlow depends on a genius approach to render the operation. All the\n",
    "computations are represented with a dataflow scheme. The dataflow graph\n",
    "has been developed to see to data dependencies between individual\n",
    "operation. Mathematical formula or algorithm are made of a number of\n",
    "successive operations. A graph is a convenient way to visualize how the\n",
    "computations are coordinated.\n",
    "\n",
    "The graph shows a **node** and an **edge**. The node is the\n",
    "representation of a operation, i.e. the unit of computation. The edge is\n",
    "the tensor, it can produce a new tensor or consume the input data. It\n",
    "depends on the dependencies between individual operation.\n",
    "\n",
    "The structure of the graph connects together the operations (i.e. the\n",
    "nodes) and how those are operation are feed. Note that the graph does\n",
    "not display the output of the operations, it only helps to visualize the\n",
    "connection between individual operations.\n",
    "\n",
    "Let’s see an example.\n",
    "\n",
    "Imagine you want to evaluate the following function:\n",
    "\n",
    "$$f(x,z) = xz + x^{2} + z + 5$$\n",
    "\n",
    "TensorFlow will create a graph to execute the function. The graph looks\n",
    "like this:\n",
    "\n",
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/11_introduction-to-tensorflow_files/image054.png)\n",
    "\n",
    "You can easily see the path that the tensors will take to reach the\n",
    "final destination.\n",
    "\n",
    "For instance, you can see the operation `add` cannot be done before\n",
    "$\\text{xz}$ and $x^{2}$. The graph explains that it will:\n",
    "\n",
    "- 1) compute $xz$ and $x_2$:  \n",
    "- 2) add 1) together\n",
    "- 3) add $z$ to 2)\n",
    "- 4) add 3)  to $c$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.get_variable(\"x\", dtype=tf.int32,\n",
    "      initializer=tf.constant([5]))\n",
    "z = tf.get_variable(\"z\", dtype=tf.int32,\n",
    "      initializer=tf.constant([6]))\n",
    "c = tf.constant([5], name = \"constant\")\n",
    "square = tf.constant([2], name = \"square\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "f =  tf.multiply(x, z) + tf.pow(x, square) + z + c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Code Explanation*\n",
    "\n",
    "-   `x`: Initialize a variable called `x` with a constant value of 5\n",
    "\n",
    "-   `z`: Initialize a variable called `z` with a constant value of 6\n",
    "\n",
    "-   `c`: Initialize a constant tensor called `c` with a constant value\n",
    "    of 5\n",
    "\n",
    "-   `square`: Initialize a constant tensor called `square` with a\n",
    "    constant value of 2\n",
    "\n",
    "-   `f`: Construct the operator\n",
    "\n",
    "In this example, we choose to keep the values of the variables fixed. We\n",
    "also created a constant tensor called `c` which is the constant\n",
    "parameter in the function `f`. It takes a fixed value of 5. In the\n",
    "graph, you can see this parameter in the tensor called `constant`.\n",
    "\n",
    "We also constructed a constant tensor for the power in the operator\n",
    "`tf.pow()`. It is not necessary. We did it so that you can see the\n",
    "name of the tensor in the graph. It is the circle called `square`.\n",
    "\n",
    "From the graph, you can understand what will happen of the tensors and\n",
    "how it can return an output of 66.\n",
    "\n",
    "The code below evaluate the function in a session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[66]\n"
     ]
    }
   ],
   "source": [
    "init = tf.global_variables_initializer() # prepare to initialize all variables\n",
    "with tf.Session() as sess:\n",
    "    init.run() # Initialize x and y\n",
    "    function_result = f.eval()\n",
    "print(function_result)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "TensorFlow works around: \n",
    "    \n",
    "\n",
    "- Graph: Computational environment containing the operations and tensors    \n",
    "- Tensors: Represents the data (or value) that will flow in the graph. It is the edge in the graph\n",
    "- Sessions: Allow the execution of the operations \n",
    "\n",
    "Create a constant tensor\n",
    "\n",
    "| constant | object                                              |\n",
    "| -------- | --------------------------------------------------- |\n",
    "| D0       | `tf.constant(1, tf.int16)`                            |\n",
    "| D1       | `tf.constant([1,3,5], tf.int16)`                      |\n",
    "| D2       | `tf.constant([ [1, 2], [3, 4] ],tf.int16)`            |\n",
    "| D3       | `tf.constant([ [[1, 2],[3, 4],  [5, 6]] ], tf.int16)` |\n",
    "\n",
    "Create an operator\n",
    "\n",
    "| Create an operator | Object            |\n",
    "| ------------------ | ----------------- |\n",
    "| a+b                | `tf.add(a, b)`      |\n",
    "| a*b                | `tf.multiply(a, b)` |\n",
    "\n",
    "Create a variable tensor\n",
    "\n",
    "| Create a variable       | object                                                       |\n",
    "| ----------------------- | ------------------------------------------------------------ |\n",
    "| randomized value        | `tf.get_variable(\"var\", [1, 2])`                               |\n",
    "| initialized first value | `tf.get_variable(\"var_init_2\", dtype=tf.int32, initializer=[ [1, 2], [3, 4] ])` |\n",
    "\n",
    "Open a session\n",
    "\n",
    "| Session           | object                     |\n",
    "| ----------------- | -------------------------- |\n",
    "| Create a session  | `tf.Session()`              |\n",
    "| Run a session     | `tf.Session.run()`           |\n",
    "| Evaluate a tensor | `variable_name.eval()`       |\n",
    "| Close a session   | `sess.close()`               |\n",
    "| Session by block  | `with tf.Session() as sess:` |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}