08_EagerExecution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "U9i2Dsh-ziXr"
   },
   "source": [
    "# Eager Execution Tutorial: Basics\n",
    "\n",
    "This notebook introduces the basics of using TensorFlow's eager execution capabilities. It covers concepts such as:\n",
    "\n",
    "* Importing required packages\n",
    "* Enabling eager execution\n",
    "* Creating and using TensorFlow Tensors and Variables\n",
    "* Using TensorFlow interactively\n",
    "* Using GPUs with eager execution enabled\n",
    "\n",
    "This notebook does *not* cover modeling topics, such as gradients."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "z1JcS5iBXMRO"
   },
   "source": [
    "# Step 1: Import Eager\n",
    "\n",
    "The key imports for eager execution are the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "RlIWhyeLoYnG"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow.contrib.eager as tfe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "H9UySOPLXdaw"
   },
   "source": [
    "# Step 2: Enable eager execution\n",
    "\n",
    "All future TensorFlow calls will execute the\n",
    "underlying TensorFlow ops immediately:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "WPTUfGq6kJ5w"
   },
   "outputs": [],
   "source": [
    "tfe.enable_eager_execution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "twBfWd5xyu_d"
   },
   "source": [
    "# Step 3: Interactively Use TensorFlow!\n",
    "\n",
    "Now you can call TensorFlow functions and get results, immediately! No more `tf.Sessions`!\n",
    "\n",
    "TensorFlow will automatically wrap native Python types for you with operator overloading for TensorFlow Tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "ngUe237Wt48W"
   },
   "outputs": [],
   "source": [
    "print(tf.add(1, 2))\n",
    "print(tf.add([1, 2], [3, 4]))\n",
    "print(tf.square(5))\n",
    "print(tf.reduce_sum([1, 2, 3]))\n",
    "print(tf.encode_base64(\"hello world\"))\n",
    "print(\"\")\n",
    "\n",
    "x = tf.constant(2)\n",
    "y = tf.constant(3)\n",
    "print(x * y + 1)\n",
    "\n",
    "# Most TensorFlow ops are directly usable with eager execution, giving\n",
    "# results immediately.\n",
    "print(tf.contrib.signal.hamming_window(x * y + 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IDY4WsYRhP81"
   },
   "source": [
    "Numpy arrays are supported, too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "lCUWzso6mbqR"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "ones = np.ones([3, 3])\n",
    "\n",
    "print(\"numpy 3x3 matrix of 1s:\")\n",
    "print(ones)\n",
    "print(\"\")\n",
    "\n",
    "print(\"Multiplied by 42:\")\n",
    "print(tf.multiply(ones, 42))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PBNP8yTRfu_X"
   },
   "source": [
    "# Step 4: Define and Print TensorFlow Variables\n",
    "\n",
    "To define TensorFlow variables, use the `get_variable()` function as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "3Twf_Rw-gQFM"
   },
   "outputs": [],
   "source": [
    "x = tf.get_variable(name=\"x\", shape=[], dtype=tf.float32, initializer=tf.zeros_initializer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "45G7094TxsMb"
   },
   "source": [
    "## Printing TensorFlow Variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "UJBJeZ5XxuwA"
   },
   "outputs": [],
   "source": [
    "# This does NOT print the Variable's actual value:\n",
    "print(\"Printing a TensorFlow Variable:\")\n",
    "print(x)\n",
    "print(\"\")\n",
    "\n",
    "# A TensorFlow variable represents a reference to a tensor.\n",
    "# The `read_value()` method provides access to the current value of the\n",
    "# variable. Tensorflow Variables are automatically initialized according to the\n",
    "# semantics defined in tf.get_variable().\n",
    "print(\"Printing a TensorFlow Variable's value using .read_value():\")\n",
    "print(x.read_value())\n",
    "print(\"\")\n",
    "\n",
    "print(\"Printing a TensorFlow Variable's value using .read_value().numpy():\")\n",
    "print(x.read_value().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2njjWHcTpBEn"
   },
   "source": [
    "## Changing a TensorFlow Variable's value\n",
    "\n",
    "To change a TensorFlow Variable's value, use its `.assign()` or `.assign_add()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "v3wr6Erbo_hB"
   },
   "outputs": [],
   "source": [
    "x.assign(42)\n",
    "print(x.read_value())\n",
    "\n",
    "x.assign_add(3)\n",
    "print(x.read_value())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uhtynjHVpTB5"
   },
   "source": [
    "## Use a Variable just like any other Tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "7PbktdnHoehR"
   },
   "outputs": [],
   "source": [
    "print(x + 3)\n",
    "\n",
    "# This code will broadcast the value across the list of numbers:\n",
    "print(x * [1, 2, 4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GVChqwlwy1SI"
   },
   "source": [
    "# Step 5: Debug Errors with Instant Feedback\n",
    "\n",
    "TensorFlow's eager execution helps you identify and debug runtime issues through interactive exploration of code snippets.\n",
    "\n",
    "Below, we'll define a length-4 vector, and attempt two `tf.slice()` operations,\n",
    "one being legal and the other being illegal, leading to a runtime error that is\n",
    "raised immediately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "23ap04N0v4k0"
   },
   "outputs": [],
   "source": [
    "vector = tf.constant([10.0, 20.0, 30.0, 40.0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "FCUMsIYxxRRa"
   },
   "outputs": [],
   "source": [
    "# Works, because the values of `begin` and `size` (the 2nd and 3rd input\n",
    "# arguments) are within the bound of `vector`.\n",
    "print(tf.slice(vector, [1], [3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "code",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "T8me2oCNxpFp"
   },
   "outputs": [],
   "source": [
    "# The following does NOT work, because the value of `size` (the 3rd\n",
    "# argument) causes the indices to go out of the bounds of `vector`. The\n",
    "# error is raised immediately.\n",
    "try:\n",
    "  print(tf.slice(vector, [1], [4]))\n",
    "except tf.OpError as e:\n",
    "  print(\"Caught error: %s\" % e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "irxJhAgar84v"
   },
   "source": [
    "# Step 6: Timing a Matrix Multiply\n",
    "\n",
    "You can place Tensors on the GPU by calling a Tensor's `.gpu()` method.\n",
    "\n",
    "The first operation executing on the GPU may be slow as TensorFlow initializes. Subsequent uses will be much faster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "7J4N9baqaKCL"
   },
   "outputs": [],
   "source": [
    "# Create some Tensors\n",
    "SIZE = 1000\n",
    "cpu_tensor = tf.random_normal([SIZE, SIZE])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "4E-2n7VbzY1n"
   },
   "outputs": [],
   "source": [
    "# Time a CPU-based matrix multiplication\n",
    "\n",
    "print(\"Time to conduct matmul on CPU:\")\n",
    "%time tf.matmul(cpu_tensor, cpu_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "Eager Execution Tutorial: Basics",
   "provenance": [
    {
     "file_id": "0B0kLcpwLFwKEVm9XNkFueGk4bTg",
     "timestamp": 1504118841551
    }
   ],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}