notebooks/tf2.keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run this notebook in IBM Watson Studio (for free), click on the button below and then click on the same button again (top right).\n",
    "\n",
    "In case you need a (free, never expiring, no credit card needed) IBM Cloud account, you can get one here: [ibm.biz/coursera](https://ibm.biz/coursera) (tracked URL)\n",
    "\n",
    "[![](https://github.com/romeokienzler/TensorFlow/raw/master/images/playbutton.png)](https://dataplatform.cloud.ibm.com/analytics/notebooks/v2/d94668e2-3d68-4ee2-bc87-00a885803726/view?access_token=76adb4b998e7cb7bed5e53b0c1a99fb99a0b0a7a24498881e766c094845f0366)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to install TensorFlow 2.x - as of writing of this notebook, there is only an alpha version available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install tensorflow==2.0.0-alpha0 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now just make sure you restart the Kernel so that the changes take effect:\n",
    "\n",
    "![](https://github.com/romeokienzler/TensorFlow/raw/master/images/restart_kernel.png)\n",
    "\n",
    "After the kernel has been restarted, we'll check if we are on TensorFlow 2.x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.0.0-alpha0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So this worked out. Now it's time to create and run a keras model. Since MNIST is getting boring, let's use the fashion MNIST dataset. If you want to learn more on the data set check out the following links\n",
    "\n",
    "[MNIST](https://en.wikipedia.org/wiki/MNIST_database)  \n",
    "[FashionMNIST](https://github.com/zalandoresearch/fashion-mnist)\n",
    "\n",
    "So in a nutsthell, MNIST contains 60000 28x28 pixel grey scale images of handwritten digits between 0-9 and the corresponding labels. Plus additional 10000 images for testing.\n",
    "\n",
    "Fashing MNIST contains 60000 28x28 pixel grey scale images of fashion articles and the corresponding labels between 0-9. It also contains 10000 test images.\n",
    "\n",
    "Luckyly, this data set is built in to Keras, so let's load it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
      "32768/29515 [=================================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
      "26427392/26421880 [==============================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
      "8192/5148 [===============================================] - 0s 0us/step\n",
      "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
      "4423680/4422102 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "fashion_mnist = tf.keras.datasets.fashion_mnist\n",
    "\n",
    "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, we get 60000 images of 28 by 28 pixels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_images.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The labels are simply a list of 60000 elements, each one is a number (label) between 0 and 9:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000,)\n",
      "[9 0 0 ... 3 0 5]\n"
     ]
    }
   ],
   "source": [
    "print(train_labels.shape)\n",
    "print(train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at one image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFFhJREFUeJzt3WtwlFWaB/D/053OhdABwiUgRvGCCqMrOhFUphxHRgcta9FxtLQsF6uswdrVqZ1ZP2ixszXuh92yrFXXWndmNyorVo3OpUZXx6IcNa7ilSEiKwqLKERAIAlEkpCkk748+yHNTICc52369jae/6+KIumnT/qku/95u/u85xxRVRCRfyJhd4CIwsHwE3mK4SfyFMNP5CmGn8hTDD+Rpxh+Ik8x/ESeYviJPFVVzhurlhqtRX05b5LIKwkMYESHJZfrFhR+EVkK4FEAUQBPqOoD1vVrUY9FsqSQmyQiwzpty/m6eb/sF5EogH8HcDWA+QBuEZH5+f48IiqvQt7zLwTwmapuV9URAL8CsKw43SKiUisk/LMB7Brz/e7sZUcQkRUi0i4i7UkMF3BzRFRMhYR/vA8VjpkfrKqtqtqiqi0x1BRwc0RUTIWEfzeA5jHfnwxgT2HdIaJyKST86wHMFZHTRKQawM0AXixOt4io1PIe6lPVlIjcDeAPGB3qW6WqnxStZ0RUUgWN86vqGgBritQXIiojnt5L5CmGn8hTDD+Rpxh+Ik8x/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTDD+Rpxh+Ik8x/ESeKuvS3RQCCVjFWY9ZfOm4RKc2mvWvvneWs9bwzPsF3XbQ7yZVMWdNkyOF3Xahgh4XS4GP2WE88hN5iuEn8hTDT+Qphp/IUww/kacYfiJPMfxEnuI4/9ecRKNmXVMpsx5ZYO+9uuXOiXb7IXctNrDQbFs1lDHrsVfazXpBY/lB5xAE3K8Q+7haSN+kyoit/XAegUd+Ik8x/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTBY3zi0gHgH4AaQApVW0pRqeoeMwxYQSP8+/63mSzfuslb5n1d7pPd9a+qJlpttU6s4yq715i1s/6+ZfOWqpjp/3DA+bMB91vQaJTpriL6bTZNt3X5y4ex1T/Ypzk8x1V3V+En0NEZcSX/USeKjT8CuAVEflARFYUo0NEVB6FvuxfrKp7RGQGgFdF5P9Ude3YK2T/KKwAgFpMKPDmiKhYCjryq+qe7P9dAJ4HcMxMDVVtVdUWVW2JoaaQmyOiIso7/CJSLyLxw18DuArAx8XqGBGVViEv+5sAPC+jUx+rADyjqi8XpVdEVHJ5h19VtwM4v4h9oRLIJBIFtR+54JBZ/8Eke059bSTprL0Zsefrf/l6s1lP/4Xdty8ejjtrmQ8vNdtO/dgea2/4cK9Z33/ZbLPe/U33gHxTwHYGU1773FmTntwjzaE+Ik8x/ESeYviJPMXwE3mK4SfyFMNP5CnRIm33m4sGadRFsqRst+cNa5npgMf30E0Xm/Wrf/qGWZ9Xu8es92dqnbURLezs8se2ftusD2yf5KxFRgK2yA4op5vspbc1aR9Xp2xw/+51yzrNtvL4dGfto7ZHcahnV077f/PIT+Qphp/IUww/kacYfiJPMfxEnmL4iTzF8BN5iuP8lSBgO+iCBDy+535g//3//hR7ym6QqLGW9IBWm20PpusLuu3ulHtKbzLgHIMnttlTfg8Z5xAAQCRlP6ZXfudDZ+2GxvVm2wfPOM9ZW6dt6NMejvMTkRvDT+Qphp/IUww/kacYfiJPMfxEnmL4iTxVjF16qVBlPNfiaNsOzTDrBxommvV9KXsL76lR9/La8ciQ2XZOzN78uTvtHscHgGjMvTT4iEbNtv/4jd+b9cS8mFmPib3096XGOgg3bv4rs209tpv1XPHIT+Qphp/IUww/kacYfiJPMfxEnmL4iTzF8BN5KnCcX0RWAbgWQJeqnpu9rBHArwHMAdAB4CZV/ap03aRSmV5jb3NdK+4ttgGgWlJmfU9yirO2behss+2nffY5CEubPjHrSWMs31pnAAgepz8pZj/dE2qfB2Ddq4ub7HH8jWY1d7kc+Z8CsPSoy+4D0KaqcwG0Zb8nohNIYPhVdS2AnqMuXgZgdfbr1QCuK3K/iKjE8n3P36SqewEg+7/9+oyIKk7Jz+0XkRUAVgBALSaU+uaIKEf5Hvk7RWQWAGT/73JdUVVbVbVFVVtiqMnz5oio2PIN/4sAlme/Xg7gheJ0h4jKJTD8IvIsgPcAnC0iu0XkDgAPALhSRLYBuDL7PRGdQALf86vqLY4SF+AvloB1+yVqzz3XlHusPTrFPc4OAN+evMmsd6cbzPrBtP05zuTooLPWn6o12/YM2T/7nJq9Zn3D4BxnbXq1PU5v9RsAOkammfW5NfvM+oOd7vg01x49uHak1JLLnDVd957Zdiye4UfkKYafyFMMP5GnGH4iTzH8RJ5i+Ik8xaW7K0HA0t1SZT9M1lDfrjvmmW2vmGAvUf1uYrZZn17Vb9atabWzanrNtvGmhFkPGmZsrHJPV+5P15ltJ0SGzXrQ731htb3s+E9eu9BZi597wGzbEDOO2cex2zuP/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTDD+RpzjOXwEkVm3WMwl7vNsybdOIWd+ftpeYnhyxp7ZWByxxbW2FfWnjDrNtd8BY/Iah08x6POreAnx6xB6nb47ZY+2bEs1mfc3AmWb9jmtfc9aebb3SbFv98rvOmqj9eI3FIz+Rpxh+Ik8x/ESeYviJPMXwE3mK4SfyFMNP5KkTa5zfWOJaquzxaokG/J2L2PVMwpjfnbHHuoNo0h6LL8Sj//mYWd+VmmzW9yXtetAS12ljgvn7Q5PMtrURe3vw6VV9Zr0vY58nYOnP2MuKW+sUAMF9v3fqNmftud7vmm2LhUd+Ik8x/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTgeP8IrIKwLUAulT13Oxl9wP4IYDu7NVWquqaQjtTyPr0QWPlag+7hmpo2UKzvus6+zyCWy/4o7O2LxU3235obGMNAJOMOfEAUB+wvn1C3edf7Bmxtw8PGiu31uUHgBnGeQBptY97XybtvgUJOv9hd8rYU+Av7bUGJj+dV5eOkcuR/ykAS8e5/BFVXZD9V3Dwiai8AsOvqmsB9JShL0RURoW8579bRD4SkVUiUthrJCIqu3zD/wsAZwBYAGAvgIdcVxSRFSLSLiLtSdjvD4mofPIKv6p2qmpaVTMAHgfg/MRKVVtVtUVVW2KoybefRFRkeYVfRGaN+fZ6AB8XpztEVC65DPU9C+ByANNEZDeAnwG4XEQWAFAAHQDuLGEfiagERAP2hi+mBmnURbKkbLc3VtWsmWY9eVqTWe+Z594LfnCmvSn6gmu2mPXbm942693pBrMeE/f5D0H70M+MHTTrr/fON+sTq+zPcazzBC6s6zDbHsy473MAOKnqK7N+72c/cNaaJthj6U+cao9eJzVj1rcm7be48Yj7vJS3Bu01/5+fP91ZW6dt6NMe+wmZxTP8iDzF8BN5iuEn8hTDT+Qphp/IUww/kacqaunu4asvMusz/n67s7agYbfZdn6dPZyWyNhLf1vTSzcPzTbbDmbsLbi3jdjDkL0pe8grKu5hp64Re0rvQzvsZaLbFv6HWf/pnvEmfP5ZpM49lHwgPdFse8NEe2luwH7M7jxlrbN2enWX2falgVlmfU/AlN+mWK9ZnxPrdta+H//UbPs83EN9x4NHfiJPMfxEnmL4iTzF8BN5iuEn8hTDT+Qphp/IU+Ud5xd7ee5F/7zebL4k/omzNqj2FMqgcfygcVvLpCp7mebhpH03dyXtKbtBzqrZ56xd37DRbLv2sUVm/VuJH5n1z6/4L7PeNuTeyro7Zf/eN++4wqxv2Nls1i+es8NZOy/+pdk26NyKeDRh1q1p1gAwkHE/X99P2Oc/FAuP/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTDD+Rp8q6dHfdzGY947a/c9Zb7/o3s/0zPRc7a8219l6ip1bvN+tTo/Z2z5Z4xB7zPTtmj/m+NHCyWX/j4Dlm/ZvxDmctJvb23pdP+Mys3/6Te8x6qtZeJbpvjvv4kqq3n3sN5x8w6z8683WzXm387gfT9jh+0P0WtAV3EGsNhnjE3hb9oWuud9be63gKvUN7uXQ3Ebkx/ESeYviJPMXwE3mK4SfyFMNP5CmGn8hTgfP5RaQZwNMAZgLIAGhV1UdFpBHArwHMAdAB4CZVNfdMjiSBCZ3u8c2X+haYfTm9zr3W+f6kvT79Hw6dZ9ZPrrO3e7a2mj7TmE8PABsTk836y93fMOsn1dnr13cmJzlrB5L1ZttBY145ADz5yMNm/aFOe93/6xs3OGvnV9vj+Acz9rFpc8B+B/2ZWmctofb6Dr0B5wHEjecDACTVjlbU2OJ7csQ+h6DvvKnOWroz9yU6cjnypwDco6rzAFwM4C4RmQ/gPgBtqjoXQFv2eyI6QQSGX1X3quqG7Nf9ALYAmA1gGYDV2autBnBdqTpJRMV3XO/5RWQOgAsArAPQpKp7gdE/EABmFLtzRFQ6OYdfRCYC+B2AH6tq0CZqY9utEJF2EWlPDQ/k00ciKoGcwi8iMYwG/5eq+lz24k4RmZWtzwIw7s6Hqtqqqi2q2lJVY3/4RETlExh+EREATwLYoqpjP/p9EcDy7NfLAbxQ/O4RUankMi6wGMBtADaJyOF1oFcCeADAb0TkDgA7AdwY9IOiIxnEdw076xm1ZyK+vt89tbWptt9suyC+y6xvHbSHjTYNneSsbag6xWxbF3Vv7w0Ak6rtKcH1Ve77DACmxdy/+2k19lbU1rRXAFifsH+3v57+hlnfmXIvif77gbPMtpsH3fc5AEwJWDJ9U5+7/WDK3jZ9OG1HI5Gyh44n1diP6UWNXzhrW2FvD959vjFN+h2z6RECw6+qbwNwpXJJ7jdFRJWEZ/gReYrhJ/IUw0/kKYafyFMMP5GnGH4iT5V3i+5DQ4i8+aGz/NtXFpvN/2HZb521NwOWt35pnz0u2zdiT22dPsF9anKDMc4OAI0x+7TmoC2+awO2e/4q5T5zcjhiT11NO0dxR+0bdk8XBoB3MnPNejLj3qJ72KgBwedH9IxMM+sn1fU6a/0p93RfAOjobzTr+3vtbbQTE+xovZ0+w1lbOtO9FT0A1HW5H7OI/VQ58rq5X5WIvk4YfiJPMfxEnmL4iTzF8BN5iuEn8hTDT+Spsm7R3SCNukjynwXce6t7i+7T/2ar2Xbh5B1mfUOfPW99pzHumwxYYjoWcS/TDAATYiNmvTZgvLs66p6TH4H9+GYCxvnro3bfgtYaaKhyz2uPR+057xFjG+tcRI3f/Y+9cwr62fGA3zul9nPikkmfO2urdlxqtp10jXtb9XXahj7t4RbdROTG8BN5iuEn8hTDT+Qphp/IUww/kacYfiJPlX+cP3qV+woZew35QgzcsMisL1q53q7H3eOy51R3mm1jsMerawPGs+sj9rBtwngMg/66vz3UbNbTAT/h9a/mmfWkMd7dOdhgto0Z5y/kwtoHYigVsEX3kD3fPxqxc5N4w15rYOpm97kbNWvs56KF4/xEFIjhJ/IUw0/kKYafyFMMP5GnGH4iTzH8RJ4KHOcXkWYATwOYCSADoFVVHxWR+wH8EEB39qorVXWN9bMKnc9fqeQie0+AoZl1Zr3mgD03vP9Uu33D5+59ASLD9kLumf/dYtbpxHI84/y5bNqRAnCPqm4QkTiAD0Tk1WztEVX9l3w7SkThCQy/qu4FsDf7db+IbAEwu9QdI6LSOq73/CIyB8AFANZlL7pbRD4SkVUiMsXRZoWItItIexL2y1siKp+cwy8iEwH8DsCPVbUPwC8AnAFgAUZfGTw0XjtVbVXVFlVticHeD4+Iyien8ItIDKPB/6WqPgcAqtqpqmlVzQB4HMDC0nWTiIotMPwiIgCeBLBFVR8ec/msMVe7HsDHxe8eEZVKLp/2LwZwG4BNIrIxe9lKALeIyAIACqADwJ0l6eEJQNdvMuv25NBgDe/m37awxa/p6yyXT/vfBsZd3N0c0yeiysYz/Ig8xfATeYrhJ/IUw0/kKYafyFMMP5GnGH4iTzH8RJ5i+Ik8xfATeYrhJ/IUw0/kKYafyFMMP5GnyrpFt4h0A/hizEXTAOwvWweOT6X2rVL7BbBv+Spm305V1em5XLGs4T/mxkXaVbUltA4YKrVvldovgH3LV1h948t+Ik8x/ESeCjv8rSHfvqVS+1ap/QLYt3yF0rdQ3/MTUXjCPvITUUhCCb+ILBWRrSLymYjcF0YfXESkQ0Q2ichGEWkPuS+rRKRLRD4ec1mjiLwqItuy/4+7TVpIfbtfRL7M3ncbReSakPrWLCL/IyJbROQTEfnb7OWh3ndGv0K538r+sl9EogA+BXAlgN0A1gO4RVU3l7UjDiLSAaBFVUMfExaRywAcAvC0qp6bvexBAD2q+kD2D+cUVb23Qvp2P4BDYe/cnN1QZtbYnaUBXAfgdoR43xn9ugkh3G9hHPkXAvhMVber6giAXwFYFkI/Kp6qrgXQc9TFywCszn69GqNPnrJz9K0iqOpeVd2Q/bofwOGdpUO974x+hSKM8M8GsGvM97tRWVt+K4BXROQDEVkRdmfG0ZTdNv3w9ukzQu7P0QJ3bi6no3aWrpj7Lp8dr4stjPCPt/tPJQ05LFbVCwFcDeCu7Mtbyk1OOzeXyzg7S1eEfHe8LrYwwr8bQPOY708GsCeEfoxLVfdk/+8C8Dwqb/fhzsObpGb/7wq5P39SSTs3j7ezNCrgvqukHa/DCP96AHNF5DQRqQZwM4AXQ+jHMUSkPvtBDESkHsBVqLzdh18EsDz79XIAL4TYlyNUys7Nrp2lEfJ9V2k7Xodykk92KONfAUQBrFLVfyp7J8YhIqdj9GgPjG5i+kyYfRORZwFcjtFZX50AfgbgvwH8BsApAHYCuFFVy/7Bm6Nvl2P0peufdm4+/B67zH37FoC3AGzCnzcqXonR99eh3XdGv25BCPcbz/Aj8hTP8CPyFMNP5CmGn8hTDD+Rpxh+Ik8x/ESeYviJPMXwE3nq/wHG6/IGFn5KEQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f87cb7a3fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure()\n",
    "plt.imshow(train_images[0])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So this is obviously a shoe :) - Let's normalize the data by making sure every pixel value is between 0..1; this is easy in this case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_images = train_images / 255.0\n",
    "\n",
    "test_images = test_images / 255.0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For those already familiar with Keras might notice, that mainly only the import statements changed. If you are new to Keras, just check out Week 2 of my DeepLearning course on Coursera, you can get it for free in audit mode: http://coursera.org/learn/ai/. So let's start with ordinary Keras by importing some requirements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras import Sequential\n",
    "from keras.layers import Flatten, Dense\n",
    "from keras.activations import relu, softmax"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to write our **hello world** softmax regression model, the following code does the job. If you are familiar with Keras, this is really basic stuff. There is only one catch. The following code doesn't run since the latest stable Keras version is incompatible with the alpha release of TensorFlow 2.0. So the following code is for illustration purposes only. Don't run it, it will destroy your hard drive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "module 'tensorflow' has no attribute 'get_default_graph'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-9-b03caac2a61c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m model = Sequential([\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0mFlatten\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m28\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m28\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m     \u001b[0mDense\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m128\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactivation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrelu\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mDense\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactivation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msoftmax\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m ])\n",
      "\u001b[0;32m/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/keras/layers/core.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m    481\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    482\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 483\u001b[0;31m         \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mFlatten\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    484\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minput_spec\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mInputSpec\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmin_ndim\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    485\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/keras/engine/topology.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m    294\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    295\u001b[0m             \u001b[0mprefix\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__class__\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 296\u001b[0;31m             \u001b[0mname\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_to_snake_case\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprefix\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'_'\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mK\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_uid\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprefix\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    297\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    298\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36mget_uid\u001b[0;34m(prefix)\u001b[0m\n\u001b[1;32m     66\u001b[0m     \"\"\"\n\u001b[1;32m     67\u001b[0m     \u001b[0;32mglobal\u001b[0m \u001b[0m_GRAPH_UID_DICTS\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 68\u001b[0;31m     \u001b[0mgraph\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_default_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     69\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mgraph\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0m_GRAPH_UID_DICTS\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     70\u001b[0m         \u001b[0m_GRAPH_UID_DICTS\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAttributeError\u001b[0m: module 'tensorflow' has no attribute 'get_default_graph'"
     ]
    }
   ],
   "source": [
    "model = Sequential([\n",
    "    Flatten(input_shape=(28, 28)),\n",
    "    Dense(128, activation=relu),\n",
    "    Dense(10, activation=softmax)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam', \n",
    "          loss='sparse_categorical_crossentropy',\n",
    "          metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_images, train_labels, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So from a migration and consistency perspective, it would be nice if we just could change the imports and leave our existing Keras code (which we all love) intact, so let's give it a try:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 9s 158us/sample - loss: 0.4980 - accuracy: 0.8248\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 9s 155us/sample - loss: 0.3724 - accuracy: 0.8668\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 9s 153us/sample - loss: 0.3368 - accuracy: 0.8774\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 9s 152us/sample - loss: 0.3144 - accuracy: 0.8845\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 9s 151us/sample - loss: 0.2921 - accuracy: 0.8925\n",
      "10000/10000 [==============================] - 1s 64us/sample - loss: 0.3627 - accuracy: 0.8732\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.3627416751146317, 0.8732]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.keras import Sequential\n",
    "from tensorflow.keras.layers import Flatten, Dense\n",
    "from tensorflow.nn import relu\n",
    "from tensorflow.nn import softmax\n",
    "\n",
    "\n",
    "    \n",
    "model = Sequential([\n",
    "    Flatten(input_shape=(28, 28)),\n",
    "    Dense(128, activation=relu),\n",
    "    Dense(10, activation=softmax)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam', \n",
    "          loss='sparse_categorical_crossentropy',\n",
    "          metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_images, train_labels, epochs=5)\n",
    "\n",
    "model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we didn't change the Keras code at all, but now all imports are coming from the tensorflow package. I felt a bit bad when I've noticed that TensorFlow has eaten up Keras, but in reality, nobody uses a Keras runtime other then TensorFlow anyway, so it doesn't really matter. Just be aware that Keras is Google now and part of TensorFlow. In the next notebook, we'll cover the stategies for parallel training. So stay tuned."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.5",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}