blogs/batch_predictions/batch_predictions_keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Passing through instance keys and features when using a keras model\n",
    "\n",
    "This notebook will show you how to modify a Keras model to perform keyed predictions or forward features through with the prediction. This it the companion code for [this blog](https://medium.com/p/5effc083265c/edit).\n",
    "\n",
    "Sometimes you'll have a unique instance key that is associated with each row and you want that key to be output along with the prediction so you know which row the prediction belongs to. You'll need to add keys when executing distributed batch predictions with a service like Cloud AI Platform batch prediction. Also, if you're performing continuous evaluation on your model and you'd like to log metadata about predictions for later analysis. There are also use-cases for forwarding a particular feature from your model out with the output, for example performing evaluation on certain slices of data.\n",
    "\n",
    "## Topics Covered\n",
    "- Modify serving signature of existing model to accept and forward keys\n",
    "- Multiple serving signatures on one model\n",
    "- Online and batch predictions with Google Cloud AI Platform\n",
    "- Forward features in model definition\n",
    "- Forward features with serving signature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.1.0-dlenv_tfe'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set GCP configs if using Cloud AI Platform\n",
    "\n",
    "import os\n",
    "\n",
    "PROJECT = \"your-gcp-project-here\" # REPLACE WITH YOUR PROJECT NAME\n",
    "REGION = \"us-central1\" # REPLACE WITH YOUR BUCKET REGION e.g. us-central1\n",
    "BUCKET = \"your-gcp-bucket-here\"\n",
    "\n",
    "# Do not change these\n",
    "os.environ[\"PROJECT\"] = PROJECT\n",
    "os.environ[\"REGION\"] = REGION\n",
    "os.environ[\"BUCKET\"] = PROJECT # DEFAULT BUCKET WILL BE PROJECT ID\n",
    "\n",
    "if PROJECT == \"your-gcp-project-here\":\n",
    "  print(\"Don't forget to update your PROJECT name! Currently:\", PROJECT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build and Train a Fashion MNIST model\n",
    "\n",
    "We will use a straightforward keras use case with the fashion mnist dataset to demonstrate building a model and then adding support for keyed predictions.\n",
    "More here on the use case:\n",
    "https://colab.sandbox.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/keras/classification.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "fashion_mnist = keras.datasets.fashion_mnist\n",
    "\n",
    "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale down dataset\n",
    "train_images = train_images / 255.0\n",
    "test_images = test_images / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "60000/60000 [==============================] - 9s 152us/sample - loss: 0.5210 - accuracy: 0.8182\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f44387647d0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Build and traing model\n",
    "\n",
    "from tensorflow.keras import Sequential, Input\n",
    "from tensorflow.keras.layers import Dense, Flatten\n",
    "\n",
    "model = Sequential([\n",
    "  Input(shape=(28,28), name=\"image\"),\n",
    "  Flatten(input_shape=(28, 28), name=\"flatten\"),\n",
    "  Dense(64, activation='relu', name=\"dense\"),\n",
    "  Dense(10, activation='softmax', name=\"preds\"),\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Only training for 1 epoch, we are not worried about model performance\n",
    "model.fit(train_images, train_labels, epochs=1, batch_size=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.9574834e-05, 1.7343391e-06, 1.5372832e-05, 6.3454769e-05,\n",
       "        4.5845241e-05, 4.0783577e-02, 1.1227881e-04, 4.5515549e-01,\n",
       "        1.0713221e-02, 4.9308944e-01]], dtype=float32)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create test_image in shape, type that will be accepted as Tensor\n",
    "test_image = np.expand_dims(test_images[0],0).astype('float32')\n",
    "model.predict(test_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SavedModel and serving signature\n",
    "\n",
    "Now save the model using tf.saved_model.save() into [SavedModel](https://www.tensorflow.org/guide/saved_model) format, not the older Keras [H5 Format](https://www.tensorflow.org/guide/keras/save_and_serialize). This will add a serving signature which we can then inspect. The serving signature indicates exactly which input names and types are expected, and what will be output by the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "INFO:tensorflow:Assets written to: ./model/assets\n"
     ]
    }
   ],
   "source": [
    "MODEL_EXPORT_PATH = './model/'\n",
    "tf.saved_model.save(model, MODEL_EXPORT_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['image'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 28, 28)\n",
      "      name: serving_default_image:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['preds'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 10)\n",
      "      name: StatefulPartitionedCall:0\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "!saved_model_cli show --tag_set serve --signature_def serving_default --dir {MODEL_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "_SignatureMap({'serving_default': <tensorflow.python.saved_model.load._WrapperFunction object at 0x7f441473b5d0>})"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the model from storage and inspect the object types\n",
    "loaded_model = tf.keras.models.load_model(MODEL_EXPORT_PATH)\n",
    "loaded_model.signatures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.saving.saved_model.load.Sequential at 0x7f441471c3d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's worth noting that original model did not have serving signature until we saved it and is a slightly different object type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.engine.sequential.Sequential at 0x7f443bab6fd0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment and expect an error since different object type\n",
    "# model.signatures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard serving function\n",
    "\n",
    "We can actually get access to the inference_function of the loaded model and is it directly to perform predictions, similar to a Keras Model.predict() call. Note the name of the output Tensor matches the serving signature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<tensorflow.python.saved_model.load._WrapperFunction object at 0x7f441473b5d0>\n"
     ]
    }
   ],
   "source": [
    "inference_function = loaded_model.signatures['serving_default']\n",
    "\n",
    "print(inference_function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'preds': <tf.Tensor: shape=(1, 10), dtype=float32, numpy=\n",
      "array([[1.9574834e-05, 1.7343391e-06, 1.5372832e-05, 6.3454769e-05,\n",
      "        4.5845241e-05, 4.0783577e-02, 1.1227881e-04, 4.5515549e-01,\n",
      "        1.0713221e-02, 4.9308944e-01]], dtype=float32)>}\n"
     ]
    }
   ],
   "source": [
    "result = inference_function(tf.convert_to_tensor(test_image))\n",
    "\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 10), dtype=float32, numpy=\n",
       "array([[1.9574834e-05, 1.7343391e-06, 1.5372832e-05, 6.3454769e-05,\n",
       "        4.5845241e-05, 4.0783577e-02, 1.1227881e-04, 4.5515549e-01,\n",
       "        1.0713221e-02, 4.9308944e-01]], dtype=float32)>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Matches serving signature\n",
    "result['preds']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Keyed Serving Function\n",
    "\n",
    "Now we'll create a new serving function that accepts and outputs a unique instance key. We use the fact that a Keras Model(x) call actually runs a prediction. The training=False parameter is included only for clarity. Then we save the model as before but provide this function as our new serving signature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "@tf.function(input_signature=[tf.TensorSpec([None], dtype=tf.string),tf.TensorSpec([None, 28, 28], dtype=tf.float32)])\n",
    "def keyed_prediction(key, image):\n",
    "    pred = loaded_model(image, training=False)\n",
    "    return {\n",
    "        'preds': pred,\n",
    "        'key': key\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./keyed_model/assets\n"
     ]
    }
   ],
   "source": [
    "# Resave model, but specify new serving signature\n",
    "KEYED_EXPORT_PATH = './keyed_model/'\n",
    "loaded_model.save(KEYED_EXPORT_PATH, signatures={'serving_default': keyed_prediction})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['image'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 28, 28)\n",
      "      name: serving_default_image:0\n",
      "  inputs['key'] tensor_info:\n",
      "      dtype: DT_STRING\n",
      "      shape: (-1)\n",
      "      name: serving_default_key:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['key'] tensor_info:\n",
      "      dtype: DT_STRING\n",
      "      shape: (-1)\n",
      "      name: StatefulPartitionedCall:0\n",
      "  outputs['preds'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 10)\n",
      "      name: StatefulPartitionedCall:1\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "!saved_model_cli show --tag_set serve --signature_def serving_default --dir {KEYED_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "keyed_model = tf.keras.models.load_model(KEYED_EXPORT_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.9574834e-05, 1.7343391e-06, 1.5372832e-05, 6.3454769e-05,\n",
       "        4.5845241e-05, 4.0783577e-02, 1.1227881e-04, 4.5515549e-01,\n",
       "        1.0713221e-02, 4.9308944e-01]], dtype=float32)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Change 'flatten_input' to 'image' after b/159022434\n",
    "keyed_model.predict({\n",
    "    'flatten_input': test_image,\n",
    "    'key': tf.constant(\"unique_key\")}\n",
    ")\n",
    "# keyed_model.predict(test_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple Signature Model\n",
    "\n",
    "Sometimes it is useful to leave both signatures in the model definition so the user can indicate if they are performing a keyed prediction or not. This can easily be done with the model.save() method as before.\n",
    "\n",
    "In general, your serving infrastructure will default to 'serving_default' unless otherwise specified in a prediction call. Google Cloud AI Platform online and batch prediction support multiple signatures, as does [TFServing](https://www.tensorflow.org/tfx/serving/api_rest#request_format_2)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./dual_signature_model/assets\n"
     ]
    }
   ],
   "source": [
    "# Using inference_function from earlier\n",
    "DUAL_SIGNATURE_EXPORT_PATH = './dual_signature_model/'\n",
    "loaded_model.save(DUAL_SIGNATURE_EXPORT_PATH, signatures={'serving_default': keyed_prediction,\n",
    "                                                  'unkeyed_signature': inference_function})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel MetaGraphDef contains SignatureDefs with the following keys:\n",
      "SignatureDef key: \"__saved_model_init_op\"\n",
      "SignatureDef key: \"serving_default\"\n",
      "SignatureDef key: \"unkeyed_signature\"\n"
     ]
    }
   ],
   "source": [
    "# Examine the multiple signatures\n",
    "!saved_model_cli show --tag_set serve --dir {DUAL_SIGNATURE_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['image'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 28, 28)\n",
      "      name: serving_default_image:0\n",
      "  inputs['key'] tensor_info:\n",
      "      dtype: DT_STRING\n",
      "      shape: (-1)\n",
      "      name: serving_default_key:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['key'] tensor_info:\n",
      "      dtype: DT_STRING\n",
      "      shape: (-1)\n",
      "      name: StatefulPartitionedCall:0\n",
      "  outputs['preds'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 10)\n",
      "      name: StatefulPartitionedCall:1\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "# Default signature\n",
    "!saved_model_cli show --tag_set serve --signature_def serving_default --dir {DUAL_SIGNATURE_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['image'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 28, 28)\n",
      "      name: unkeyed_signature_image:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['preds'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 10)\n",
      "      name: StatefulPartitionedCall_1:0\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "# Alternative unkeyed signature\n",
    "!saved_model_cli show --tag_set serve --signature_def unkeyed_signature --dir {DUAL_SIGNATURE_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the model and perform predictions\n",
    "\n",
    "Now we'll deploy the model to AI Platform serving and perform both online and batch keyed predictions. Deployment will take 2-3 minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"MODEL_LOCATION\"] = DUAL_SIGNATURE_EXPORT_PATH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fashion_mnist v1\n",
      "Model fashion_mnist already exists\n",
      "v1 gs://dhodun1/d42f33c1391fa46ba5837061282b1c8567bfa204fe3db9f72c2a0da394f0ed4f/ READY\n",
      "Deleting version v1\n",
      "Creating version v1 from ./dual_signature_model/\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
      "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
      "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
      "This will delete version [v1]...\n",
      "\n",
      "Do you want to continue (Y/n)?  \n",
      "Deleting version [v1]......\n",
      "..............................................................................................................................................................done.\n",
      "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
      "Creating version (this might take a few minutes)......\n",
      ".....................................................................................................................................................................................................................................................................................................................................................................................................................................done.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "MODEL_NAME=fashion_mnist\n",
    "MODEL_VERSION=v1\n",
    "\n",
    "TFVERSION=2.1\n",
    "# REGION and BUCKET and MODEL_LOCATION set earlier\n",
    "\n",
    "# create the model if it doesn't already exist\n",
    "modelname=$(gcloud ai-platform models list | grep -w \"$MODEL_NAME\")\n",
    "echo $modelname\n",
    "if [ -z \"$modelname\" ]; then\n",
    "   echo \"Creating model $MODEL_NAME\"\n",
    "   gcloud ai-platform models create ${MODEL_NAME} --regions $REGION\n",
    "else\n",
    "   echo \"Model $MODEL_NAME already exists\"\n",
    "fi\n",
    "\n",
    "# delete the model version if it already exists\n",
    "modelver=$(gcloud ai-platform versions list --model \"$MODEL_NAME\" | grep -w \"$MODEL_VERSION\")\n",
    "echo $modelver\n",
    "if [ \"$modelver\" ]; then\n",
    "   echo \"Deleting version $MODEL_VERSION\"\n",
    "   yes | gcloud ai-platform versions delete ${MODEL_VERSION} --model ${MODEL_NAME}\n",
    "   sleep 10\n",
    "fi\n",
    "\n",
    "\n",
    "echo \"Creating version $MODEL_VERSION from $MODEL_LOCATION\"\n",
    "gcloud ai-platform versions create ${MODEL_VERSION} \\\n",
    "       --model ${MODEL_NAME} --origin ${MODEL_LOCATION} --staging-bucket gs://${BUCKET} \\\n",
    "       --runtime-version $TFVERSION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create keyed test_image file\n",
    "\n",
    "with open(\"keyed_input.json\", \"w\") as file:\n",
    "    print(f'{{\"image\": {test_image.tolist()}, \"key\": \"image_id_1234\"}}', file=file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;33mWARNING:\u001b[0m Using endpoint [https://ml.googleapis.com/]\n",
      "KEY            PREDS\n",
      "image_id_1234  [1.9574799807742238e-05, 1.7343394347335561e-06, 1.537282150820829e-05, 6.345478323055431e-05, 4.584524504025467e-05, 0.0407835878431797, 0.00011227882350794971, 0.4551553726196289, 0.010713225230574608, 0.493089497089386]\n"
     ]
    }
   ],
   "source": [
    "# Single online keyed prediction, --signature-name is not required since we're hitting the default but shown for clarity\n",
    "\n",
    "!gcloud ai-platform predict --model fashion_mnist --json-instances keyed_input.json --version v1 --signature-name serving_default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create unkeyed test_image file\n",
    "\n",
    "with open(\"unkeyed_input.json\", \"w\") as file:\n",
    "    print(f'{{\"image\": {test_image.tolist()}}}', file=file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;33mWARNING:\u001b[0m Using endpoint [https://ml.googleapis.com/]\n",
      "PREDS\n",
      "[1.9574799807742238e-05, 1.7343394347335561e-06, 1.537282150820829e-05, 6.345478323055431e-05, 4.584524504025467e-05, 0.0407835878431797, 0.00011227882350794971, 0.4551553726196289, 0.010713225230574608, 0.493089497089386]\n"
     ]
    }
   ],
   "source": [
    "# Single online unkeyed prediction using alternative serving signature\n",
    "\n",
    "!gcloud ai-platform predict --model fashion_mnist --json-instances unkeyed_input.json --version v1 --signature-name unkeyed_signature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch Predictions\n",
    "\n",
    "Now we'll create multiple keyed prediction files and create a job to perform these predictions in a scalable, distributed manner. The keys will be retained so the results can be stored and associated with the initial inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Data files:\n",
    "import shutil\n",
    "\n",
    "DATA_DIR = './batch_data'\n",
    "shutil.rmtree(DATA_DIR, ignore_errors=True)\n",
    "os.makedirs(DATA_DIR)\n",
    "\n",
    "# Create 10 files with 10 images each\n",
    "for i in range(10):\n",
    "    with open(f'{DATA_DIR}/keyed_batch_{i}.json', \"w\") as file:\n",
    "        for z in range(10):\n",
    "            key = f'key_{i}_{z}'\n",
    "            print(f'{{\"image\": {test_images[z].tolist()}, \"key\": \"{key}\"}}', file=file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Copying file://./batch_data/keyed_batch_1.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_9.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_8.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_3.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_5.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_0.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_7.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_2.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_4.json [Content-Type=application/json]...\n",
      "Copying file://./batch_data/keyed_batch_6.json [Content-Type=application/json]...\n",
      "/ [10/10 files][870.0 KiB/870.0 KiB] 100% Done                                  \n",
      "Operation completed over 10 objects/870.0 KiB.                                   \n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "gsutil -m cp -r ./batch_data gs://$BUCKET/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This following batch prediction job took me 8-10 minutes, most of the time spent in infrastructure spin up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jobId: fashion_mnist_batch_predict_20200615_153958\n",
      "state: QUEUED\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Job [fashion_mnist_batch_predict_20200615_153958] submitted successfully.\n",
      "Your job is still active. You may view the status of your job with the command\n",
      "\n",
      "  $ gcloud ai-platform jobs describe fashion_mnist_batch_predict_20200615_153958\n",
      "\n",
      "or continue streaming the logs with the command\n",
      "\n",
      "  $ gcloud ai-platform jobs stream-logs fashion_mnist_batch_predict_20200615_153958\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "DATA_FORMAT=\"text\" # JSON data format\n",
    "INPUT_PATHS=\"gs://${BUCKET}/batch_data/*\"\n",
    "OUTPUT_PATH=\"gs://${BUCKET}/batch_predictions\"\n",
    "MODEL_NAME='fashion_mnist'\n",
    "VERSION_NAME='v1'\n",
    "now=$(date +\"%Y%m%d_%H%M%S\")\n",
    "JOB_NAME=\"fashion_mnist_batch_predict_$now\"\n",
    "LABELS=\"team=engineering,phase=test,owner=drew\"\n",
    "SIGNATURE_NAME=\"serving_default\"\n",
    "\n",
    "gcloud ai-platform jobs submit prediction $JOB_NAME \\\n",
    "    --model $MODEL_NAME \\\n",
    "    --version $VERSION_NAME \\\n",
    "    --input-paths $INPUT_PATHS \\\n",
    "    --output-path $OUTPUT_PATH \\\n",
    "    --region $REGION \\\n",
    "    --data-format $DATA_FORMAT \\\n",
    "    --labels $LABELS \\\n",
    "    --signature-name $SIGNATURE_NAME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can stream the logs, this cell will block until the job completes.\n",
    "# Copy and paste from the previous cell's output based to grab your job name\n",
    "\n",
    "# gcloud ai-platform jobs stream-logs fashion_mnist_batch_predict_20200611_151356"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gs://dhodun1/temp/batch_predictions/prediction.errors_stats-00000-of-00001\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00000-of-00001\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00000-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00001-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00002-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00003-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00004-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00005-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00006-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00007-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00008-of-00010\n",
      "gs://dhodun1/temp/batch_predictions/prediction.results-00009-of-00010\n"
     ]
    }
   ],
   "source": [
    "!gsutil ls gs://$BUCKET/batch_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"preds\": [9.211135329678655e-05, 1.0087987902807072e-05, 4.698108750744723e-05, 2.715539994824212e-05, 0.00019967503612861037, 0.174049511551857, 0.0005865175626240671, 0.2705112099647522, 0.002678860677406192, 0.5517978668212891], \"key\": \"key_7_0\"}\n",
      "{\"preds\": [0.002726481296122074, 7.680333510506898e-06, 0.7650008797645569, 0.0001010186315397732, 0.038841612637043, 3.7204465286322375e-08, 0.19325067102909088, 1.9686152707976134e-09, 7.169554010033607e-05, 1.4934888395434776e-11], \"key\": \"key_7_1\"}\n",
      "{\"preds\": [0.00010154004121432081, 0.9996353387832642, 7.832989467715379e-06, 0.0002033840719377622, 4.973643081029877e-05, 2.2773709584811286e-09, 1.5637542674085125e-06, 1.9033164377901812e-08, 1.7880319092000718e-07, 4.1070393308473285e-07], \"key\": \"key_7_2\"}\n",
      "{\"preds\": [3.554671638994478e-05, 0.9984777569770813, 1.888977931230329e-05, 0.0013873657444491982, 6.41861479380168e-05, 2.0880989382021653e-08, 2.1890855350648053e-06, 1.9535197282039007e-07, 4.799605335392698e-07, 1.3420240065897815e-05], \"key\": \"key_7_3\"}\n",
      "{\"preds\": [0.08514387905597687, 0.00020077414228580892, 0.0692749172449112, 0.06048263981938362, 0.06608160585165024, 7.142244430724531e-05, 0.714330792427063, 2.0437615603441373e-05, 0.004391715861856937, 1.7476610310040996e-06], \"key\": \"key_7_4\"}\n",
      "{\"preds\": [0.004108963534235954, 0.9902229309082031, 0.0002155099791707471, 0.0025740088894963264, 0.0025837370194494724, 8.710831167491051e-08, 0.00028743583243340254, 8.160312603422426e-08, 5.302461886458332e-06, 2.0205186501698336e-06], \"key\": \"key_7_5\"}\n",
      "{\"preds\": [0.014112361706793308, 0.0008088137838058174, 0.01675746962428093, 0.002164569217711687, 0.9211376309394836, 0.00014193437527865171, 0.041439879685640335, 7.985366210050415e-06, 0.0034278519451618195, 1.5230604049065732e-06], \"key\": \"key_7_6\"}\n",
      "{\"preds\": [0.0018075327388942242, 0.0005742131033912301, 0.013851719908416271, 0.0030816274229437113, 0.1138634979724884, 0.000244705326622352, 0.8649102449417114, 5.37119649379747e-06, 0.0016507108230143785, 1.044568489305675e-05], \"key\": \"key_7_7\"}\n",
      "{\"preds\": [0.006706225220113993, 0.003986412659287453, 0.007010308559983969, 0.006693772505968809, 0.005332822911441326, 0.8451583981513977, 0.018238505348563194, 0.08709842711687088, 0.014139154925942421, 0.00563598470762372], \"key\": \"key_7_8\"}\n",
      "{\"preds\": [2.508779289200902e-05, 2.406256135145668e-05, 3.5246364859631285e-05, 3.418887354200706e-05, 2.4260680220322683e-05, 0.0027981558814644814, 9.72664711298421e-05, 0.9939435124397278, 0.0019495101878419518, 0.0010686515597626567], \"key\": \"key_7_9\"}\n"
     ]
    }
   ],
   "source": [
    "# View predictions with keys\n",
    "!gsutil cat gs://$BUCKET/batch_predictions/prediction.results-00000-of-00010"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Forward Models\n",
    "\n",
    "There are also times where it's desirable to forward some or all of the input features along with the output. This can be achieved in a very similar manner as adding keyed outputs to our model.\n",
    "\n",
    "Note that this will be a little trickier to grab a subset of features if you are feeding all of your input features as a single Input() layer in the Keras model. This example takes multiple Inputs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build and train Boston Housing model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build a toy model using the Boston Housing dataset\n",
    "# https://www.kaggle.com/c/boston-housing\n",
    "# Prediction target is median value of homes in $1000's\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) = keras.datasets.boston_housing.load_data()\n",
    "\n",
    "# Extract just two of the features for simplicity's sake\n",
    "train_tax_rate = train_data[:,10]\n",
    "train_rooms = train_data[:,5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build a toy model with multiple inputs\n",
    "# This time using the Keras functional API\n",
    "\n",
    "from tensorflow.keras.layers import Input\n",
    "from tensorflow.keras import Model\n",
    "\n",
    "\n",
    "tax_rate = Input(shape=(1,), dtype=tf.float32, name=\"tax_rate\")\n",
    "rooms = Input(shape=(1,), dtype=tf.float32, name=\"rooms\")\n",
    "\n",
    "x = tf.keras.layers.Concatenate()([tax_rate, rooms])\n",
    "x = tf.keras.layers.Dense(64, activation='relu')(x)\n",
    "price = tf.keras.layers.Dense(1, activation=None, name=\"price\")(x)\n",
    "\n",
    "# Functional API model instead of Sequential\n",
    "model = Model(inputs=[tax_rate, rooms], outputs=[price])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 404 samples\n",
      "Epoch 1/10\n",
      "404/404 [==============================] - 0s 1ms/sample - loss: 540.8722 - accuracy: 0.0000e+00\n",
      "Epoch 2/10\n",
      "404/404 [==============================] - 0s 89us/sample - loss: 423.0400 - accuracy: 0.0000e+00\n",
      "Epoch 3/10\n",
      "404/404 [==============================] - 0s 94us/sample - loss: 328.7296 - accuracy: 0.0000e+00\n",
      "Epoch 4/10\n",
      "404/404 [==============================] - 0s 107us/sample - loss: 255.8777 - accuracy: 0.0000e+00\n",
      "Epoch 5/10\n",
      "404/404 [==============================] - 0s 98us/sample - loss: 202.7813 - accuracy: 0.0000e+00\n",
      "Epoch 6/10\n",
      "404/404 [==============================] - 0s 99us/sample - loss: 162.3239 - accuracy: 0.0000e+00\n",
      "Epoch 7/10\n",
      "404/404 [==============================] - 0s 102us/sample - loss: 136.0128 - accuracy: 0.0000e+00\n",
      "Epoch 8/10\n",
      "404/404 [==============================] - 0s 104us/sample - loss: 119.6358 - accuracy: 0.0000e+00\n",
      "Epoch 9/10\n",
      "404/404 [==============================] - 0s 110us/sample - loss: 110.2340 - accuracy: 0.0000e+00\n",
      "Epoch 10/10\n",
      "404/404 [==============================] - 0s 101us/sample - loss: 105.8428 - accuracy: 0.0000e+00\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f4438703490>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.compile(\n",
    "        optimizer='adam',\n",
    "        loss='mean_squared_error',\n",
    "        metrics=['accuracy']\n",
    "    )\n",
    "# Again, we're not concerned with model performance\n",
    "model.fit([train_tax_rate, train_rooms], train_targets, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature forward and non feature forward predictions\n",
    "\n",
    "Using the Keras sequential API, we create another model with slightly different inputs and outputs, but retaining the weights of the existing model. Notice the predictions with and without feature forwarding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[22.294939]], dtype=float32)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict({\n",
    "    'tax_rate': tf.convert_to_tensor([20.2]),\n",
    "    'rooms': tf.convert_to_tensor([6.2])\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./boston_model/assets\n"
     ]
    }
   ],
   "source": [
    "BOSTON_EXPORT_PATH = './boston_model/'\n",
    "model.save(BOSTON_EXPORT_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Will retain weights from trained model but also forward out a feature\n",
    "forward_model = Model(inputs=[tax_rate, rooms], outputs=[price, tax_rate])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([[7.927329]], dtype=float32), array([[5.]], dtype=float32)]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Notice we get both outputs now\n",
    "forward_model.predict({\n",
    "    'tax_rate': tf.convert_to_tensor([5.0]),\n",
    "    'rooms': tf.convert_to_tensor([6.2])\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./forward_model/assets\n"
     ]
    }
   ],
   "source": [
    "FORWARD_EXPORT_PATH = './forward_model/'\n",
    "forward_model.save(FORWARD_EXPORT_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['rooms'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: serving_default_rooms:0\n",
      "  inputs['tax_rate'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: serving_default_tax_rate:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['price'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: StatefulPartitionedCall:0\n",
      "  outputs['tax_rate'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: StatefulPartitionedCall:1\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "!saved_model_cli show --tag_set serve --signature_def serving_default --dir {FORWARD_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forwarding by changing serving signature\n",
    "\n",
    "We could have employed the same method as before to also modify the serving signature and save out the model to achieve the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['rooms'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: serving_default_rooms:0\n",
      "  inputs['tax_rate'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: serving_default_tax_rate:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['price'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: StatefulPartitionedCall:0\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "!saved_model_cli show --tag_set serve --signature_def serving_default --dir {BOSTON_EXPORT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In our previous example, we leverage and inference function pulled off of a loaded model\n",
    "# In this case we will need to create ourselves since we haven't saved it out yet\n",
    "@tf.function(input_signature=[tf.TensorSpec([None, 1], dtype=tf.float32), tf.TensorSpec([None, 1], dtype=tf.float32)])\n",
    "def standard_forward_prediction(tax_rate, rooms):\n",
    "    pred = model([tax_rate, rooms], training=False)\n",
    "    return {\n",
    "        'price': pred,\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Return out the feature of interest as well as the prediction\n",
    "@tf.function(input_signature=[tf.TensorSpec([None, 1], dtype=tf.float32), tf.TensorSpec([None, 1], dtype=tf.float32)])\n",
    "def feature_forward_prediction(tax_rate, rooms):\n",
    "    pred = model([tax_rate, rooms], training=False)\n",
    "    return {\n",
    "        'price': pred,\n",
    "        'tax_rate': tax_rate\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: ./dual_signature_forward_model/assets\n"
     ]
    }
   ],
   "source": [
    "# Save out the model with both signatures\n",
    "DUAL_SIGNATURE_FORWARD_PATH = './dual_signature_forward_model/'\n",
    "model.save(DUAL_SIGNATURE_FORWARD_PATH, signatures={'serving_default': standard_forward_prediction,\n",
    "                                   'feature_forward': feature_forward_prediction})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The given SavedModel SignatureDef contains the following input(s):\n",
      "  inputs['rooms'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: feature_forward_rooms:0\n",
      "  inputs['tax_rate'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: feature_forward_tax_rate:0\n",
      "The given SavedModel SignatureDef contains the following output(s):\n",
      "  outputs['price'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: StatefulPartitionedCall:0\n",
      "  outputs['tax_rate'] tensor_info:\n",
      "      dtype: DT_FLOAT\n",
      "      shape: (-1, 1)\n",
      "      name: StatefulPartitionedCall:1\n",
      "Method name is: tensorflow/serving/predict\n"
     ]
    }
   ],
   "source": [
    "# Inspect just the feature_forward signature, but we also have standard serving_default\n",
    "!saved_model_cli show --tag_set serve --signature_def feature_forward --dir {DUAL_SIGNATURE_FORWARD_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2020 Google Inc.\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
    "http://www.apache.org/licenses/LICENSE-2.0\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  }
 ],
 "metadata": {
  "environment": {
   "name": "tf2-gpu.2-1.m46",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-1:m46"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}