site/en/tutorials/structured_data/preprocessing_layers.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zg02FZzDyEqd"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "2mapZ9afGJ69"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sMYQvJuBi7MS"
      },
      "source": [
        "# Classify structured data using Keras preprocessing layers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8FaL4wnr22oy"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/structured_data/preprocessing_layers\">\n",
        "    <img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />\n",
        "    View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/structured_data/preprocessing_layers.ipynb\">\n",
        "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />\n",
        "    Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/structured_data/preprocessing_layers.ipynb\">\n",
        "    <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />\n",
        "    View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/structured_data/preprocessing_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nna1tOKxyEqe"
      },
      "source": [
        "This tutorial demonstrates how to classify structured data, such as tabular data, using a simplified version of the <a href=\"https://www.kaggle.com/c/petfinder-adoption-prediction\" class=\"external\">PetFinder dataset from a Kaggle competition</a> stored in a CSV file.\n",
        "\n",
        "You will use [Keras](https://www.tensorflow.org/guide/keras) to define the model, and [Keras preprocessing layers](https://www.tensorflow.org/guide/keras/preprocessing_layers) as a bridge to map from columns in a CSV file to features used to train the model. The goal is to predict if a pet will be adopted.\n",
        "\n",
        "This tutorial contains complete code for:\n",
        "\n",
        "* Loading a CSV file into a <a href=\"https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html\" class=\"external\">DataFrame</a> using <a href=\"https://pandas.pydata.org/\" class=\"external\">pandas</a>.\n",
        "* Building an input pipeline to batch and shuffle the rows using `tf.data`. (Visit [tf.data: Build TensorFlow input pipelines](../../guide/data.ipynb) for more details.)\n",
        "* Mapping from columns in the CSV file to features used to train the model with the Keras preprocessing layers.\n",
        "* Building, training, and evaluating a model using the Keras built-in methods."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h5xkXCicjFQD"
      },
      "source": [
        "Note: This tutorial is similar to [Classify structured data with feature columns](../structured_data/feature_columns.ipynb). This version uses the [Keras preprocessing layers](https://www.tensorflow.org/guide/keras/preprocessing_layers) instead of the `tf.feature_column` API, as the former are more intuitive and can be easily included inside your model to simplify deployment."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZHxU1FMNpomc"
      },
      "source": [
        "## The PetFinder.my mini dataset\n",
        "\n",
        "There are several thousand rows in the PetFinder.my mini's CSV dataset file, where each row describes a pet (a dog or a cat) and each column describes an attribute, such as age, breed, color, and so on.\n",
        "\n",
        "In the dataset's summary below, notice there are mostly numerical and categorical columns. In this tutorial, you will only be dealing with those two feature types, dropping `Description` (a free text feature) and `AdoptionSpeed` (a classification feature) during data preprocessing.\n",
        "\n",
        "| Column          | Pet description               | Feature type   | Data type |\n",
        "| --------------- | ----------------------------- | -------------- | --------- |\n",
        "| `Type`          | Type of animal (`Dog`, `Cat`) | Categorical    | String    |\n",
        "| `Age`           | Age                           | Numerical      | Integer   |\n",
        "| `Breed1`        | Primary breed                 | Categorical    | String    |\n",
        "| `Color1`        | Color 1                       | Categorical    | String    |\n",
        "| `Color2`        | Color 2                       | Categorical    | String    |\n",
        "| `MaturitySize`  | Size at maturity              | Categorical    | String    |\n",
        "| `FurLength`     | Fur length                    | Categorical    | String    |\n",
        "| `Vaccinated`    | Pet has been vaccinated       | Categorical    | String    |\n",
        "| `Sterilized`    | Pet has been sterilized       | Categorical    | String    |\n",
        "| `Health`        | Health condition              | Categorical    | String    |\n",
        "| `Fee`           | Adoption fee                  | Numerical      | Integer   |\n",
        "| `Description`   | Profile write-up              | Text           | String    |\n",
        "| `PhotoAmt`      | Total uploaded photos         | Numerical      | Integer   |\n",
        "| `AdoptionSpeed` | Categorical speed of adoption | Classification | Integer   |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vjFbdBldyEqf"
      },
      "source": [
        "## Import TensorFlow and other libraries\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LklnLlt6yEqf"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow.keras import layers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TKU7RyoQGVKB"
      },
      "outputs": [],
      "source": [
        "tf.__version__"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UXvBvobayEqi"
      },
      "source": [
        "## Load the dataset and read it into a pandas DataFrame\n",
        "\n",
        "<a href=\"https://pandas.pydata.org/\" class=\"external\">pandas</a> is a Python library with many helpful utilities for loading and working with structured data. Use `tf.keras.utils.get_file` to download and extract the CSV file with the PetFinder.my mini dataset, and load it into a <a href=\"https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html\" class=\"external\">DataFrame</a> with <a href=\"https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html\" class=\"external\">`pandas.read_csv`</a>:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qJ4Ajn-YyEqj"
      },
      "outputs": [],
      "source": [
        "dataset_url = 'http://storage.googleapis.com/download.tensorflow.org/data/petfinder-mini.zip'\n",
        "csv_file = 'datasets/petfinder-mini/petfinder-mini.csv'\n",
        "\n",
        "tf.keras.utils.get_file('petfinder_mini.zip', dataset_url,\n",
        "                        extract=True, cache_dir='.')\n",
        "dataframe = pd.read_csv(csv_file)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "efa6910dfa5f"
      },
      "source": [
        "Inspect the dataset by checking the first five rows of the DataFrame:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3uiq4hoIGyXI"
      },
      "outputs": [],
      "source": [
        "dataframe.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C3zDbrozyEqq"
      },
      "source": [
        "## Create a target variable\n",
        "\n",
        "The original task in Kaggle's <a href=\"https://www.kaggle.com/c/petfinder-adoption-prediction\" class=\"external\">PetFinder.my Adoption Prediction competition</a> was to predict the speed at which a pet will be adopted (e.g. in the first week, the first month, the first three months, and so on).\n",
        "\n",
        "In this tutorial, you will simplify the task by transforming it into a binary classification problem, where you simply have to predict whether a pet was adopted or not.\n",
        "\n",
        "After modifying the `AdoptionSpeed` column, `0` will indicate the pet was not adopted, and `1` will indicate it was."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wmMDc46-yEqq"
      },
      "outputs": [],
      "source": [
        "# In the original dataset, `'AdoptionSpeed'` of `4` indicates\n",
        "# a pet was not adopted.\n",
        "dataframe['target'] = np.where(dataframe['AdoptionSpeed']==4, 0, 1)\n",
        "\n",
        "# Drop unused features.\n",
        "dataframe = dataframe.drop(columns=['AdoptionSpeed', 'Description'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sp0NCbswyEqs"
      },
      "source": [
        "## Split the DataFrame into training, validation, and test sets\n",
        "\n",
        "The dataset is in a single pandas DataFrame. Split it into training, validation, and test sets using a, for example, 80:10:10 ratio, respectively:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XvSinthO8oMj"
      },
      "outputs": [],
      "source": [
        "train, val, test = np.split(dataframe.sample(frac=1), [int(0.8*len(dataframe)), int(0.9*len(dataframe))])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "U02Q1moWoPwQ"
      },
      "outputs": [],
      "source": [
        "print(len(train), 'training examples')\n",
        "print(len(val), 'validation examples')\n",
        "print(len(test), 'test examples')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C_7uVu-xyEqv"
      },
      "source": [
        "## Create an input pipeline using tf.data\n",
        "\n",
        "Next, create a utility function that converts each training, validation, and test set DataFrame into a `tf.data.Dataset`, then shuffles and batches the data.\n",
        "\n",
        "Note: If you were working with a very large CSV file (so large that it does not fit into memory), you would use the `tf.data` API to read it from disk directly. That is not covered in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7r4j-1lRyEqw"
      },
      "outputs": [],
      "source": [
        "def df_to_dataset(dataframe, shuffle=True, batch_size=32):\n",
        "  df = dataframe.copy()\n",
        "  labels = df.pop('target')\n",
        "  df = {key: value.to_numpy()[:,tf.newaxis] for key, value in dataframe.items()}\n",
        "  ds = tf.data.Dataset.from_tensor_slices((dict(df), labels))\n",
        "  if shuffle:\n",
        "    ds = ds.shuffle(buffer_size=len(dataframe))\n",
        "  ds = ds.batch(batch_size)\n",
        "  ds = ds.prefetch(batch_size)\n",
        "  return ds"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PYxIXH579uS9"
      },
      "source": [
        "Now, use the newly created function (`df_to_dataset`) to check the format of the data the input pipeline helper function returns by calling it on the training data, and use a small batch size to keep the output readable:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tYiNH-QI96Jo"
      },
      "outputs": [],
      "source": [
        "batch_size = 5\n",
        "train_ds = df_to_dataset(train, batch_size=batch_size)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nFYir6S8HgIJ"
      },
      "outputs": [],
      "source": [
        "[(train_features, label_batch)] = train_ds.take(1)\n",
        "print('Every feature:', list(train_features.keys()))\n",
        "print('A batch of ages:', train_features['Age'])\n",
        "print('A batch of targets:', label_batch )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "geqHWW54Hmte"
      },
      "source": [
        "As the output demonstrates, the training set returns a dictionary of column names (from the DataFrame) that map to column values from rows."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-v50jBIuj4gb"
      },
      "source": [
        "## Apply the Keras preprocessing layers\n",
        "\n",
        "The Keras preprocessing layers allow you to build Keras-native input processing pipelines, which can be used as independent preprocessing code in non-Keras workflows, combined directly with Keras models, and exported as part of a Keras SavedModel.\n",
        "\n",
        "In this tutorial, you will use the following four preprocessing layers to demonstrate how to perform preprocessing, structured data encoding, and feature engineering:\n",
        "\n",
        "- `tf.keras.layers.Normalization`: Performs feature-wise normalization of input features.\n",
        "- `tf.keras.layers.CategoryEncoding`: Turns integer categorical features into one-hot, multi-hot, or <a href=\"https://en.wikipedia.org/wiki/Tf%E2%80%93idf\" class=\"external\">tf-idf</a>\n",
        "dense representations.\n",
        "- `tf.keras.layers.StringLookup`: Turns string categorical values into integer indices.\n",
        "- `tf.keras.layers.IntegerLookup`: Turns integer categorical values into integer indices.\n",
        "\n",
        "You can learn more about the available layers in the [Working with preprocessing layers](https://www.tensorflow.org/guide/keras/preprocessing_layers) guide.\n",
        "\n",
        "- For _numerical features_ of the PetFinder.my mini dataset, you will use a `tf.keras.layers.Normalization` layer to standardize the distribution of the data.\n",
        "- For _categorical features_, such as pet `Type`s (`Dog` and `Cat` strings), you will transform them to multi-hot encoded tensors with `tf.keras.layers.CategoryEncoding`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "twXBSxnT66o8"
      },
      "source": [
        "### Numerical columns\n",
        "\n",
        "For each numeric feature in the PetFinder.my mini dataset, you will use a `tf.keras.layers.Normalization` layer to standardize the distribution of the data.\n",
        "\n",
        "Define a new utility function that returns a layer which applies feature-wise normalization to numerical features using that Keras preprocessing layer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D6OuEKMMyEq1"
      },
      "outputs": [],
      "source": [
        "def get_normalization_layer(name, dataset):\n",
        "  # Create a Normalization layer for the feature.\n",
        "  normalizer = layers.Normalization(axis=None)\n",
        "\n",
        "  # Prepare a Dataset that only yields the feature.\n",
        "  feature_ds = dataset.map(lambda x, y: x[name])\n",
        "\n",
        "  # Learn the statistics of the data.\n",
        "  normalizer.adapt(feature_ds)\n",
        "\n",
        "  return normalizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lL4TRreQCPjV"
      },
      "source": [
        "Next, test the new function by calling it on the total uploaded pet photo features to normalize `'PhotoAmt'`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MpKgUDyk69bM"
      },
      "outputs": [],
      "source": [
        "photo_count_col = train_features['PhotoAmt']\n",
        "layer = get_normalization_layer('PhotoAmt', train_ds)\n",
        "layer(photo_count_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "foWY00YBUx9N"
      },
      "source": [
        "Note: If you have many numeric features (hundreds, or more), it is more efficient to concatenate them first and use a single `tf.keras.layers.Normalization` layer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVD--2WZ7vmh"
      },
      "source": [
        "### Categorical columns\n",
        "\n",
        "Pet `Type`s in the dataset are represented as strings—`Dog`s and `Cat`s—which need to be multi-hot encoded before being fed into the model. The `Age` feature\n",
        "\n",
        "Define another new utility function that returns a layer which maps values from a vocabulary to integer indices and multi-hot encodes the features using the `tf.keras.layers.StringLookup`, `tf.keras.layers.IntegerLookup`, and `tf.keras.CategoryEncoding` preprocessing layers:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GmgaeRjlDoUO"
      },
      "outputs": [],
      "source": [
        "def get_category_encoding_layer(name, dataset, dtype, max_tokens=None):\n",
        "  # Create a layer that turns strings into integer indices.\n",
        "  if dtype == 'string':\n",
        "    index = layers.StringLookup(max_tokens=max_tokens)\n",
        "  # Otherwise, create a layer that turns integer values into integer indices.\n",
        "  else:\n",
        "    index = layers.IntegerLookup(max_tokens=max_tokens)\n",
        "\n",
        "  # Prepare a `tf.data.Dataset` that only yields the feature.\n",
        "  feature_ds = dataset.map(lambda x, y: x[name])\n",
        "\n",
        "  # Learn the set of possible values and assign them a fixed integer index.\n",
        "  index.adapt(feature_ds)\n",
        "\n",
        "  # Encode the integer indices.\n",
        "  encoder = layers.CategoryEncoding(num_tokens=index.vocabulary_size())\n",
        "\n",
        "  # Apply multi-hot encoding to the indices. The lambda function captures the\n",
        "  # layer, so you can use them, or include them in the Keras Functional model later.\n",
        "  return lambda feature: encoder(index(feature))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7b3DwtTeCPjX"
      },
      "source": [
        "Test the `get_category_encoding_layer` function by calling it on pet `'Type'` features to turn them into multi-hot encoded tensors:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X2t2ff9K8PcT"
      },
      "outputs": [],
      "source": [
        "test_type_col = train_features['Type']\n",
        "test_type_layer = get_category_encoding_layer(name='Type',\n",
        "                                              dataset=train_ds,\n",
        "                                              dtype='string')\n",
        "test_type_layer(test_type_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j6eDongw8knz"
      },
      "source": [
        "Repeat the process on the pet `'Age'` features:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7FjBioQ38oNE"
      },
      "outputs": [],
      "source": [
        "test_age_col = train_features['Age']\n",
        "test_age_layer = get_category_encoding_layer(name='Age',\n",
        "                                             dataset=train_ds,\n",
        "                                             dtype='int64',\n",
        "                                             max_tokens=5)\n",
        "test_age_layer(test_age_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SiE0glOPkMyh"
      },
      "source": [
        "## Preprocess selected features to train the model on\n",
        "\n",
        "You have learned how to use several types of Keras preprocessing layers. Next, you will:\n",
        "\n",
        "- Apply the preprocessing utility functions defined earlier on 13 numerical and categorical features from the PetFinder.my mini dataset.\n",
        "- Add all the feature inputs to a list.\n",
        "\n",
        "As mentioned in the beginning, to train the model, you will use the PetFinder.my mini dataset's numerical (`'PhotoAmt'`, `'Fee'`) and categorical (`'Age'`, `'Type'`, `'Color1'`, `'Color2'`, `'Gender'`, `'MaturitySize'`, `'FurLength'`, `'Vaccinated'`, `'Sterilized'`, `'Health'`, `'Breed1'`) features.\n",
        "\n",
        "Note: If your aim is to build an accurate model, try a larger dataset of your own, and think carefully about which features are the most meaningful to include, and how they should be represented."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uj1GoHSZ9R3H"
      },
      "source": [
        "Earlier, you used a small batch size to demonstrate the input pipeline. Let's now create a new input pipeline with a larger batch size of 256:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Rcv2kQTTo23h"
      },
      "outputs": [],
      "source": [
        "batch_size = 256\n",
        "train_ds = df_to_dataset(train, batch_size=batch_size)\n",
        "val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)\n",
        "test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5bIGNYN2V7iR"
      },
      "source": [
        "Normalize the numerical features (the number of pet photos and the adoption fee), and add them to one list of inputs called `encoded_features`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q3RBa51VkaAn"
      },
      "outputs": [],
      "source": [
        "all_inputs = {}\n",
        "encoded_features = []\n",
        "\n",
        "# Numerical features.\n",
        "for header in ['PhotoAmt', 'Fee']:\n",
        "  numeric_col = tf.keras.Input(shape=(1,), name=header)\n",
        "  normalization_layer = get_normalization_layer(header, train_ds)\n",
        "  encoded_numeric_col = normalization_layer(numeric_col)\n",
        "  all_inputs[header] = numeric_col\n",
        "  encoded_features.append(encoded_numeric_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVcUAFd6bvlT"
      },
      "source": [
        "Turn the integer categorical values from the dataset (the pet age) into integer indices, perform multi-hot encoding, and add the resulting feature inputs to `encoded_features`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1FOMGfZflhoA"
      },
      "outputs": [],
      "source": [
        "age_col = tf.keras.Input(shape=(1,), name='Age', dtype='int64')\n",
        "\n",
        "encoding_layer = get_category_encoding_layer(name='Age',\n",
        "                                             dataset=train_ds,\n",
        "                                             dtype='int64',\n",
        "                                             max_tokens=5)\n",
        "encoded_age_col = encoding_layer(age_col)\n",
        "all_inputs['Age'] = age_col\n",
        "encoded_features.append(encoded_age_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QYzynk6wdqKe"
      },
      "source": [
        "Repeat the same step for the string categorical values:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "K8C8xyiXm-Ie"
      },
      "outputs": [],
      "source": [
        "categorical_cols = ['Type', 'Color1', 'Color2', 'Gender', 'MaturitySize',\n",
        "                    'FurLength', 'Vaccinated', 'Sterilized', 'Health', 'Breed1']\n",
        "\n",
        "for header in categorical_cols:\n",
        "  categorical_col = tf.keras.Input(shape=(1,), name=header, dtype='string')\n",
        "  encoding_layer = get_category_encoding_layer(name=header,\n",
        "                                               dataset=train_ds,\n",
        "                                               dtype='string',\n",
        "                                               max_tokens=5)\n",
        "  encoded_categorical_col = encoding_layer(categorical_col)\n",
        "  all_inputs[header] = categorical_col\n",
        "  encoded_features.append(encoded_categorical_col)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YHSnhz2fyEq3"
      },
      "source": [
        "## Create, compile, and train the model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IDGyN_wpo0XS"
      },
      "source": [
        "The next step is to create a model using the [Keras Functional API](https://www.tensorflow.org/guide/keras/functional). For the first layer in your model, merge the list of feature inputs—`encoded_features`—into one vector via concatenation with `tf.keras.layers.concatenate`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EtkwHC-akvcv"
      },
      "outputs": [],
      "source": [
        "encoded_features"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6Yrj-_pr6jyL"
      },
      "outputs": [],
      "source": [
        "all_features = tf.keras.layers.concatenate(encoded_features)\n",
        "x = tf.keras.layers.Dense(32, activation=\"relu\")(all_features)\n",
        "x = tf.keras.layers.Dropout(0.5)(x)\n",
        "output = tf.keras.layers.Dense(1)(x)\n",
        "\n",
        "model = tf.keras.Model(all_inputs, output)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NRLDRcYAefTA"
      },
      "source": [
        "Configure the model with Keras `Model.compile`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GZDb_lJdelSg"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
        "              metrics=[\"accuracy\"],\n",
        "              run_eagerly=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f6mNMfG6yEq5"
      },
      "source": [
        "Let's visualize the connectivity graph:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y7Bkx4c7yEq5"
      },
      "outputs": [],
      "source": [
        "# Use `rankdir='LR'` to make the graph horizontal.\n",
        "tf.keras.utils.plot_model(model, show_shapes=True, show_layer_names=True, rankdir=\"LR\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CED6OStLyEq7"
      },
      "source": [
        "Next, train and test the model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OQfE3PC6yEq8"
      },
      "outputs": [],
      "source": [
        "model.fit(train_ds, epochs=10, validation_data=val_ds)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T8N2uAdU2Cni"
      },
      "outputs": [],
      "source": [
        "result = model.evaluate(test_ds, return_dict=True)\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LmZMnTKaCZda"
      },
      "source": [
        "## Perform inference\n",
        "\n",
        "The model you have developed can now classify a row from a CSV file directly after you've included the preprocessing layers inside the model itself.\n",
        "\n",
        "You can now [save and reload the Keras model](../keras/save_and_load.ipynb) with `Model.save` and `Model.load_model` before performing inference on new data:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QH9Zy1sBvwOH"
      },
      "outputs": [],
      "source": [
        "model.save('my_pet_classifier.keras')\n",
        "reloaded_model = tf.keras.models.load_model('my_pet_classifier.keras')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D973plJrdwQ9"
      },
      "source": [
        "To get a prediction for a new sample, you can simply call the Keras `Model.predict` method. There are just two things you need to do:\n",
        "\n",
        "1.   Wrap scalars into a list so as to have a batch dimension (`Model`s only process batches of data, not single samples).\n",
        "2.   Call `tf.convert_to_tensor` on each feature."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rKq4pxtdDa7i"
      },
      "outputs": [],
      "source": [
        "sample = {\n",
        "    'Type': 'Cat',\n",
        "    'Age': 3,\n",
        "    'Breed1': 'Tabby',\n",
        "    'Gender': 'Male',\n",
        "    'Color1': 'Black',\n",
        "    'Color2': 'White',\n",
        "    'MaturitySize': 'Small',\n",
        "    'FurLength': 'Short',\n",
        "    'Vaccinated': 'No',\n",
        "    'Sterilized': 'No',\n",
        "    'Health': 'Healthy',\n",
        "    'Fee': 100,\n",
        "    'PhotoAmt': 2,\n",
        "}\n",
        "\n",
        "input_dict = {name: tf.convert_to_tensor([value]) for name, value in sample.items()}\n",
        "predictions = reloaded_model.predict(input_dict)\n",
        "prob = tf.nn.sigmoid(predictions[0])\n",
        "\n",
        "print(\n",
        "    \"This particular pet had a %.1f percent probability \"\n",
        "    \"of getting adopted.\" % (100 * prob)\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJQQZEiH2FaB"
      },
      "source": [
        "Note: You will typically have better results with deep learning with larger and more complex datasets. When working with a small dataset, such as the simplified PetFinder.my one, you can use a <a href=\"https://developers.google.com/machine-learning/glossary#decision-tree\" class=\"external\">decision tree</a> or a <a href=\"https://developers.google.com/machine-learning/glossary#random-forest\" class=\"external\">random forest</a> as a strong baseline. The goal of this tutorial is to demonstrate the mechanics of working with structured data, so you have a starting point when working with your own datasets in the future.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k0QAY2Tb2HYG"
      },
      "source": [
        "## Next steps\n",
        "\n",
        "To learn more about classifying structured data, try working with other datasets. To improve accuracy during training and testing your models, think carefully about which features to include in your model and how they should be represented.\n",
        "\n",
        "Below are some suggestions for datasets:\n",
        "\n",
        "- [TensorFlow Datasets: MovieLens](https://www.tensorflow.org/datasets/catalog/movie_lens): A set of movie ratings from a movie recommendation service.\n",
        "- [TensorFlow Datasets: Wine Quality](https://www.tensorflow.org/datasets/catalog/wine_quality): Two datasets related to red and white variants of the Portuguese \"Vinho Verde\" wine. You can also find the Red Wine Quality dataset on <a href=\"https://www.kaggle.com/uciml/red-wine-quality-cortez-et-al-2009\" class=\"external\">Kaggle</a>.\n",
        "- <a href=\"https://www.kaggle.com/Cornell-University/arxiv\" class=\"external\">Kaggle: arXiv Dataset</a>: A corpus of 1.7 million scholarly articles from arXiv, covering physics, computer science, math, statistics, electrical engineering, quantitative biology, and economics.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "preprocessing_layers.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}