notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Vertex AI Explanations with TabNet models\n",
        "\n",
        "<table align=\"left\">\n",
        "\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/tabnet/ai-explanations-tabnet-algorithm.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>                                                                                               \n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ce49f48f9bb2"
      },
      "source": [
        "## Overview\n",
        "\n",
        "Vertex AI provides a algorithm called on [TabNet] (https://arxiv.org/abs/1908.07442). TabNet is an interpretable deep learning architecture for tabular (structured) data, the most common data type among enterprises. TabNet combines the best of two worlds: it is explainable, like simpler tree-based models, and can achieve the high accuracy of complex black-box models and ensembles, meaning it is precise without obscuring how the model works. This makes TabNet well-suited for a wide range of tabular data tasks where model explainability is just as important as accuracy.\n",
        "\n",
        "The goal of the tutorial is to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm.\n",
        "\n",
        "Learn more about [Tabular Workflow for TabNet](https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/tabnet)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to provide a sample plotting tool to visualize the output of TabNet, which is helpful in explaining the algorithm. \n",
        "\n",
        "\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
        "- Vertex Explainable AI\n",
        "- TabNet builtin algorithm\n",
        "\n",
        "The steps performed are:\n",
        "* Setup the project.\n",
        "* Download the prediction data of pretrain model onf Syn2 data.\n",
        "* Visualize and understand the feature importance based on the masks output.\n",
        "* Clean up the resource created by this tutorial."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d631370325e"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "This tutorial uses Synthetic_2 (Syn2) data, described in Section 4.1 of the [Learning to Explain](https://arxiv.org/pdf/1802.07814.pdf) paper. The input feature X is generated from a 10-dimensional standard Gaussian. The response variable Y is generated from feature X[3:6] only. The data has been split into training and prediction sets and has been uploaded to Google Cloud Storage:\n",
        "* Training data: gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_input/syn2_train.csv.\n",
        "* Prediction output data: gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2\n",
        "\n",
        "At this time, the TabNet pre-trained model file is not publicly available."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2eba58ad71"
      },
      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xOMNWzTbftDr"
      },
      "source": [
        "## Installation\n",
        "\n",
        "Install the following packages required to execute this notebook. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9eb4e79842bf"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform tensorflow"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "restart"
      },
      "source": [
        "### Colab only: Uncomment the following cell to restart the kernel.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bzPxhxS5lugp"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d2qpIurSjmpT"
      },
      "source": [
        "## Before you begin\n",
        "\n",
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
        "2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
        "\n",
        "3. [Enable the following APIs: Vertex AI API, Cloud Resource Manager API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,cloudresourcemanager.googleapis.com).\n",
        "\n",
        "4. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "project_id"
      },
      "source": [
        "#### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, try the following:\n",
        "* Run `gcloud config list`.\n",
        "* Run `gcloud projects list`.\n",
        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wsePm9c4jmpT"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# Set the project id\n",
        "! gcloud config set project {PROJECT_ID}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a54f9d7c1876"
      },
      "source": [
        "#### Region\n",
        "\n",
        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3aaadaaf9b30"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e663bd062c6f"
      },
      "source": [
        "#### UUID\n",
        "\n",
        "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a uuid for each instance session, and append it onto the name of resources you create in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "953fa6e5ddda"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "import string\n",
        "\n",
        "\n",
        "# Generate a uuid of a specifed length(default=8)\n",
        "def generate_uuid(length: int = 8) -> str:\n",
        "    return \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n",
        "\n",
        "\n",
        "UUID = generate_uuid()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5c0404984792"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
        "\n",
        "**1. Vertex AI Workbench**\n",
        "* Do nothing as you are already authenticated.\n",
        "\n",
        "**2. Local JupyterLab instance, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nt8cEM2GjmpU"
      },
      "outputs": [],
      "source": [
        "# ! gcloud auth login"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XUSL_JcpjmpU"
      },
      "source": [
        "**3. Colab, uncomment and run:**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_2zemfGvjmpU"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCPJ38n7jmpU"
      },
      "source": [
        "**4. Service account or other**\n",
        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:custom"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets.\n",
        "\n",
        "When you submit a training job using the Cloud SDK, you upload a Python package\n",
        "containing your training code to a Cloud Storage bucket. Vertex AI runs\n",
        "the code from this package. In this tutorial, Vertex AI also saves the\n",
        "trained model that results from your job in the same bucket. Using this model artifact, you can then\n",
        "create Vertex AI Model resource and use for prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = \"gs://your-bucket-name-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oz8J0vmSlugt"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XoEqT2Y4DJmf"
      },
      "source": [
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pRUOFELefqf1"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "import matplotlib.cm as cm\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from google.cloud import storage\n",
        "\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "06f6abd8c40e"
      },
      "source": [
        "## Reading a sample TabNet prediction on syn2 data\n",
        "\n",
        "After training and serving your model, you upload the output to Google Cloud Storage. \n",
        "\n",
        "Sample prediction data is stored on Google Cloud at gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2. You can use your own set of prediction data, but you must ensure that the format of the prediction data is the same as the format of the training data.\n",
        "\n",
        "Each prediction in TabNet contains a mask that is used to explain the predictions. The mask is stored in an **aggregated_mask_values** field.\n",
        "\n",
        "Information about the training set and the model are better suited for the Dataset section at the previous section of the notebook.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0f55ca32f9b3"
      },
      "outputs": [],
      "source": [
        "!gsutil cp gs://cloud-samples-data/ai-platform-unified/datasets/tabnet/tab_net_output/syn2 $BUCKET_URI\n",
        "\n",
        "# Replace your the BUCKET_URI and PREDICTION_FILE\n",
        "# BUCKET_NAME = \"[<your-bucket-name>]\"\n",
        "# PREDICTION_FILE = \"[<your-prediction-file>]\"\n",
        "\n",
        "BUCKET_NAME = BUCKET_URI[5:]\n",
        "PREDICTION_FILE = \"syn2\"\n",
        "\n",
        "MASK_KEY = \"aggregated_mask_values\"\n",
        "\n",
        "HEADER = [(\"feat_\" + str(i)) for i in range(1, 12)]\n",
        "HEADER"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c4782ae3d628"
      },
      "source": [
        "### Download and preprocess the predictions."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4aeaa7143830"
      },
      "outputs": [],
      "source": [
        "storage_client = storage.Client()\n",
        "bucket = storage_client.get_bucket(BUCKET_NAME)\n",
        "blob = bucket.blob(PREDICTION_FILE)\n",
        "f = blob.download_as_string(client=None).decode(\"utf-8\").strip()\n",
        "predictions = f.split(\"\\n\")\n",
        "predictions[:1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3f9f6b27280a"
      },
      "source": [
        "## Parse the mask values in prediction. Then, concatenate the mask values.\n",
        "The output is a matrix having Nxk (N is the number of outputs, k is the size of each mask). Concatenating mask values are used to visualize the feature importance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7f5b5c11f3dc"
      },
      "outputs": [],
      "source": [
        "masks = []\n",
        "for prediction in predictions:\n",
        "    prediction = json.loads(prediction)\n",
        "    masks.append(prediction[MASK_KEY])\n",
        "masks = np.matrix(masks)\n",
        "masks.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8455a22d412c"
      },
      "source": [
        "## Visualize the mask value matrix.\n",
        "The lighter color indicates more important feature. For example, only features 3-6 are meaningful in prediction output in Syn2 data. In the plot, the column 3-6 have light color."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ab63558c2593"
      },
      "outputs": [],
      "source": [
        "fig = plt.figure(figsize=(20, 10))\n",
        "ax = fig.add_subplot(121)\n",
        "ax.imshow(masks[:50, :], interpolation=\"bilinear\", cmap=cm.Greys_r)\n",
        "ax.set_xlabel(\"Features\")\n",
        "ax.set_ylabel(\"Sample index\")\n",
        "ax.xaxis.set_ticks(np.arange(len(HEADER)))\n",
        "ax.set_xticklabels(HEADER, rotation=\"vertical\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TpV-iwP9qw9c"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sx_vKniMq9ZX"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "# Delete Cloud Storage that were created\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    ! gsutil -m rm -r $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cb0f06c7ba0d"
      },
      "source": [
        "## What's next?\n",
        "\n",
        "To learn more about TabNet, check out the resources here.\n",
        "\n",
        "* [TabNet: Attentive Interpretable Tabular Learning](https://arxiv.org/abs/1908.07442)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "ai-explanations-tabnet-algorithm.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}