site/zh-cn/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RUymE2l9GZfO"
      },
      "source": [
        "##### Copyright 2018 The TensorFlow Hub Authors.\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "code",
        "id": "JMyTNwSJGGWg"
      },
      "outputs": [],
      "source": [
        "# Copyright 2018 The TensorFlow Hub Authors. All Rights Reserved.\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     http://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License.\n",
        "# =============================================================================="
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "co7MV6sX7Xto"
      },
      "source": [
        "# Universal Sentence Encoder\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MfBg1C5NB3X0"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>     <a target=\"_blank\" href=\"https://tensorflow.google.cn/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\">在 TensorFlow.org 上查看</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 运行</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 上查看源代码</a>\n",
        "</td>\n",
        "  <td>     <a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a>\n",
        "</td>\n",
        "  <td>     <a href=\"https://tfhub.dev/s?q=google%2Funiversal-sentence-encoder%2F4%20OR%20google%2Funiversal-sentence-encoder-large%2F5\"><img src=\"https://tensorflow.google.cn/images/hub_logo_32px.png\">查看 TF Hub 模型</a>   </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eAVQGidpL8v5"
      },
      "source": [
        "此笔记本演示了如何访问 Universal Sentence Encoder，并将它用于句子相似度和句子分类任务。\n",
        "\n",
        "Universal Sentence Encoder 使获取句子级别的嵌入向量变得与以往查找单个单词的嵌入向量一样容易。之后，您可以轻松地使用句子嵌入向量计算句子级别的语义相似度，以及使用较少监督的训练数据在下游分类任务中实现更好的性能。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pOTzp8O36CyQ"
      },
      "source": [
        "## 设置\n",
        "\n",
        "本部分将设置访问 TF Hub 上通用句子编码器的环境，并提供将编码器应用于单词、句子和段落的示例。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lVjNK8shFKOC"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "!pip3 install seaborn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "63Pd3nJnTl-i"
      },
      "source": [
        "有关安装 Tensorflow 的更多详细信息，请访问 [https://tensorflow.google.cn/install/](https://tensorflow.google.cn/install/)。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "zwty8Z6mAkdV"
      },
      "outputs": [],
      "source": [
        "#@title Load the Universal Sentence Encoder's TF Hub module\n",
        "from absl import logging\n",
        "\n",
        "import tensorflow as tf\n",
        "\n",
        "import tensorflow_hub as hub\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import os\n",
        "import pandas as pd\n",
        "import re\n",
        "import seaborn as sns\n",
        "\n",
        "module_url = \"https://tfhub.dev/google/universal-sentence-encoder/4\" #@param [\"https://tfhub.dev/google/universal-sentence-encoder/4\", \"https://tfhub.dev/google/universal-sentence-encoder-large/5\"]\n",
        "model = hub.load(module_url)\n",
        "print (\"module %s loaded\" % module_url)\n",
        "def embed(input):\n",
        "  return model(input)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q8F4LNGFqOiq"
      },
      "outputs": [],
      "source": [
        "#@title Compute a representation for each message, showing various lengths supported.\n",
        "word = \"Elephant\"\n",
        "sentence = \"I am a sentence for which I would like to get its embedding.\"\n",
        "paragraph = (\n",
        "    \"Universal Sentence Encoder embeddings also support short paragraphs. \"\n",
        "    \"There is no hard limit on how long the paragraph is. Roughly, the longer \"\n",
        "    \"the more 'diluted' the embedding will be.\")\n",
        "messages = [word, sentence, paragraph]\n",
        "\n",
        "# Reduce logging output.\n",
        "logging.set_verbosity(logging.ERROR)\n",
        "\n",
        "message_embeddings = embed(messages)\n",
        "\n",
        "for i, message_embedding in enumerate(np.array(message_embeddings).tolist()):\n",
        "  print(\"Message: {}\".format(messages[i]))\n",
        "  print(\"Embedding size: {}\".format(len(message_embedding)))\n",
        "  message_embedding_snippet = \", \".join(\n",
        "      (str(x) for x in message_embedding[:3]))\n",
        "  print(\"Embedding: [{}, ...]\\n\".format(message_embedding_snippet))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BnvjATdy64eR"
      },
      "source": [
        "# 语义文本相似度任务示例\n",
        "\n",
        "Universal Sentence Encoder 生成的嵌入向量会被近似归一化。两个句子的语义相似度可以作为编码的内积轻松进行计算。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h1FFCTKm7ba4"
      },
      "outputs": [],
      "source": [
        "def plot_similarity(labels, features, rotation):\n",
        "  corr = np.inner(features, features)\n",
        "  sns.set(font_scale=1.2)\n",
        "  g = sns.heatmap(\n",
        "      corr,\n",
        "      xticklabels=labels,\n",
        "      yticklabels=labels,\n",
        "      vmin=0,\n",
        "      vmax=1,\n",
        "      cmap=\"YlOrRd\")\n",
        "  g.set_xticklabels(labels, rotation=rotation)\n",
        "  g.set_title(\"Semantic Textual Similarity\")\n",
        "\n",
        "def run_and_plot(messages_):\n",
        "  message_embeddings_ = embed(messages_)\n",
        "  plot_similarity(messages_, message_embeddings_, 90)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "339tuJ5Pwqqv"
      },
      "source": [
        "## 可视化相似度\n",
        "\n",
        "下面，我们在热图中显示相似度。最终的图形是一个 9x9 矩阵，其中每个条目 `[i, j]` 都根据句子 `i` 和 `j` 的编码的内积进行着色。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cPMCaxrZwp7t"
      },
      "outputs": [],
      "source": [
        "messages = [\n",
        "    # Smartphones\n",
        "    \"I like my phone\",\n",
        "    \"My phone is not good.\",\n",
        "    \"Your cellphone looks great.\",\n",
        "\n",
        "    # Weather\n",
        "    \"Will it snow tomorrow?\",\n",
        "    \"Recently a lot of hurricanes have hit the US\",\n",
        "    \"Global warming is real\",\n",
        "\n",
        "    # Food and health\n",
        "    \"An apple a day, keeps the doctors away\",\n",
        "    \"Eating strawberries is healthy\",\n",
        "    \"Is paleo better than keto?\",\n",
        "\n",
        "    # Asking about age\n",
        "    \"How old are you?\",\n",
        "    \"what is your age?\",\n",
        "]\n",
        "\n",
        "run_and_plot(messages)\n",
        "               "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6FjdeCqPJeg-"
      },
      "source": [
        "## 评估：STS（语义文本相似度）基准\n",
        "\n",
        "[**STS 基准**](https://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark)会根据从句子嵌入向量计算得出的相似度得分与人为判断的一致程度，提供内部评估。该基准要求系统为不同的句子对选择返回相似度得分。然后使用[皮尔逊相关](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)来评估机器相似度得分相对于人为判断的质量。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q5nuBbI1iFQR"
      },
      "source": [
        "### 下载数据"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VOs8ZfOnJeBF"
      },
      "outputs": [],
      "source": [
        "import pandas\n",
        "import scipy\n",
        "import math\n",
        "import csv\n",
        "\n",
        "sts_dataset = tf.keras.utils.get_file(\n",
        "    fname=\"Stsbenchmark.tar.gz\",\n",
        "    origin=\"http://ixa2.si.ehu.es/stswiki/images/4/48/Stsbenchmark.tar.gz\",\n",
        "    extract=True)\n",
        "sts_dev = pandas.read_table(\n",
        "    os.path.join(os.path.dirname(sts_dataset), \"stsbenchmark\", \"sts-dev.csv\"),\n",
        "    skip_blank_lines=True,\n",
        "    usecols=[4, 5, 6],\n",
        "    names=[\"sim\", \"sent_1\", \"sent_2\"])\n",
        "sts_test = pandas.read_table(\n",
        "    os.path.join(\n",
        "        os.path.dirname(sts_dataset), \"stsbenchmark\", \"sts-test.csv\"),\n",
        "    quoting=csv.QUOTE_NONE,\n",
        "    skip_blank_lines=True,\n",
        "    usecols=[4, 5, 6],\n",
        "    names=[\"sim\", \"sent_1\", \"sent_2\"])\n",
        "# cleanup some NaN values in sts_dev\n",
        "sts_dev = sts_dev[[isinstance(s, str) for s in sts_dev['sent_2']]]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8OKy8WhnKRe_"
      },
      "source": [
        "### 评估句子嵌入向量"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "W-q2r7jyZGb7"
      },
      "outputs": [],
      "source": [
        "sts_data = sts_dev #@param [\"sts_dev\", \"sts_test\"] {type:\"raw\"}\n",
        "\n",
        "def run_sts_benchmark(batch):\n",
        "  sts_encode1 = tf.nn.l2_normalize(embed(tf.constant(batch['sent_1'].tolist())), axis=1)\n",
        "  sts_encode2 = tf.nn.l2_normalize(embed(tf.constant(batch['sent_2'].tolist())), axis=1)\n",
        "  cosine_similarities = tf.reduce_sum(tf.multiply(sts_encode1, sts_encode2), axis=1)\n",
        "  clip_cosine_similarities = tf.clip_by_value(cosine_similarities, -1.0, 1.0)\n",
        "  scores = 1.0 - tf.acos(clip_cosine_similarities) / math.pi\n",
        "  \"\"\"Returns the similarity scores\"\"\"\n",
        "  return scores\n",
        "\n",
        "dev_scores = sts_data['sim'].tolist()\n",
        "scores = []\n",
        "for batch in np.array_split(sts_data, 10):\n",
        "  scores.extend(run_sts_benchmark(batch))\n",
        "\n",
        "pearson_correlation = scipy.stats.pearsonr(scores, dev_scores)\n",
        "print('Pearson correlation coefficient = {0}\\np-value = {1}'.format(\n",
        "    pearson_correlation[0], pearson_correlation[1]))"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "RUymE2l9GZfO"
      ],
      "name": "semantic_similarity_with_tf_hub_universal_encoder.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}