DeepChem Jupyter Notebooks/使用 TensorFlow 和 PyTorch 创建模型.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UyDvil_3p0NA"
      },
      "source": [
        "#  使用 TensorFlow 和 PyTorch 创建模型\n",
        "\n",
        "在之前的教程中，我们使用的是 DeepChem 提供的标准模型。这对于许多应用来说都没问题，但是迟早你会希望使用你自己定义的体系结构创建一个全新的模型。DeepChem提供了与 TensorFlow (Keras) 和 PyTorch 的集成，所以你可以在这两个框架的模型中使用它。\n",
        "\n",
        "## Colab\n",
        "\n",
        "This tutorial and the rest in this sequence are designed to be done in Google colab. If you'd like to open this notebook in colab, you can use the following link.\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/Creating_Models_with_TensorFlow_and_PyTorch.ipynb)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6KbbaYENp0NE"
      },
      "outputs": [],
      "source": [
        "!pip install --pre deepchem"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IUG4mG9rp0NH"
      },
      "source": [
        "实际上，在DeepChem中使用TensorFlow或PyTorch模型时，你可以采用两种不同的方法。这取决于你是想使用TensorFlow/PyTorch APIs 还是DeepChem APIs 来训练和评估你的模型。对于前一种情况，DeepChem的 `Dataset` 类有一些方法可以方便地将其与其他框架一起使用。 `make_tf_dataset()` 返回一个遍历数据的 `tensorflow.data.Dataset` 对象。 `make_pytorch_dataset()` 返回一个遍历数据的 `torch.utils.data.IterableDataset` 对象。这让你可以使用 DeepChem 的数据集（datasets）、加载器（loaders）、特征器（featurizers）、变换器（transformers）、拆分器（splitters）等，并轻松将它们集成到你现有的 TensorFlow 或 PyTorch 代码中。\n",
        "\n",
        "但 DeepChem 还提供了许多其他有用的功能。另一种让你使用这些功能的方法是将你的模型包装在一个 DeepChem 的 `Model` 对象中。让我们看看如何做到这一点。\n",
        "\n",
        "## KerasModel\n",
        "\n",
        "`KerasModel` 是DeepChem `Model` 类的子类。它充当 `tensorflow.keras.Model` 的包装器。让我们看一个使用它的例子。对于本例，我们创建了一个由两个密集层组成的简单顺序（sequential）模型。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "N4slcJjQp0NI"
      },
      "outputs": [],
      "source": [
        "import deepchem as dc\n",
        "import tensorflow as tf\n",
        "\n",
        "keras_model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(1000, activation='relu'),\n",
        "    tf.keras.layers.Dropout(rate=0.5),\n",
        "    tf.keras.layers.Dense(1)\n",
        "])\n",
        "model = dc.models.KerasModel(keras_model, dc.models.losses.L2Loss())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_HJiUFj_p0NK"
      },
      "source": [
        "对于本例，我们使用了Keras的 `Sequential` 类。我们的模型由一个具有 ReLU 激活的密集层、50% 的 dropout 提供正则化和最后一个产生标量输出的层组成。我们还需要指定在训练模型时使用的损失函数，在本例中 L<sub>2</sub> 损失。我们现在可以训练和评估该模型，就像我们对任何其他 DeepChem 模型一样。例如，让我们加载 Delaney 溶解度数据集。我们的模型如何基于分子的扩展连通性指纹 (ECFPs) 来预测分子的溶解性?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dlCT4pk4p0NL",
        "outputId": "5e82a4ec-88eb-4491-86f8-fdfa2e368c84"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "training set score: {'pearson_r2_score': 0.9766804253639305}\n",
            "test set score: {'pearson_r2_score': 0.7048814451615332}\n"
          ]
        }
      ],
      "source": [
        "tasks, datasets, transformers = dc.molnet.load_delaney(featurizer='ECFP', splitter='random')\n",
        "train_dataset, valid_dataset, test_dataset = datasets\n",
        "model.fit(train_dataset, nb_epoch=50)\n",
        "metric = dc.metrics.Metric(dc.metrics.pearson_r2_score)\n",
        "print('training set score:', model.evaluate(train_dataset, [metric]))\n",
        "print('test set score:', model.evaluate(test_dataset, [metric]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z0-wyR0Rp0NM"
      },
      "source": [
        "## TorchModel\n",
        "\n",
        "`TorchModel` 的工作原理与 `KerasModel` 类似，只不过它包装了一个 `torch.nn.Module`。让我们使用PyTorch来创建另一个模型，就像前面的模型一样，并用相同的数据训练它。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "14aM8LLwp0NN",
        "outputId": "a8d391d4-d55d-4f04-cc68-75f3e03ebc9b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "training set score: {'pearson_r2_score': 0.9760781898204121}\n",
            "test set score: {'pearson_r2_score': 0.6981331812360332}\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "\n",
        "pytorch_model = torch.nn.Sequential(\n",
        "    torch.nn.Linear(1024, 1000),\n",
        "    torch.nn.ReLU(),\n",
        "    torch.nn.Dropout(0.5),\n",
        "    torch.nn.Linear(1000, 1)\n",
        ")\n",
        "model = dc.models.TorchModel(pytorch_model, dc.models.losses.L2Loss())\n",
        "\n",
        "model.fit(train_dataset, nb_epoch=50)\n",
        "print('training set score:', model.evaluate(train_dataset, [metric]))\n",
        "print('test set score:', model.evaluate(test_dataset, [metric]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rIuncKuIp0NP"
      },
      "source": [
        "## 损失的计算\n",
        "\n",
        "现在让我们看一个更高级的例子。在上述模型中，损失是直接从模型的输出计算出来的。这通常是可以的，但并非总是如此。考虑一个输出概率分布的分类模型。虽然从概率中计算损失是可能的，但从 logits 中计算损失在数值上更稳定。\n",
        "\n",
        "为此，我们创建一个返回多个输出的模型，包括概率和 logits。 `KerasModel` 和 `TorchModel` 让你指定一个“输出类型（output_types）”列表。如果一个特定的输出具有 `'prediction'` 类型，这意味着它是一个正常的输出，在调用 `predict()` 时应该返回。如果它有 `'loss'` 类型，这意味着它应该传递给损失函数，而不是正常的输出。\n",
        "\n",
        "顺序模型不允许多个输出，因此我们使用子类化样式模型（subclassing style model）。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-2g9yxs3p0NR"
      },
      "outputs": [],
      "source": [
        "class ClassificationModel(tf.keras.Model):\n",
        "    \n",
        "    def __init__(self):\n",
        "        super(ClassificationModel, self).__init__()\n",
        "        self.dense1 = tf.keras.layers.Dense(1000, activation='relu')\n",
        "        self.dense2 = tf.keras.layers.Dense(1)\n",
        "\n",
        "    def call(self, inputs, training=False):\n",
        "        y = self.dense1(inputs)\n",
        "        if training:\n",
        "            y = tf.nn.dropout(y, 0.5)\n",
        "        logits = self.dense2(y)\n",
        "        output = tf.nn.sigmoid(logits)\n",
        "        return output, logits\n",
        "\n",
        "keras_model = ClassificationModel()\n",
        "output_types = ['prediction', 'loss']\n",
        "model = dc.models.KerasModel(keras_model, dc.models.losses.SigmoidCrossEntropy(), output_types=output_types)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-SD72U4Zp0NS"
      },
      "source": [
        "我们可以在 BACE 数据集中训练我们的模型。这是一个二元分类任务，试图预测一个分子是否会抑制BACE-1酶。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VwxiVh4cp0NT",
        "outputId": "b9d60989-c3c6-4d77-d685-829c252b7167"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "training set score: {'roc_auc_score': 0.9995809177900399}\n",
            "test set score: {'roc_auc_score': 0.7629528985507246}\n"
          ]
        }
      ],
      "source": [
        "tasks, datasets, transformers = dc.molnet.load_bace_classification(feturizer='ECFP', splitter='scaffold')\n",
        "train_dataset, valid_dataset, test_dataset = datasets\n",
        "model.fit(train_dataset, nb_epoch=100)\n",
        "metric = dc.metrics.Metric(dc.metrics.roc_auc_score)\n",
        "print('training set score:', model.evaluate(train_dataset, [metric]))\n",
        "print('test set score:', model.evaluate(test_dataset, [metric]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sf4tIfaPs1Iw"
      },
      "source": [
        "类似地，我们将创建一个自定义的分类器模型类来与 `TorchModel` 一起使用。理由跟与上面的 `KerasModel` 相似，自定义模型允许轻松得到第二个密集层的未缩放输出(Tensorflow 中的 logits)。自定义类允许定义如何向前传递；在最终的sigmoid被应用产生预测之前得到 logits。\n",
        "\n",
        "最后，用一个需要概率和 logits 的 `ClassificationModel` 的实例与一个损失函数搭配生成一个 `TorchModel` 的实例进行训练。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jMOOa20Yszz2"
      },
      "outputs": [],
      "source": [
        "class ClassificationModel(torch.nn.Module):\n",
        "    \n",
        "    def __init__(self):\n",
        "        super(ClassificationModel, self).__init__()\n",
        "        self.dense1 = torch.nn.Linear(1024, 1000)\n",
        "        self.dense2 = torch.nn.Linear(1000, 1)\n",
        "\n",
        "    def forward(self, inputs):\n",
        "        y = torch.nn.functional.relu( self.dense1(inputs) )\n",
        "        y = torch.nn.functional.dropout(y, p=0.5, training=self.training)\n",
        "        logits = self.dense2(y)\n",
        "        output = torch.sigmoid(logits)\n",
        "        return output, logits\n",
        "\n",
        "torch_model = ClassificationModel()\n",
        "output_types = ['prediction', 'loss']\n",
        "model = dc.models.TorchModel(torch_model, dc.models.losses.SigmoidCrossEntropy(), output_types=output_types)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LQqp2m6iGdgv"
      },
      "source": [
        "我们将使用相同的 BACE 数据集。和以前一样，该模型将尝试进行二元分类任务，试图预测一个分子是否会抑制BACE-1酶。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N49-Xsic-9iY",
        "outputId": "09a3e30b-a811-423e-89d4-a888fbc6a738"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "training set score: {'roc_auc_score': 0.9996340015366347}\n",
            "test set score: {'roc_auc_score': 0.7615036231884058}\n"
          ]
        }
      ],
      "source": [
        "tasks, datasets, transformers = dc.molnet.load_bace_classification(feturizer='ECFP', splitter='scaffold')\n",
        "train_dataset, valid_dataset, test_dataset = datasets\n",
        "model.fit(train_dataset, nb_epoch=100)\n",
        "metric = dc.metrics.Metric(dc.metrics.roc_auc_score)\n",
        "print('training set score:', model.evaluate(train_dataset, [metric]))\n",
        "print('test set score:', model.evaluate(test_dataset, [metric]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3LY7QO9ip0NU"
      },
      "source": [
        "## 其他功能\n",
        "\n",
        "`KerasModel` 和 `TorchModel` 有很多其他的功能。下面列一些比较重要的。\n",
        "\n",
        "- 训练过程中自动保存检查点（checkpoints）。\n",
        "- 将进度记录到控制台（console）或者传送到[TensorBoard](https://www.tensorflow.org/tensorboard)或[Weights & Biases](https://docs.wandb.com/) 。\n",
        "- 以 `f(输出，标签，权重)` 的形式定义损失函数。\n",
        "- 使用 `ValidationCallback` 类来提前停止。\n",
        "- 从已训练的模型加载参数。\n",
        "- 估计模型输出的不确定性。\n",
        "- 通过显著性映射（saliency mapping）识别重要特征。\n",
        "\n",
        "通过将你自己的模型包装在 `KerasModel` 或 `TorchModel` 中，你就可以使用所有这些功能。有关它们的详细信息，请参阅API文档。\n",
        "\n",
        "# Congratulations! Time to join the Community!\n",
        "\n",
        "Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:\n",
        "\n",
        "## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)\n",
        "This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.\n",
        "\n",
        "## Join the DeepChem Gitter\n",
        "The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pOBd6-YdQSvF"
      },
      "source": [
        "## Citing This Tutorial\n",
        "If you found this tutorial useful please consider citing it using the provided BibTeX. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KZUk_9yIYw0c"
      },
      "outputs": [],
      "source": [
        "@manual{Intro1, \n",
        " title={5}, \n",
        " organization={DeepChem},\n",
        " author={Ramsundar, Bharath and Rebel, Alles}, \n",
        " howpublished = {\\url{https://github.com/deepchem/deepchem/blob/master/examples/tutorials/Creating_Models_with_TensorFlow_and_PyTorch.ipynb}}, \n",
        " year={2021}, \n",
        "} "
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "Copy of Creating_Models_with_TensorFlow_and_PyTorch.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.13 64-bit",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "vscode": {
      "interpreter": {
        "hash": "f7976576504ac6c456dadd405d7477574ca2a64265ee4724cfbc25daae5f6d94"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}