notebooks/2023_10_05_GPT_Literature_Review_Assistant.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "mount_file_id": "14RoayoY48SCreWP18JnjxNf7hOZCYQEN",
      "authorship_tag": "ABX9TyMDSUVgWihBeUG7YWp9swei",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/michaelachmann/social-media-lab/blob/main/notebooks/2023_10_05_GPT_Literature_Review_Assistant.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Literature Review Assistant [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8199901.svg)](https://doi.org/10.5281/zenodo.8199901)\n",
        "\n",
        "![Notes on (Computational) Social Media Research Banner](https://raw.githubusercontent.com/michaelachmann/social-media-lab/main/images/banner.png)\n",
        "\n",
        "## Overview\n",
        "\n",
        "This Jupyter notebook is a part of the social-media-lab.net project, which is a work-in-progress textbook on computational social media analysis. The notebook is intended for use in my classes.\n",
        "\n",
        "The **Literature Review Assistant** uses search result lists of Publish or Perish to get started with prompting. It uses GPT to extract information from abstracts to assist the literature review process.\n",
        "\n",
        "### Project Information\n",
        "\n",
        "- Project Website: [social-media-lab.net](https://social-media-lab.net/)\n",
        "- GitHub Repository: [https://github.com/michaelachmann/social-media-lab](https://github.com/michaelachmann/social-media-lab)\n",
        "\n",
        "## License Information\n",
        "\n",
        "This notebook, along with all other notebooks in the project, is licensed under the following terms:\n",
        "\n",
        "- License: [GNU General Public License version 3.0 (GPL-3.0)](https://www.gnu.org/licenses/gpl-3.0.de.html)\n",
        "- License File: [LICENSE.md](https://github.com/michaelachmann/social-media-lab/blob/main/LICENSE.md)\n",
        "\n",
        "\n",
        "## Citation\n",
        "\n",
        "If you use or reference this notebook in your work, please cite it appropriately. Here is an example of the citation:\n",
        "\n",
        "```\n",
        "Michael Achmann. (2023). michaelachmann/social-media-lab: 27.10.2023 (v0.0.1). Zenodo. https://doi.org/10.5281/zenodo.8199901\n",
        "```"
      ],
      "metadata": {
        "id": "TQNXTZvZL3t3"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "XlJ90uBPmknH",
        "outputId": "8aea6a20-db99-4bcc-c80d-3d814ce5bbd9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Install Packages\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.0/77.0 kB\u001b[0m \u001b[31m2.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for ratelimit (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "llmx 0.0.15a0 requires tiktoken, which is not installed.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ],
      "source": [
        "#@title Setup\n",
        "#@markdown At first we need to install necessary packages. Hit run and wait.\n",
        "\n",
        "print(\"Install Packages\")\n",
        "!pip install -q openai crossref-commons"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Import from Publish or Perish Data.\n",
        "#@markdown If this is the start of your review process, upload the `csv` file exported from [Publish or Perish](https://harzing.com/resources/publish-or-perish) in the left-hand *Files* pane. Enter the filename in `publish_or_perish_file_name`. Define the output name in `file_name`. If you want to save the imported file in the google drive add `/content/drive/MyDrive/` to the path. <br/> **Skip this cell if you want to work with a file that has been imported in the past.**\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import io\n",
        "\n",
        "publish_or_perish_file_name = \"scholar.csv\" # @param {type: \"string\"}\n",
        "file_name = \"2023-10-31-Literature-Review.csv\" # @param {type: \"string\"}\n",
        "\n",
        "# Initialize empty DataFrame\n",
        "all_data = pd.DataFrame()\n",
        "\n",
        "\n",
        "try:\n",
        "    all_data = pd.read_csv(publish_or_perish_file_name)\n",
        "\n",
        "    # Remove Duplicates\n",
        "    initial_len = len(all_data)\n",
        "    all_data = all_data.drop_duplicates(subset='DOI', keep='first')\n",
        "    removed_len = initial_len - len(all_data)\n",
        "    print(f'Removed {removed_len} duplicates based on DOI.')\n",
        "\n",
        "    # Remove missing DOIs\n",
        "    initial_len = len(all_data)\n",
        "    all_data = all_data[~pd.isna(all_data['DOI'])]\n",
        "    removed_len = initial_len - len(all_data)\n",
        "    print(f'Removed {removed_len} rows without DOI.')\n",
        "\n",
        "    all_data = all_data.sort_values(by='Cites', ascending=False).reset_index(drop=True)\n",
        "\n",
        "    print('Sorted Table by Cites.')\n",
        "\n",
        "    # Create empty columns for Literature Review\n",
        "    all_data[\"Relevant\"] = \"\"\n",
        "    all_data[\"Notes\"] = \"\"\n",
        "    all_data[\"Checked\"] = False\n",
        "\n",
        "    print('Initialized Columns')\n",
        "\n",
        "    all_data.to_csv(file_name)\n",
        "    print(f\"Success: Saved data to {file_name}\")\n",
        "\n",
        "    print(f'Success: Data loaded from File \"{file_name}\".')\n",
        "except Exception as e:\n",
        "    print(f\"Error: Failed to load data from File. {str(e)}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "qVDJxnENyXyq",
        "outputId": "5ddc5646-1b57-4f03-fb3b-46cbfd22001b"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Removed 172 duplicates based on DOI.\n",
            "Removed 1 rows without DOI.\n",
            "Sorted Table by Cites.\n",
            "Initializes Columns\n",
            "Success: Saved data to 2023-10-31-Literature-Review.csv\n",
            "Success: Data loaded from File \"2023-10-31-Literature-Review.csv\".\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Read previously imported File\n",
        "#@markdown If you want to keep going with a former review process, we can read an uploaded file / a file from google drive. **Only run one cell, this one or the above.**\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import io\n",
        "\n",
        "file_name = \"2023-10-31-Literature-Review.csv\" # @param {type: \"string\"}\n",
        "\n",
        "try:\n",
        "    all_data = pd.read_csv(file_name)\n",
        "\n",
        "    print(f'Success: Data loaded from File \"{file_name}\".')\n",
        "except Exception as e:\n",
        "    print(f\"Error: Failed to load data from File. {str(e)}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "iixJcNUUm3QF",
        "outputId": "405622bc-05b3-40ad-984b-47bff749c27c"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Success: Data loaded from File \"2023-10-31-Literature-Review.csv\".\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this example we've saved the file locally. When working with Colab, the file will be deleted when we disconnect. For colab you should link your google drive (open the files pane on the left, click the Google Drive button). Once connected, save the file in the folder `/content/drive/MyDrive/YOUR-FILENAME.csv`. It will be accessible through Drive, and Colab is from now on going to connect automatically to drive.\n",
        "\n",
        "Check the imported data. We're using pandas, the imported data is saved in the `all_data`variable. `head(2)`displays the two top rows of the table. Additionally, we have added three columns: `Relevant`, `Notes`, and `Checked`. We are going to make use of them to keep track of our progress."
      ],
      "metadata": {
        "id": "8xMdEPeOobA_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the structure (and content) of the file\n",
        "all_data.head(2)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 352
        },
        "id": "5qaAJSF1omZe",
        "outputId": "de7a592e-b57a-431e-e2b6-8575e046de13"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Unnamed: 0  Cites                  Authors  \\\n",
              "0           0    479  M Tiggemann, M Zaccardo   \n",
              "1           1    116         D Ging, S Garvey   \n",
              "\n",
              "                                               Title    Year  \\\n",
              "0  'Strong is the new skinny': A content analysis...  2018.0   \n",
              "1  'Written in these scars are the stories I can'...  2018.0   \n",
              "\n",
              "                         Source             Publisher  \\\n",
              "0  Journal of health psychology  journals.sagepub.com   \n",
              "1           New Media & Society  journals.sagepub.com   \n",
              "\n",
              "                                          ArticleURL  \\\n",
              "0  https://journals.sagepub.com/doi/abs/10.1177/1...   \n",
              "1  https://journals.sagepub.com/doi/abs/10.1177/1...   \n",
              "\n",
              "                                            CitesURL  GSRank  ...  \\\n",
              "0  https://scholar.google.com/scholar?cites=12266...       3  ...   \n",
              "1  https://scholar.google.com/scholar?cites=17319...      19  ...   \n",
              "\n",
              "  CitesPerYear CitesPerAuthor AuthorCount  Age  \\\n",
              "0       119.75            240           2  4.0   \n",
              "1        29.00             58           2  4.0   \n",
              "\n",
              "                                            Abstract  \\\n",
              "0  … This study provides a content analysis of fi...   \n",
              "1  … such as Instagram. Using a dataset of 7560 i...   \n",
              "\n",
              "                                         FullTextURL  \\\n",
              "0                                                NaN   \n",
              "1  https://www.researchgate.net/profile/Debbie-Gi...   \n",
              "\n",
              "                                          RelatedURL  Relevant  Notes  Checked  \n",
              "0  https://scholar.google.com/scholar?q=related:c...       NaN    NaN    False  \n",
              "1  https://scholar.google.com/scholar?q=related:-...       NaN    NaN    False  \n",
              "\n",
              "[2 rows x 30 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4f428f1f-ed7d-4752-95ec-8ff0038e270e\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>Cites</th>\n",
              "      <th>Authors</th>\n",
              "      <th>Title</th>\n",
              "      <th>Year</th>\n",
              "      <th>Source</th>\n",
              "      <th>Publisher</th>\n",
              "      <th>ArticleURL</th>\n",
              "      <th>CitesURL</th>\n",
              "      <th>GSRank</th>\n",
              "      <th>...</th>\n",
              "      <th>CitesPerYear</th>\n",
              "      <th>CitesPerAuthor</th>\n",
              "      <th>AuthorCount</th>\n",
              "      <th>Age</th>\n",
              "      <th>Abstract</th>\n",
              "      <th>FullTextURL</th>\n",
              "      <th>RelatedURL</th>\n",
              "      <th>Relevant</th>\n",
              "      <th>Notes</th>\n",
              "      <th>Checked</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>479</td>\n",
              "      <td>M Tiggemann, M Zaccardo</td>\n",
              "      <td>'Strong is the new skinny': A content analysis...</td>\n",
              "      <td>2018.0</td>\n",
              "      <td>Journal of health psychology</td>\n",
              "      <td>journals.sagepub.com</td>\n",
              "      <td>https://journals.sagepub.com/doi/abs/10.1177/1...</td>\n",
              "      <td>https://scholar.google.com/scholar?cites=12266...</td>\n",
              "      <td>3</td>\n",
              "      <td>...</td>\n",
              "      <td>119.75</td>\n",
              "      <td>240</td>\n",
              "      <td>2</td>\n",
              "      <td>4.0</td>\n",
              "      <td>… This study provides a content analysis of fi...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>https://scholar.google.com/scholar?q=related:c...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>116</td>\n",
              "      <td>D Ging, S Garvey</td>\n",
              "      <td>'Written in these scars are the stories I can'...</td>\n",
              "      <td>2018.0</td>\n",
              "      <td>New Media &amp; Society</td>\n",
              "      <td>journals.sagepub.com</td>\n",
              "      <td>https://journals.sagepub.com/doi/abs/10.1177/1...</td>\n",
              "      <td>https://scholar.google.com/scholar?cites=17319...</td>\n",
              "      <td>19</td>\n",
              "      <td>...</td>\n",
              "      <td>29.00</td>\n",
              "      <td>58</td>\n",
              "      <td>2</td>\n",
              "      <td>4.0</td>\n",
              "      <td>… such as Instagram. Using a dataset of 7560 i...</td>\n",
              "      <td>https://www.researchgate.net/profile/Debbie-Gi...</td>\n",
              "      <td>https://scholar.google.com/scholar?q=related:-...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>2 rows × 30 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4f428f1f-ed7d-4752-95ec-8ff0038e270e')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4f428f1f-ed7d-4752-95ec-8ff0038e270e button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4f428f1f-ed7d-4752-95ec-8ff0038e270e');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-6c9c95c3-2541-45c8-8d7c-e8c3d88d93f4\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6c9c95c3-2541-45c8-8d7c-e8c3d88d93f4')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-6c9c95c3-2541-45c8-8d7c-e8c3d88d93f4 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In the next step we are going to start our literature review:\n",
        "1. We filter for the first unchecked row, ordered by the cite count.\n",
        "2. We retrieve the abstract from [CrossRef API](https://www.crossref.org/documentation/retrieve-metadata/rest-api/) using the [DOI](https://en.wikipedia.org/wiki/Digital_object_identifier).\n",
        "3. We display all information\n",
        "4. We answer whether the paper appear to be relevant by entering y or n for **y**es or **n**o.\n",
        "\n",
        "For our session, the cell only runs through one row and finishes afterwards. For a real world application you'd probably like to add some kind of loop."
      ],
      "metadata": {
        "id": "LsiM0hlypQj-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from crossref_commons.retrieval import get_publication_as_json\n",
        "import json\n",
        "import openai\n",
        "import textwrap\n",
        "import IPython\n",
        "import re\n",
        "\n",
        "# Get one row: Not checked, highest Citation count.\n",
        "highest_cites_unchecked = all_data[all_data['Checked'] == False].sort_values(by=\"Cites\", ascending=False).iloc[0]\n",
        "index = highest_cites_unchecked.name\n",
        "\n",
        "# Retrieve Abstract from Crossref\n",
        "response = get_publication_as_json(highest_cites_unchecked['DOI'])\n",
        "abstract =  response.get(\"abstract\", \"\")\n",
        "\n",
        "# Remove XML\n",
        "abstract = re.sub(r'<[^>]+>', '', abstract)\n",
        "all_data.loc[index, 'Abstract'] = abstract\n",
        "\n",
        "\n",
        "# Display all information\n",
        "IPython.display.clear_output(wait=True)\n",
        "title_disp = IPython.display.HTML(\"<h2>{}</h2>\".format(highest_cites_unchecked['Title']))\n",
        "authors_disp = IPython.display.HTML(\"<p>{}</p>\".format(highest_cites_unchecked['Authors']))\n",
        "doi_disp = IPython.display.HTML(\"<p><a target='_blank' href='https://doi.org/{}'>{}</a></p>\".format(highest_cites_unchecked['DOI'],highest_cites_unchecked['DOI']))\n",
        "display(title_disp, authors_disp, doi_disp)\n",
        "print(textwrap.fill(abstract, 80))\n",
        "relevant_input = input('Relevant? (y/n): ').lower().strip() == 'y'\n",
        "\n",
        "# Save user input\n",
        "all_data.loc[index, 'Checked'] = True\n",
        "all_data.loc[index, 'Relevant'] = relevant_input"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 287
        },
        "id": "tlmQi90Hd_Hw",
        "outputId": "22dab199-09d1-480c-e450-cf0289b37929"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<h2>'Strong is the new skinny': A content analysis of# fitspiration images on Instagram</h2>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<p>M Tiggemann, M Zaccardo</p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<p><a target='_blank' href='https://doi.org/10.1177/1359105316639436'>10.1177/1359105316639436</a></p>"
            ]
          },
          "metadata": {}
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " ‘Fitspiration’ is an online trend designed to inspire viewers towards a\n",
            "healthier lifestyle by promoting exercise and healthy food. This study provides\n",
            "a content analysis of fitspiration imagery on the social networking site\n",
            "Instagram. A set of 600 images were coded for body type, activity,\n",
            "objectification and textual elements. Results showed that the majority of images\n",
            "of women contained only one body type: thin and toned. In addition, most images\n",
            "contained objectifying elements. Accordingly, while fitspiration images may be\n",
            "inspirational for viewers, they also contain a number of elements likely to have\n",
            "negative effects on the viewer’s body image.\n",
            "Relevant? (y/n): y\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we check whether our input has been saved:"
      ],
      "metadata": {
        "id": "TEMUGVWaqWPn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the result\n",
        "all_data.iloc[index]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UPvuxfbrhIzK",
        "outputId": "8b01c7f6-d7ca-4752-c3c3-b78ce9fa26d3"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Unnamed: 0                                                        0\n",
              "Cites                                                           479\n",
              "Authors                                     M Tiggemann, M Zaccardo\n",
              "Title             'Strong is the new skinny': A content analysis...\n",
              "Year                                                         2018.0\n",
              "Source                                 Journal of health psychology\n",
              "Publisher                                      journals.sagepub.com\n",
              "ArticleURL        https://journals.sagepub.com/doi/abs/10.1177/1...\n",
              "CitesURL          https://scholar.google.com/scholar?cites=12266...\n",
              "GSRank                                                            3\n",
              "QueryDate                                       2022-09-08 10:36:01\n",
              "Type                                                            NaN\n",
              "DOI                                        10.1177/1359105316639436\n",
              "ISSN                                                            NaN\n",
              "CitationURL                                                     NaN\n",
              "Volume                                                          NaN\n",
              "Issue                                                           NaN\n",
              "StartPage                                                       NaN\n",
              "EndPage                                                         NaN\n",
              "ECC                                                             479\n",
              "CitesPerYear                                                 119.75\n",
              "CitesPerAuthor                                                  240\n",
              "AuthorCount                                                       2\n",
              "Age                                                             4.0\n",
              "Abstract           ‘Fitspiration’ is an online trend designed to...\n",
              "FullTextURL                                                     NaN\n",
              "RelatedURL        https://scholar.google.com/scholar?q=related:c...\n",
              "Relevant                                                       True\n",
              "Notes                                                           NaN\n",
              "Checked                                                        True\n",
              "Name: 0, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Using GPT to extract information from abstracts\n",
        "\n",
        "Now for the fun part: Is it possible to use GPT to help us during the review process? We are going to try and extract text features automatically. For the moment we are going to use `gpt3.5-turbo`.\n",
        "\n",
        "**Note:**\n",
        "Please feel free to test different prompts and questions. The [Promptingguide](https://www.promptingguide.ai/) is a good resource to learn more about different prompting techniques. Use the [ChatGPT](https://chat.openai.com/) interface to cheaply test prompt prior to using them with the API. Use the [OpenAI Playground](https://platform.openai.com/playground) to optimize your prompts with a visual user interface for different settings and a prompting history (trust me, this can save your life!).\n",
        "\n",
        "**Prompts:** We're going to use the **system prompt** for our instructions, and the **user prompt** to send our content."
      ],
      "metadata": {
        "id": "iHqbR57EiKLE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "system_prompt = \"\"\"\n",
        "You're an advanced AI research assistant. Your task is to extract **research questions**, **operationalization**, **data sources**, **population**, and **scientific disciplines** from user input. Return \"None\" if you can't find the information in user input.\n",
        "\n",
        "**Formatting**\n",
        "Return a markdown table, one row for each extracted feature: **research questions**, **operationalization**, **data sources**, **population**, and **scientific disciplines**.\n",
        "\"\"\""
      ],
      "metadata": {
        "id": "S55s7wPAieZn"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Please enter your API-Code in the next code cell for the `openai.api_key` variable. We have changed the cell to include the `gpt_prompt` variable, which sends the title and abstract as a **user prompt**. We're using the `openai.ChatCompletion.create()` method to send our request to the API. We expect the response in `api_response['choices'][0]['message']['content']` to be markdown (see prompt above), as such we display the markdown in our notebook."
      ],
      "metadata": {
        "id": "c6doRGjfrEtK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from crossref_commons.retrieval import get_publication_as_json\n",
        "import json\n",
        "import openai\n",
        "import textwrap\n",
        "import IPython\n",
        "import re\n",
        "\n",
        "# Enter OpenAI API-Code\n",
        "openai.api_key = \"sk-XXXX\"\n",
        "\n",
        "# Get one row: Not checked, highest Citation count.\n",
        "highest_cites_unchecked = all_data[all_data['Checked'] == False].sort_values(by=\"Cites\", ascending=False).iloc[0]\n",
        "index = highest_cites_unchecked.name\n",
        "\n",
        "# Retrieve Abstract from Crossref\n",
        "response = get_publication_as_json(highest_cites_unchecked['DOI'])\n",
        "abstract = response.get(\"abstract\", \"\")\n",
        "\n",
        "# Remove XML\n",
        "abstract = re.sub(r'<[^>]+>', '', abstract)\n",
        "all_data.loc[index, 'Abstract'] = abstract\n",
        "\n",
        "# Display all information (before we send the request to OpenAI)\n",
        "IPython.display.clear_output(wait=True)\n",
        "title_disp = IPython.display.HTML(\"<h2>{}</h2>\".format(highest_cites_unchecked['Title']))\n",
        "authors_disp = IPython.display.HTML(\"<p>{}</p>\".format(highest_cites_unchecked['Authors']))\n",
        "doi_disp = IPython.display.HTML(\"<p><a target='_blank' href='https://doi.org/{}'>{}</a></p>\".format(highest_cites_unchecked['DOI'],highest_cites_unchecked['DOI']))\n",
        "display(title_disp, authors_disp, doi_disp)\n",
        "print(textwrap.fill(abstract, 80))\n",
        "\n",
        "gpt_prompt = f\"\"\"\n",
        "**Title**: {highest_cites_unchecked['Title']}\n",
        "**Abstract**: {abstract}\n",
        "\"\"\"\n",
        "\n",
        "# Sending request, takes a moment. In the meantime you may read the abstract.\n",
        "messages = [\n",
        "    {\"role\": \"system\", \"content\": system_prompt},\n",
        "    {\"role\": \"user\", \"content\": abstract}\n",
        "]\n",
        "\n",
        "try:\n",
        "  api_response = openai.ChatCompletion.create(\n",
        "      model=\"gpt-3.5-turbo\",\n",
        "      messages=messages,\n",
        "      temperature=0,\n",
        "      timeout=30\n",
        "    )\n",
        "\n",
        "  gpt_result = api_response['choices'][0]['message']['content']\n",
        "\n",
        "  # Display the GPT result\n",
        "  display(IPython.display.HTML(f\"<h3>GPT Extracted Data</h3>\"))\n",
        "  display(IPython.display.Markdown(gpt_result))\n",
        "except:\n",
        "  print(\"GPT API Error\")\n",
        "\n",
        "relevant_input = input('Relevant? (y/n): ').lower().strip() == 'y'\n",
        "\n",
        "# Save user input\n",
        "all_data.loc[index, 'Checked'] = True\n",
        "all_data.loc[index, 'Relevant'] = relevant_input"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 655
        },
        "id": "uHbHo0F1iiCG",
        "outputId": "55f56ab1-d945-47f7-8a86-f3f0a8859c78"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<h2>'Written in these scars are the stories I can't explain': a content analysis of pro-ana and thinspiration image sharing on Instagram</h2>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<p>D Ging, S Garvey</p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<p><a target='_blank' href='https://doi.org/10.1177/1461444816687288'>10.1177/1461444816687288</a></p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Since pro-anorexia websites began to appear in the 1990s, there has been a\n",
            "growing body of academic work on pro-ana and thinspiration communities online.\n",
            "Underpinned by a range of (inter)disciplinary perspectives, most of this work\n",
            "focuses on websites and blogs. There is a dearth of research and, in particular,\n",
            "gender-aware research on pro-ana practices and discourses in the context of\n",
            "newer mobile social platforms such as Instagram. Using a dataset of 7560 images,\n",
            "this study employs content analysis to ask whether, to what extent and how pro-\n",
            "ana identities and discourses manifest themselves on a more open, image-based\n",
            "platform such as Instagram. We demonstrate that, by mainstreaming pro-ana,\n",
            "Instagram has rendered visible pro-ana sensibilities such as abstinence and\n",
            "self-discipline in the broader context of distressed girls’ lives and Western\n",
            "culture more generally. We conclude that this increased visibility may in fact\n",
            "be a positive development.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<h3>GPT Extracted Data</h3>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "| Feature                | Value                                                                                                                                                                                                                   |\n|------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Research Questions     | - To what extent and how do pro-ana identities and discourses manifest themselves on Instagram? <br>- Has Instagram mainstreamed pro-ana sensibilities such as abstinence and self-discipline? <br>- Is the increased visibility of pro-ana on Instagram a positive development? |\n| Operationalization     | Content analysis of a dataset of 7560 images                                                                                                                                                                            |\n| Data Sources           | Instagram                                                                                                                                                                                                               |\n| Population             | Distressed girls and Western culture                                                                                                                                                                                    |\n| Scientific Disciplines | - Social sciences <br>- Gender studies <br>- Cultural studies                                                                                                                                                           |"
          },
          "metadata": {}
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Relevant? (y/n): y\n"
          ]
        }
      ]
    }
  ]
}