notebooks/qg/tableqg_inference.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "489bc0df",
   "metadata": {},
   "source": [
    "# Table Question Generation: Inference example\n",
    "\n",
    "In this notebook, we will show how to use a pretrained TableQG model to generate questions. We first sample SQL queries from a given table, and then use a text-to-text transformer (T5) to transcribe the SQL query to a natural language question. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bbe1173",
   "metadata": {},
   "source": [
    "## Dependencies\n",
    "\n",
    "If not already done, make sure to install PrimeQA with notebooks extras before getting started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "815cec1c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5d06939ddf004ebcbece7036b791630a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading https://raw.githubusercontent.com/stanfordnlp/stanza-resources/main/resources_1.4.0.json:   0%|   …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-08-01 12:12:25 INFO: Downloading default packages for language: multilingual (multilingual)...\n",
      "2022-08-01 12:12:26 INFO: File exists: /u/jaydesen/stanza_resources/multilingual/default.zip\n",
      "2022-08-01 12:12:26 INFO: Finished downloading models and saved to /u/jaydesen/stanza_resources.\n"
     ]
    }
   ],
   "source": [
    "from primeqa.qg.models.qg_model import QGModel\n",
    "from tabulate import tabulate # only used to visualize table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9f309b8",
   "metadata": {},
   "source": [
    "## Loading pretrained model from huggingface\n",
    "\n",
    "This model was trained using PrimeQA library and uploaded to huggingface."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2a3c5377",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'PrimeQA/t5-base-table-question-generator'\n",
    "table_qg_model = QGModel(model_name, modality='table')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "984e88f0",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "## Sample table\n",
    "\n",
    "Tables should be passed a `list` of `dicts`. Each `dict` will correspond to a table with keys `\"header\"` and `\"rows\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "18419939",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n",
      "| Player        |   No. | Nationality   | Position       | Years in Toronto   | School Team    |\n",
      "+===============+=======+===============+================+====================+================+\n",
      "| Antonio Lang  |    21 | United States | Guard-Forward  | 1999-2000          | Duke           |\n",
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n",
      "| Voshon Lenard |     2 | United States | Guard          | 2002-03            | Minnesota      |\n",
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n",
      "| Martin Lewis  |    32 | United States | Guard-Forward  | 1996-97            | Butler CC (KS) |\n",
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n",
      "| Brad Lohaus   |    33 | United States | Forward-Center | 1996               | Iowa           |\n",
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n",
      "| Art Long      |    42 | United States | Forward-Center | 2002-03            | Cincinnati     |\n",
      "+---------------+-------+---------------+----------------+--------------------+----------------+\n"
     ]
    }
   ],
   "source": [
    "table_list = [\n",
    "    {\"header\": [\"Player\", \"No.\", \"Nationality\", \"Position\", \"Years in Toronto\", \"School Team\"],\n",
    "      \"rows\": [\n",
    "            [\"Antonio Lang\", 21, \"United States\", \"Guard-Forward\", \"1999-2000\", \"Duke\"],\n",
    "            [\"Voshon Lenard\", 2, \"United States\", \"Guard\", \"2002-03\", \"Minnesota\"],\n",
    "            [\"Martin Lewis\", 32, \"United States\", \"Guard-Forward\", \"1996-97\", \"Butler CC (KS)\"],\n",
    "            [\"Brad Lohaus\", 33, \"United States\", \"Forward-Center\", \"1996\", \"Iowa\"],\n",
    "            [\"Art Long\", 42, \"United States\", \"Forward-Center\", \"2002-03\", \"Cincinnati\"]\n",
    "        ]\n",
    "    }\n",
    "]\n",
    "# [optional] include an id_list aligned with table_list\n",
    "id_list = [\"abcID123\"]\n",
    "\n",
    "print(tabulate(table_list[0]['rows'], headers=table_list[0]['header'], tablefmt='grid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b002086c",
   "metadata": {},
   "source": [
    "## Generate questions\n",
    "\n",
    "There are some arguments to control the type of questions generated.\n",
    "#### Controls:\n",
    "- `num_questions_per_instance`: Number of questions to generate per table (default=5)\n",
    "- `agg_prob`: Probability distribution over aggregates. `agg_prob` should be a probability vector of length 6, with each index giving prob of an aggregate ops appearing in this order `['select', 'maximum', 'minimum', 'count', 'sum', 'average']`. (default=`[1,0,0,0,0,0]`)\n",
    "- `num_where_prob`: Should be a vector of size 5 with probablities of number of where clauses to use while generating sqls. If k where clause can't be generated the code tries to generate k-1 where clause query, and so on. (default: [0,1,0,0,0])\n",
    "- `ineq_prob`: Probability of generating inequality clauses in SQLs. It should be a float.(default=0.0)\n",
    "- `id_list`: Include an id_list of tables aligned with table_list, defaults to empty list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "40b38a3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'context_id': 'abcID123',\n",
       "  'question': 'Name the years in toronto for duke',\n",
       "  'answer': '1999-2000'},\n",
       " {'context_id': 'abcID123',\n",
       "  'question': 'Who is the player with number 32.0?',\n",
       "  'answer': 'martin lewis'},\n",
       " {'context_id': 'abcID123',\n",
       "  'question': \"What is Voshon Lenard's nationality?\",\n",
       "  'answer': 'united states'},\n",
       " {'context_id': 'abcID123',\n",
       "  'question': 'Who is the guard?',\n",
       "  'answer': 'voshon lenard'},\n",
       " {'context_id': 'abcID123',\n",
       "  'question': 'Name the years in toronto for duke',\n",
       "  'answer': '1999-2000'}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table_qg_model.generate_questions(table_list, \n",
    "                                    num_questions_per_instance = 5,\n",
    "                                    agg_prob = [1.,0,0,0,0,0],\n",
    "                                    num_where_prob = [0,1.,0,0,0],\n",
    "                                    ineq_prob = 0.0,\n",
    "                                    id_list=id_list\n",
    "                                )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05abc088",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.13",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "7d21c3d80586c4cada912fbe8386ca6018c05632b838068c2d847893c1029454"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}