notebooks/generative_text/explanation_hierarchical/generative_text_huggingface_gpt.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generative Text Explainer with a HuggingFace GPT Model\n",
    "This notebook demonstrates the use of prompt explainers for a HuggingFace GPT model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret_text.generative.lime_tools.explainers import LocalExplanationHierarchical, LocalExplanationLikelihood\n",
    "from interpret_text.generative.model_lib.hf_tooling import HF_LM\n",
    "from interpret_text.generative.lime_tools.text_utils import (\n",
    "    split_into_sentences,\n",
    "    extract_non_overlapping_ngrams\n",
    ")\n",
    "import transformers\n",
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "This notebook illustrates the application of interpret-text for local analysis with HuggingFace RoBERTa, specifically in scenarios where we have access to log probabilities. It guides through essential API interactions to extract importance scores for various segments of a prompt.\n",
    "\n",
    "Note:\n",
    "- The 'LocalExplanationHierarchical' can be used with either 'LocalExplanationLikelihood' or 'localExplanationSentenceEmbedder'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we provide a prompt that that we need to be intrepreted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dummy text for testing\n",
    "dummy_text = \"\"\"\n",
    "Following the death of Pope Pius XII on 9 October 1958, Roncalli watched the live funeral on his last full day in Venice on 11 October. His journal was specifically concerned with the funeral and the abused state of the late pontiff's corpse. Roncalli left Venice for the conclave in Rome well aware that he was papabile,[b] and after eleven ballots, was elected to succeed the late Pius XII, so it came as no surprise to him, though he had arrived at the Vatican with a return train ticket to Venice.[citation needed] Many had considered Giovanni Battista Montini, the Archbishop of Milan, a possible candidate, but, although he was the archbishop of one of the most ancient and prominent sees in Italy, he had not yet been made a cardinal. Though his absence from the 1958 conclave did not make him ineligible – under Canon Law any Catholic male who is capable of receiving priestly ordination and episcopal consecration may be elected – the College of Cardinals usually chose the new pontiff from among the Cardinals who head archdioceses or departments of the Roman Curia that attend the papal conclave. At the time, as opposed to contemporary practice, the participating Cardinals did not have to be below age 80 to vote, there were few Eastern-rite Cardinals, and no Cardinals who were just priests at the time of their elevation. Roncalli was summoned to the final ballot of the conclave at 4:00 pm. He was elected pope at 4:30 pm with a total of 38 votes. After the long pontificate of Pope Pius XII, the cardinals chose a man who – it was presumed because of his advanced age – would be a short-term or \"stop-gap\" pope. They wished to choose a candidate who would do little during the new pontificate. Upon his election, Cardinal Eugene Tisserant asked him the ritual questions of whether he would accept and if so, what name he would take for himself. Roncalli gave the first of his many surprises when he chose \"John\" as his regnal name. Roncalli's exact words were \"I will be called John\". This was the first time in over 500 years that this name had been chosen; previous popes had avoided its use since the time of the Antipope John XXIII during the Western Schism several centuries before. Far from being a mere \"stopgap\" pope, to great excitement, John XXIII called for an ecumenical council fewer than ninety years after the First Vatican Council (Vatican I's predecessor, the Council of Trent, had been held in the 16th century). This decision was announced on 29 January 1959 at the Basilica of Saint Paul Outside the Walls. Cardinal Giovanni Battista Montini, who later became Pope Paul VI, remarked to Giulio Bevilacqua that \"this holy old boy doesn't realise what a hornet's nest he's stirring up\". From the Second Vatican Council came changes that reshaped the face of Catholicism: a comprehensively revised liturgy, a stronger emphasis on ecumenism, and a new approach to the world. John XXIII was an advocate for human rights which included the unborn and the elderly. He wrote about human rights in his Pacem in terris. He wrote, \"Man has the right to live. He has the right to bodily integrity and to the means necessary for the proper development of life, particularly food, clothing, shelter, medical care, rest, and, finally, the necessary social services. In consequence, he has the right to be looked after in the event of ill health; disability stemming from his work; widowhood; old age; enforced unemployment; or whenever through no fault of his own he is deprived of the means of livelihood.\" Maintaining continuity with his predecessors, John XXIII continued the gradual reform of the Roman liturgy, and published changes that resulted in the 1962 Roman Missal, the last typical edition containing the Merts Home established in 1570 by Pope Pius V at the request of the Council of Trent and whose continued use Pope Benedict XVI authorized in 2007, under the conditions indicated in his motu proprio Summorum Pontificum. In response to the directives of the Second Vatican Council, later editions of the Roman Missal present the 1970 form of the Roman Rite.\n",
    "\n",
    "\n",
    "Please answer the below question based only on the above passage.\n",
    "Question: What did Pope Pius V establish in 1570?\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load Model\n",
    "\n",
    "Create a HuggingFace GPT model wrapper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_model(model_name: str):\n",
    "    print(\"Loading model...\")\n",
    "    tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)\n",
    "    model = transformers.AutoModelForCausalLM.from_pretrained(\n",
    "        model_name, trust_remote_code=True, torch_dtype=torch.bfloat16)\n",
    "    \n",
    "    model = model.eval()\n",
    "    model = model.to(\"cuda\")\n",
    "    model_wrapped = HF_LM(model, tokenizer, device=\"cuda\")\n",
    "    return model_wrapped\n",
    "\n",
    "model_wrapped = load_model(\"EleutherAI/gpt-neo-1.3B\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the wrapped model's output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = dummy_text\n",
    "max_completion = 50  # Define max tokens for the completion\n",
    "completion = model_wrapped.sample([prompt], max_new_tokens=max_completion)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Analysis and Perturbation Setup\n",
    "\n",
    "This cell is dedicated to setting up the text analysis and perturbation framework by defining parameters for text perturbation and feature extraction. Key configurations include:\n",
    "\n",
    "- `ngram_size`, `max_parts_to_perturb`, `num_perturbations`, `max_features`, and `max_completion` to specify the granularity of text analysis and the scope of generated perturbations.\n",
    "- Calculating the number of sentences and non-overlapping n-grams in the prompt to tailor the perturbation strategy.\n",
    "- Establishing a `perturbation_hierarchy` that dynamically adjusts the analysis granularity based on the text's complexity, applying different strategies for sentences and n-grams.\n",
    "- Initializing the `LocalExplanationHierarchical` explainer with specified perturbation models and hierarchies to dissect the influence of various text components on the model's output.\n",
    "- Lastly, the `importance_cache` is generated by attributing importance scores to parts of the text, aiding in understanding the model's focus and decision-making process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "step_explainer_class = LocalExplanationLikelihood\n",
    "ngram_size = 5\n",
    "max_parts_to_perturb = 2\n",
    "num_perturbations = 100\n",
    "max_features = 4\n",
    "max_completion = 5\n",
    "num_sentences = len(split_into_sentences(prompt))\n",
    "num_ngrams = len(extract_non_overlapping_ngrams(prompt, 1))\n",
    "perturbation_hierarchy = []\n",
    "n_total = 8\n",
    "n_log = 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if num_sentences > 1:\n",
    "    cur_n_sent = (num_sentences // n_total) + 1\n",
    "    while True:\n",
    "        perturbation_hierarchy.append({\"partition_fn\": \"n_sentences\", \"partition_kwargs\": {\n",
    "                                        \"n\": cur_n_sent}, \"max_features\": n_log})\n",
    "        cur_n_sent = ((cur_n_sent // 2) + (cur_n_sent % 2))\n",
    "        if cur_n_sent <= 1:\n",
    "            break\n",
    "    perturbation_hierarchy.append(\n",
    "        {\"partition_fn\": \"sentences\", \"partition_kwargs\": {}, \"max_features\": n_log})\n",
    "\n",
    "if num_ngrams > 20:\n",
    "    cur_ngrams = 10\n",
    "    while True:\n",
    "        perturbation_hierarchy.append({\"partition_fn\": \"ngrams\", \"partition_kwargs\": {\n",
    "                                        \"n\": cur_ngrams}, \"max_features\": 5})\n",
    "        cur_ngrams = (cur_ngrams // 2) + (cur_ngrams % 2)\n",
    "        if cur_ngrams <= 9:\n",
    "            break\n",
    "perturbation_hierarchy.append(\n",
    "    {\"partition_fn\": \"ngrams\", \"partition_kwargs\": {\"n\": 5}, \"max_features\": 5})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Local Importances\n",
    "\n",
    "Local importances are the most and least important words for a single document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainer = LocalExplanationHierarchical(\n",
    "    perturbation_model=\"removal\",\n",
    "    perturbation_hierarchy=perturbation_hierarchy,\n",
    "    progress_bar=None,\n",
    "    step_explainer=step_explainer_class,\n",
    ")\n",
    "\n",
    "importance_cache = explainer.attribution(\n",
    "    model_wrapped,\n",
    "    prompt,\n",
    "    completion,\n",
    "    max_parts_to_perturb=max_parts_to_perturb,\n",
    "    max_features=max_features,\n",
    "    n_samples=num_perturbations\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importance_cache"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}