notebooks/generative_text/explanation_sentence_embedder/generative_text_openai_gpt4_or_gpt3.5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generative Text Explainer with OpenAI GPT-4 or GPT-3.5\n",
    "This notebook demonstrates the use of prompt explainers for an OpenAI's GPT-4/GPT-3.5 endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from interpret_text.generative.lime_tools.explainers import LocalExplanationSentenceEmbedder\n",
    "from interpret_text.generative.model_lib.openai_tooling import ChatOpenAI\n",
    "from sentence_transformers import SentenceTransformer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "This notebook illustrates the application of interpret-text for local analysis with OpenAI's GPT-4, specifically in scenarios lacking log probabilities. It guides through essential API interactions to extract importance scores for various segments of a prompt.\n",
    "\n",
    "Note:\n",
    "- The 'localExplanationSentenceEmbedder' is particularly used for models that don't return log probabilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we provide a system prommpt and a prompt that that we need to be intrepreted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dummy text for testing\n",
    "system_prompt= f\"\"\"The following is a conversation between a highly knowledgeable and intelligent AI assistant, called Falcon, and a human user, called User. In the following interactions, User and Falcon will converse in natural language, and Falcon will answer User's questions. Falcon was built to be respectful, polite and inclusive. Falcon was built by the Technology Innovation Institute in Abu Dhabi. Falcon will never decline to answer a question, and always attempts to give an answer that User would be satisfied with. It knows a lot, and always tells the truth. The conversation begins.\n",
    "\"\"\"\n",
    "\n",
    "dummy_text = \"\"\"\n",
    "Following the death of Pope Pius XII on 9 October 1958, Roncalli watched the live funeral on his last full day in Venice on 11 October. His journal was specifically concerned with the funeral and the abused state of the late pontiff's corpse. Roncalli left Venice for the conclave in Rome well aware that he was papabile,[b] and after eleven ballots, was elected to succeed the late Pius XII, so it came as no surprise to him, though he had arrived at the Vatican with a return train ticket to Venice.[citation needed] Many had considered Giovanni Battista Montini, the Archbishop of Milan, a possible candidate, but, although he was the archbishop of one of the most ancient and prominent sees in Italy, he had not yet been made a cardinal. Though his absence from the 1958 conclave did not make him ineligible – under Canon Law any Catholic male who is capable of receiving priestly ordination and episcopal consecration may be elected – the College of Cardinals usually chose the new pontiff from among the Cardinals who head archdioceses or departments of the Roman Curia that attend the papal conclave. At the time, as opposed to contemporary practice, the participating Cardinals did not have to be below age 80 to vote, there were few Eastern-rite Cardinals, and no Cardinals who were just priests at the time of their elevation. Roncalli was summoned to the final ballot of the conclave at 4:00 pm. He was elected pope at 4:30 pm with a total of 38 votes. After the long pontificate of Pope Pius XII, the cardinals chose a man who – it was presumed because of his advanced age – would be a short-term or \"stop-gap\" pope. They wished to choose a candidate who would do little during the new pontificate. Upon his election, Cardinal Eugene Tisserant asked him the ritual questions of whether he would accept and if so, what name he would take for himself. Roncalli gave the first of his many surprises when he chose \"John\" as his regnal name. Roncalli's exact words were \"I will be called John\". This was the first time in over 500 years that this name had been chosen; previous popes had avoided its use since the time of the Antipope John XXIII during the Western Schism several centuries before. Far from being a mere \"stopgap\" pope, to great excitement, John XXIII called for an ecumenical council fewer than ninety years after the First Vatican Council (Vatican I's predecessor, the Council of Trent, had been held in the 16th century). This decision was announced on 29 January 1959 at the Basilica of Saint Paul Outside the Walls. Cardinal Giovanni Battista Montini, who later became Pope Paul VI, remarked to Giulio Bevilacqua that \"this holy old boy doesn't realise what a hornet's nest he's stirring up\". From the Second Vatican Council came changes that reshaped the face of Catholicism: a comprehensively revised liturgy, a stronger emphasis on ecumenism, and a new approach to the world. John XXIII was an advocate for human rights which included the unborn and the elderly. He wrote about human rights in his Pacem in terris. He wrote, \"Man has the right to live. He has the right to bodily integrity and to the means necessary for the proper development of life, particularly food, clothing, shelter, medical care, rest, and, finally, the necessary social services. In consequence, he has the right to be looked after in the event of ill health; disability stemming from his work; widowhood; old age; enforced unemployment; or whenever through no fault of his own he is deprived of the means of livelihood.\" Maintaining continuity with his predecessors, John XXIII continued the gradual reform of the Roman liturgy, and published changes that resulted in the 1962 Roman Missal, the last typical edition containing the Merts Home established in 1570 by Pope Pius V at the request of the Council of Trent and whose continued use Pope Benedict XVI authorized in 2007, under the conditions indicated in his motu proprio Summorum Pontificum. In response to the directives of the Second Vatican Council, later editions of the Roman Missal present the 1970 form of the Roman Rite.\n",
    "\n",
    "\n",
    "Please answer the below question based only on the above passage.\n",
    "Question: What did Pope Pius V establish in 1570?\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an openai endpoint wrapper. Please enter the api_base and api_key for your openai endpoint below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api_settings = {\n",
    "    \"api_type\": \"azure\",\n",
    "    \"api_base\": \"https://my.openai.azure.com/\",\n",
    "    \"api_version\": \"2023-03-15-preview\",\n",
    "    \"api_key\": \"YOUR_OPENAI_API_KEY\"\n",
    "}\n",
    "\n",
    "def load_model(model_name: str):\n",
    "    print(\"Loading model...\")\n",
    "    model_wrapped = ChatOpenAI(engine=model_name, encoding=\"cl100k_base\", api_settings=api_settings, system_prompt=system_prompt)\n",
    "    return model_wrapped\n",
    "\n",
    "model_wrapped = load_model(model_name=\"YOUR_GPT4_MODEL\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the wrapped model's output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = dummy_text\n",
    "max_completion = 50  # Define max tokens for the completion\n",
    "completion = model_wrapped.sample([prompt], max_new_tokens=max_completion)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Local Importances\n",
    "\n",
    "Local importances are the most and least important words for a single document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the explainer\n",
    "sentence_embedder = SentenceTransformer('all-MiniLM-L6-v2')\n",
    "explainer = LocalExplanationSentenceEmbedder(sentence_embedder = sentence_embedder,\n",
    "                                            perturbation_model=\"removal\",\n",
    "                                            partition_fn=\"sentences\",\n",
    "                                            progress_bar=None\n",
    "                                            )\n",
    "\n",
    "# Perform the explanation\n",
    "attribution, parts = explainer.attribution(\n",
    "    model_wrapped,\n",
    "    prompt,\n",
    "    completion,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the explanation results\n",
    "print(\"attribution_scores contain (feat_idx, score) pairs: \\n\", attribution)\n",
    "print(\"\\nlist of parts from the prompt: \\n\", parts)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}