decoding-python-with-stack-overflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a309187a",
   "metadata": {
    "papermill": {
     "duration": 0.014822,
     "end_time": "2024-03-03T20:17:31.560724",
     "exception": false,
     "start_time": "2024-03-03T20:17:31.545902",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Decoding Python with Stack Overflow: A Personalized Assistant with Gemma 2b\n",
    "\n",
    "![Image description](https://helios-i.mashable.com/imagery/articles/00veJ5qeI90cfXdfFzUfCrv/hero-image.fill.size_1248x702.v1708464912.jpg)\n",
    "\n",
    "\n",
    "\n",
    "I've developed a personalized assistant powered by Gemma 2b, designed to help you navigate Python programming queries sourced from Stack Overflow. This project leverages Gemma 2b, an advanced open model tailored for assisting with Python questions, to provide tailored assistance and solutions.\n",
    "\n",
    "## Introduction to Gemma 2b\n",
    "\n",
    "Gemma 2b, a part of the Gemma model family, offers pre-trained and instruction-tuned variants specifically optimized for assisting with Python programming queries. Developed by Google DeepMind and other teams at Google, Gemma 2b inherits the cutting-edge research and technology from Gemini models, providing developers with valuable tools and guidance for responsible AI development.\n",
    "\n",
    "### Key Features of Gemma 2b\n",
    "\n",
    "- **Tailored Assistance**: Gemma 2b provides personalized assistance for Python programming queries sourced from Stack Overflow, offering solutions tailored to the specific needs of developers.\n",
    "\n",
    "- **Responsible AI Toolkit**: Developers have access to a comprehensive toolkit that guides responsible and ethical usage of Gemma 2b, ensuring the development of safer AI applications.\n",
    "\n",
    "- **Framework Integration**: Gemma 2b seamlessly integrates with major frameworks like JAX, PyTorch, and TensorFlow, offering toolchains for inference and supervised fine-tuning (SFT).\n",
    "\n",
    "- **Ready-to-Use Notebooks**: Ready-to-use Colab and Kaggle notebooks simplify the process of getting started with Gemma 2b, enabling developers to dive into their projects quickly.\n",
    "\n",
    "- **Deployment Flexibility**: Gemma 2b models can be deployed on various platforms, including laptops, workstations, and Google Cloud, with easy deployment options on Vertex AI and Google Kubernetes Engine (GKE).\n",
    "\n",
    "- **Performance Optimization**: Gemma 2b is optimized for industry-leading performance across multiple AI hardware platforms, including NVIDIA GPUs and Google Cloud TPUs..\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bbbc95f8",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:17:31.583769Z",
     "iopub.status.busy": "2024-03-03T20:17:31.583476Z",
     "iopub.status.idle": "2024-03-03T20:17:31.610444Z",
     "shell.execute_reply": "2024-03-03T20:17:31.609438Z"
    },
    "papermill": {
     "duration": 0.040718,
     "end_time": "2024-03-03T20:17:31.612507",
     "exception": false,
     "start_time": "2024-03-03T20:17:31.571789",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/data-assistants-with-gemma/submission_categories.txt\n",
      "/kaggle/input/data-assistants-with-gemma/submission_instructions.txt\n",
      "/kaggle/input/gemma/pytorch/2b-it/1/config.json\n",
      "/kaggle/input/gemma/pytorch/2b-it/1/gemma-2b-it.ckpt\n",
      "/kaggle/input/gemma/pytorch/2b-it/1/tokenizer.model\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/2/config.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/2/tokenizer.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/2/metadata.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/2/model.weights.h5\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/2/assets/tokenizer/vocabulary.spm\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/1/config.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/1/tokenizer.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/1/metadata.json\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/1/model.weights.h5\n",
      "/kaggle/input/gemma/keras/gemma_2b_en/1/assets/tokenizer/vocabulary.spm\n",
      "/kaggle/input/gemma/transformers/2b-it/2/model.safetensors.index.json\n",
      "/kaggle/input/gemma/transformers/2b-it/2/gemma-2b-it.gguf\n",
      "/kaggle/input/gemma/transformers/2b-it/2/config.json\n",
      "/kaggle/input/gemma/transformers/2b-it/2/model-00001-of-00002.safetensors\n",
      "/kaggle/input/gemma/transformers/2b-it/2/model-00002-of-00002.safetensors\n",
      "/kaggle/input/gemma/transformers/2b-it/2/tokenizer.json\n",
      "/kaggle/input/gemma/transformers/2b-it/2/tokenizer_config.json\n",
      "/kaggle/input/gemma/transformers/2b-it/2/special_tokens_map.json\n",
      "/kaggle/input/gemma/transformers/2b-it/2/.gitattributes\n",
      "/kaggle/input/gemma/transformers/2b-it/2/tokenizer.model\n",
      "/kaggle/input/gemma/transformers/2b-it/2/generation_config.json\n",
      "/kaggle/input/pythonquestions/Answers.csv\n",
      "/kaggle/input/pythonquestions/Questions.csv\n",
      "/kaggle/input/pythonquestions/Tags.csv\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec108d27",
   "metadata": {
    "papermill": {
     "duration": 0.008934,
     "end_time": "2024-03-03T20:17:31.630622",
     "exception": false,
     "start_time": "2024-03-03T20:17:31.621688",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Installing Project Dependencies\n",
    "\n",
    "This section outlines the installation process for the required dependencies used in this project. \n",
    "\n",
    "Each line represents a specific command executed within the notebook environment to install a Python package using `pip3`:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "63c676fb",
   "metadata": {
    "_kg_hide-input": false,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:17:31.649721Z",
     "iopub.status.busy": "2024-03-03T20:17:31.649469Z",
     "iopub.status.idle": "2024-03-03T20:18:22.983756Z",
     "shell.execute_reply": "2024-03-03T20:18:22.982692Z"
    },
    "papermill": {
     "duration": 51.346602,
     "end_time": "2024-03-03T20:18:22.986269",
     "exception": false,
     "start_time": "2024-03-03T20:17:31.639667",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /opt/conda/lib/python3.10/site-packages (23.3.2)\r\n",
      "Collecting pip\r\n",
      "  Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)\r\n",
      "Downloading pip-24.0-py3-none-any.whl (2.1 MB)\r\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m34.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hInstalling collected packages: pip\r\n",
      "  Attempting uninstall: pip\r\n",
      "    Found existing installation: pip 23.3.2\r\n",
      "    Uninstalling pip-23.3.2:\r\n",
      "      Successfully uninstalled pip-23.3.2\r\n",
      "Successfully installed pip-24.0\r\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n",
      "tensorflow 2.15.0 requires keras<2.16,>=2.15.0, but you have keras 3.0.5 which is incompatible.\r\n",
      "tensorflowjs 4.16.0 requires packaging~=23.1, but you have packaging 21.3 which is incompatible.\u001b[0m\u001b[31m\r\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "# !pip install -U keras-nlp\n",
    "# !pip install -U keras\n",
    "# !pip install -U trl\n",
    "\n",
    "# !pip install -U datasets\n",
    "# !pip install -U tf_keras\n",
    "# !pip install -U nltk\n",
    "# !pip install -U tensorflow-text\n",
    "# !pip install -U adapters\n",
    "!pip install --upgrade pip\n",
    "!pip install -q -U keras-nlp\n",
    "!pip install -q -U keras>=3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "007fd13d",
   "metadata": {
    "papermill": {
     "duration": 0.010079,
     "end_time": "2024-03-03T20:18:23.007189",
     "exception": false,
     "start_time": "2024-03-03T20:18:22.997110",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Importing Modules\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "74b4054d",
   "metadata": {
    "_kg_hide-input": false,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:18:23.029474Z",
     "iopub.status.busy": "2024-03-03T20:18:23.029167Z",
     "iopub.status.idle": "2024-03-03T20:18:35.308829Z",
     "shell.execute_reply": "2024-03-03T20:18:35.307833Z"
    },
    "papermill": {
     "duration": 12.293097,
     "end_time": "2024-03-03T20:18:35.311281",
     "exception": false,
     "start_time": "2024-03-03T20:18:23.018184",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-03-03 20:18:24.672147: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2024-03-03 20:18:24.672272: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2024-03-03 20:18:24.803359: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n"
     ]
    }
   ],
   "source": [
    "# from datasets import load_dataset\n",
    "# from trl import SFTTrainer\n",
    "# from peft import LoraConfig\n",
    "# from transformers import AutoTokenizer, AutoModelForCausalLM\n",
    "# from transformers import BitsAndBytesConfig, GemmaTokenizer\n",
    "# import torch\n",
    "# import transformers\n",
    "import keras_nlp\n",
    "import keras\n",
    "import os\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dda89149",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:18:35.333355Z",
     "iopub.status.busy": "2024-03-03T20:18:35.332824Z",
     "iopub.status.idle": "2024-03-03T20:18:35.337275Z",
     "shell.execute_reply": "2024-03-03T20:18:35.336418Z"
    },
    "papermill": {
     "duration": 0.017438,
     "end_time": "2024-03-03T20:18:35.339223",
     "exception": false,
     "start_time": "2024-03-03T20:18:35.321785",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Select JAX as Keras backend \n",
    "os.environ[\"KERAS_BACKEND\"] = \"jax\"\n",
    "# Pre-allocate 100% of memory to minimize memory fragmentation and allocation overhead\n",
    "os.environ[\"XLA_PYTHON_CLIENT_MEM_FRACTION\"]=\"1.00\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "804b4e5a",
   "metadata": {
    "papermill": {
     "duration": 0.009772,
     "end_time": "2024-03-03T20:18:35.358931",
     "exception": false,
     "start_time": "2024-03-03T20:18:35.349159",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Choosing the Gemma Model for Kaggle\n",
    "\n",
    "Gemma offers a variety of large language models, each with different parameter sizes and fine-tuning options. Selecting the right model for your Kaggle notebook is crucial for both memory constraints and optimal performance.\n",
    "\n",
    "**Available Models:**\n",
    "\n",
    "| Model Name | Parameters | Fine-tuned? | Kaggle-Compatible? |\n",
    "|---|---|---|---|\n",
    "| Gemma 7b instruct | 7 billion | Yes | No (exceeds memory limit) |\n",
    "| Gemma 2b instruct | 2 billion | Yes | Yes (recommended) |\n",
    "| Gemma 7b | 7 billion | No | No (exceeds memory limit) |\n",
    "| Gemma 2b | 2 billion | No | Yes |\n",
    "\n",
    "**Memory Limitation:**\n",
    "\n",
    "Kaggle notebooks have a 30 GB memory limit. Opting for models larger than 2 billion parameters (like Gemma 7b or 7b instruct) will likely lead to kernel crashes.\n",
    "\n",
    "**Recommendation:**\n",
    "\n",
    "For Kaggle notebooks, **Gemma 2b** is the sweet spot, offering fine-tuning capabilities while staying within memory constraints.\n",
    "\n",
    "**Additional Tips:**\n",
    "\n",
    "- Consider your specific task and needs when choosing a model.\n",
    "- Consult the Gemma documentation for detailed information: https://ai.google.dev/gemma/\n",
    "- Remember, larger models often require more data and computational resources.\n",
    "\n",
    "I hope this formatted Markdown version provides a clear and informative guide!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "368a94df",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:18:35.379870Z",
     "iopub.status.busy": "2024-03-03T20:18:35.379597Z",
     "iopub.status.idle": "2024-03-03T20:19:36.756989Z",
     "shell.execute_reply": "2024-03-03T20:19:36.755981Z"
    },
    "papermill": {
     "duration": 61.390544,
     "end_time": "2024-03-03T20:19:36.759366",
     "exception": false,
     "start_time": "2024-03-03T20:18:35.368822",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Kaggle notebook...\n",
      "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Kaggle notebook...\n",
      "Attaching 'model.weights.h5' from model 'keras/gemma/keras/gemma_2b_en/2' to your Kaggle notebook...\n",
      "Attaching 'tokenizer.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Kaggle notebook...\n",
      "Attaching 'assets/tokenizer/vocabulary.spm' from model 'keras/gemma/keras/gemma_2b_en/2' to your Kaggle notebook...\n",
      "normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.\n"
     ]
    }
   ],
   "source": [
    "# Create an instance of GemmaCausalLM from the preset\n",
    "gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(\"gemma_2b_en\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10ba34d2",
   "metadata": {
    "papermill": {
     "duration": 0.010341,
     "end_time": "2024-03-03T20:19:36.780687",
     "exception": false,
     "start_time": "2024-03-03T20:19:36.770346",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Testing the Model :\n",
    "## Generating Text with a Single Prompt\n",
    "\n",
    "We're using the `generate` method of the `gemma_lm` language model to generate text based on a single prompt. The prompt provided is \"a quote in 30 words\", and the maximum length of the generated text is restricted to 30 words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9234819b",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:19:36.802617Z",
     "iopub.status.busy": "2024-03-03T20:19:36.802327Z",
     "iopub.status.idle": "2024-03-03T20:19:57.562636Z",
     "shell.execute_reply": "2024-03-03T20:19:57.561539Z"
    },
    "papermill": {
     "duration": 20.774126,
     "end_time": "2024-03-03T20:19:57.565098",
     "exception": false,
     "start_time": "2024-03-03T20:19:36.790972",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
      "I0000 00:00:1709497196.408782      26 device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.\n",
      "W0000 00:00:1709497196.474946      26 graph_launch.cc:671] Fallback to op-by-op mode because memset node breaks graph update\n",
      "W0000 00:00:1709497196.519525      26 graph_launch.cc:671] Fallback to op-by-op mode because memset node breaks graph update\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single Prompt Result:\n",
      "A quote in 30 words or less.\n",
      "\n",
      "“I’m not a fan of the word ‘diversity’ because it’s so broad. It’s like saying ‘I’m not a fan of the word ‘love’\n"
     ]
    }
   ],
   "source": [
    "# Generate text with a single prompt\n",
    "input_text = \"A quote in 30 words\"\n",
    "# tokenized_ids = tokenizer(input_text, return_tensors=\"pt\")\n",
    "single_prompt_result = gemma_lm.generate(input_text, max_length=50)\n",
    "\n",
    "# Print the result for the single prompt\n",
    "print(\"Single Prompt Result:\")\n",
    "print(single_prompt_result)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "15385d5c",
   "metadata": {
    "_kg_hide-input": false,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:19:57.592089Z",
     "iopub.status.busy": "2024-03-03T20:19:57.591707Z",
     "iopub.status.idle": "2024-03-03T20:19:57.629584Z",
     "shell.execute_reply": "2024-03-03T20:19:57.628507Z"
    },
    "papermill": {
     "duration": 0.053938,
     "end_time": "2024-03-03T20:19:57.631905",
     "exception": false,
     "start_time": "2024-03-03T20:19:57.577967",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Preprocessor: \"gemma_causal_lm_preprocessor\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mPreprocessor: \"gemma_causal_lm_preprocessor\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Tokenizer (type)                                   </span>┃<span style=\"font-weight: bold\">                                             Vocab # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ gemma_tokenizer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaTokenizer</span>)                   │                                             <span style=\"color: #00af00; text-decoration-color: #00af00\">256,000</span> │\n",
       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mTokenizer (type)                                  \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m                                            Vocab #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ gemma_tokenizer (\u001b[38;5;33mGemmaTokenizer\u001b[0m)                   │                                             \u001b[38;5;34m256,000\u001b[0m │\n",
       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"gemma_causal_lm\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"gemma_causal_lm\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                  </span>┃<span style=\"font-weight: bold\"> Output Shape              </span>┃<span style=\"font-weight: bold\">         Param # </span>┃<span style=\"font-weight: bold\"> Connected to               </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ padding_mask (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ token_ids (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ gemma_backbone                │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2048</span>)        │   <span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> │ padding_mask[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>],        │\n",
       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaBackbone</span>)               │                           │                 │ token_ids[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]            │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ token_embedding               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256000</span>)      │     <span style=\"color: #00af00; text-decoration-color: #00af00\">524,288,000</span> │ gemma_backbone[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]       │\n",
       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">ReversibleEmbedding</span>)         │                           │                 │                            │\n",
       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                 \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape             \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m        Param #\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mConnected to              \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ padding_mask (\u001b[38;5;33mInputLayer\u001b[0m)     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ token_ids (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ gemma_backbone                │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2048\u001b[0m)        │   \u001b[38;5;34m2,506,172,416\u001b[0m │ padding_mask[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m],        │\n",
       "│ (\u001b[38;5;33mGemmaBackbone\u001b[0m)               │                           │                 │ token_ids[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]            │\n",
       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
       "│ token_embedding               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m256000\u001b[0m)      │     \u001b[38;5;34m524,288,000\u001b[0m │ gemma_backbone[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]       │\n",
       "│ (\u001b[38;5;33mReversibleEmbedding\u001b[0m)         │                           │                 │                            │\n",
       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> (9.34 GB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m2,506,172,416\u001b[0m (9.34 GB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> (9.34 GB)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m2,506,172,416\u001b[0m (9.34 GB)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "#Fetching models summary involving all the parameters\n",
    "print(gemma_lm.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0aec3254",
   "metadata": {
    "papermill": {
     "duration": 0.013694,
     "end_time": "2024-03-03T20:19:57.659943",
     "exception": false,
     "start_time": "2024-03-03T20:19:57.646249",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Read and Process Data\n",
    "\n",
    "We're reading data from a CSV file named \"Questions.csv\" located at \"/kaggle/input/pythonquestions/\". The data is read into a pandas DataFrame called `df_questions` using the `read_csv` function from the pandas library. We specify the encoding as \"ISO-8859-1\" to handle any special characters in the data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9effce63",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:19:57.689498Z",
     "iopub.status.busy": "2024-03-03T20:19:57.689104Z",
     "iopub.status.idle": "2024-03-03T20:20:27.793464Z",
     "shell.execute_reply": "2024-03-03T20:20:27.792351Z"
    },
    "papermill": {
     "duration": 30.122032,
     "end_time": "2024-03-03T20:20:27.795915",
     "exception": false,
     "start_time": "2024-03-03T20:19:57.673883",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#questions table\n",
    "df_questions = pd.read_csv('../input/pythonquestions/Questions.csv',\n",
    "                            encoding = \"ISO-8859-1\",\n",
    "                            usecols = ['Id','Score','Title'])\n",
    "#answers table\n",
    "df_answers = pd.read_csv('../input/pythonquestions/Answers.csv',\n",
    "                            encoding = \"ISO-8859-1\",\n",
    "                            usecols = ['ParentId','Score','Body'],#parent id links to the questions table\n",
    "                            )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7f56413",
   "metadata": {
    "papermill": {
     "duration": 0.011884,
     "end_time": "2024-03-03T20:20:27.820243",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.808359",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Only taking a subset of training examples for now due to limited RAM . You can try with the entire dataset\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4b0c5a37",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:27.845161Z",
     "iopub.status.busy": "2024-03-03T20:20:27.844817Z",
     "iopub.status.idle": "2024-03-03T20:20:27.849003Z",
     "shell.execute_reply": "2024-03-03T20:20:27.848120Z"
    },
    "papermill": {
     "duration": 0.018934,
     "end_time": "2024-03-03T20:20:27.850874",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.831940",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "questions = df_questions\n",
    "answers = df_answers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afce8aec",
   "metadata": {
    "papermill": {
     "duration": 0.012278,
     "end_time": "2024-03-03T20:20:27.875558",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.863280",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "#### Here, you can see the list of questions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f7b5e897",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:27.900313Z",
     "iopub.status.busy": "2024-03-03T20:20:27.900012Z",
     "iopub.status.idle": "2024-03-03T20:20:27.913225Z",
     "shell.execute_reply": "2024-03-03T20:20:27.912346Z"
    },
    "papermill": {
     "duration": 0.027897,
     "end_time": "2024-03-03T20:20:27.915218",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.887321",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>Score</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>469</td>\n",
       "      <td>21</td>\n",
       "      <td>How can I find the full path to a font from it...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>502</td>\n",
       "      <td>27</td>\n",
       "      <td>Get a preview JPEG of a PDF on Windows?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>535</td>\n",
       "      <td>40</td>\n",
       "      <td>Continuous Integration System for a Python Cod...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>594</td>\n",
       "      <td>25</td>\n",
       "      <td>cx_Oracle: How do I iterate over a result set?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>683</td>\n",
       "      <td>28</td>\n",
       "      <td>Using 'in' to match an attribute of Python obj...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Id  Score                                              Title\n",
       "0  469     21  How can I find the full path to a font from it...\n",
       "1  502     27            Get a preview JPEG of a PDF on Windows?\n",
       "2  535     40  Continuous Integration System for a Python Cod...\n",
       "3  594     25     cx_Oracle: How do I iterate over a result set?\n",
       "4  683     28  Using 'in' to match an attribute of Python obj..."
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "questions[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf0df03e",
   "metadata": {
    "papermill": {
     "duration": 0.012017,
     "end_time": "2024-03-03T20:20:27.940303",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.928286",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Pre-processing:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e2263db4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:27.966608Z",
     "iopub.status.busy": "2024-03-03T20:20:27.966285Z",
     "iopub.status.idle": "2024-03-03T20:20:28.182599Z",
     "shell.execute_reply": "2024-03-03T20:20:28.181810Z"
    },
    "papermill": {
     "duration": 0.232189,
     "end_time": "2024-03-03T20:20:28.184973",
     "exception": false,
     "start_time": "2024-03-03T20:20:27.952784",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "questions = questions[questions['Score'] > 0]\n",
    "answers = answers[answers['Score'] > 0]\\\n",
    "    .sort_values('Score',ascending=False)\\\n",
    "    .drop_duplicates(subset=['ParentId'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f1c1e0e5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:28.211008Z",
     "iopub.status.busy": "2024-03-03T20:20:28.210694Z",
     "iopub.status.idle": "2024-03-03T20:20:28.394599Z",
     "shell.execute_reply": "2024-03-03T20:20:28.393792Z"
    },
    "papermill": {
     "duration": 0.199189,
     "end_time": "2024-03-03T20:20:28.396871",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.197682",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "qa = questions.merge(answers,left_on = 'Id', right_on = 'ParentId')\\\n",
    "    .rename(columns={'Title':'Question','Body':'Answer'})[['Question','Answer','Score_x']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a9495b8",
   "metadata": {
    "papermill": {
     "duration": 0.011817,
     "end_time": "2024-03-03T20:20:28.421226",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.409409",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Choosing first 5000 subset from the entire qa dataframe\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a712724d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:28.446512Z",
     "iopub.status.busy": "2024-03-03T20:20:28.446194Z",
     "iopub.status.idle": "2024-03-03T20:20:28.490190Z",
     "shell.execute_reply": "2024-03-03T20:20:28.489429Z"
    },
    "papermill": {
     "duration": 0.059191,
     "end_time": "2024-03-03T20:20:28.492403",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.433212",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "qa = qa.sort_values(\"Score_x\",ascending=False).head(5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65619528",
   "metadata": {
    "papermill": {
     "duration": 0.012347,
     "end_time": "2024-03-03T20:20:28.517224",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.504877",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Pre-processing of the question dataset \n",
    "- Formatting the data using regex\n",
    "- The function `remove_html_tags_with_space` takes a list of lists (`html_list`) containing strings with HTML tags and removes these tags while replacing them with spaces.\n",
    "- It iterates over each inner list in `html_list`, then iterates over each item in the inner list.\n",
    "- For each item, it removes HTML tags using regular expressions (`re.sub`) and replaces them with spaces.\n",
    "- Additionally, it handles newline characters (`\\n`), consecutive newlines (`\\n\\n`), and carriage return followed by newlines (`\\r\\n\\r\\n`) by replacing them with spaces as well.\n",
    "- Finally, it removes leading and trailing spaces from each cleaned item before appending it to the `clean_inner_list`.\n",
    "- The function returns a new list of lists (`clean_list`) with the HTML tags removed and replaced by spaces in each string.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c40f020a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:28.542803Z",
     "iopub.status.busy": "2024-03-03T20:20:28.542478Z",
     "iopub.status.idle": "2024-03-03T20:20:28.548846Z",
     "shell.execute_reply": "2024-03-03T20:20:28.547978Z"
    },
    "papermill": {
     "duration": 0.021409,
     "end_time": "2024-03-03T20:20:28.550726",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.529317",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def remove_html_tags_with_space(html_list):\n",
    "    clean_list = []\n",
    "    for inner_list in html_list:\n",
    "        clean_inner_list = []\n",
    "        for item in inner_list:\n",
    "            clean_item = re.sub(r'<[^>]*>', ' ', item)  # Remove HTML tags and replace with spaces\n",
    "            clean_item1 = re.sub(r'\\n', ' ',  clean_item)\n",
    "            clean_item2 = re.sub(r'\\n\\n', ' ',  clean_item1)\n",
    "            clean_item3 = re.sub(r'\\r\\n\\r\\n', ' ',  clean_item2)\n",
    "            clean_item4 = re.sub(r'&gt;&gt;&gt;', ' ',  clean_item3)\n",
    "            clean_inner_list.append(clean_item4.strip())  # Remove leading and trailing spaces\n",
    "        clean_list.append(clean_inner_list)\n",
    "    return clean_list\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ce8c1de",
   "metadata": {
    "papermill": {
     "duration": 0.011825,
     "end_time": "2024-03-03T20:20:28.574768",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.562943",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Formatting the Data to get in desired form :\n",
    "\n",
    "**```Desired form : [question : question , answer :[answer]]```**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "877cfbfb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:28.599988Z",
     "iopub.status.busy": "2024-03-03T20:20:28.599715Z",
     "iopub.status.idle": "2024-03-03T20:20:28.892372Z",
     "shell.execute_reply": "2024-03-03T20:20:28.891582Z"
    },
    "papermill": {
     "duration": 0.307854,
     "end_time": "2024-03-03T20:20:28.894540",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.586686",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "train = []\n",
    "for index, row in qa.iterrows():\n",
    "    train.append(f\"Question:\\n{row['Question']}\\n\\nAnswer:\\n{row['Answer']}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c9c8775",
   "metadata": {
    "papermill": {
     "duration": 0.011781,
     "end_time": "2024-03-03T20:20:28.918609",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.906828",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "#### Viewing first 5 examples of our formatted_data variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "89192c43",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:28.943601Z",
     "iopub.status.busy": "2024-03-03T20:20:28.943301Z",
     "iopub.status.idle": "2024-03-03T20:20:28.950196Z",
     "shell.execute_reply": "2024-03-03T20:20:28.949342Z"
    },
    "papermill": {
     "duration": 0.022521,
     "end_time": "2024-03-03T20:20:28.953051",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.930530",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Question:\\nWhat does the \"yield\" keyword do?\\n\\nAnswer:\\n<p>To understand what <code>yield</code> does, you must understand what <em>generators</em> are. And before generators come <em>iterables</em>.</p>\\n\\n<h2>Iterables</h2>\\n\\n<p>When you create a list, you can read its items one by one. Reading its items one by one is called iteration:</p>\\n\\n<pre><code>&gt;&gt;&gt; mylist = [1, 2, 3]\\n&gt;&gt;&gt; for i in mylist:\\n...    print(i)\\n1\\n2\\n3\\n</code></pre>\\n\\n<p><code>mylist</code> is an <em>iterable</em>. When you use a list comprehension, you create a list, and so an iterable:</p>\\n\\n<pre><code>&gt;&gt;&gt; mylist = [x*x for x in range(3)]\\n&gt;&gt;&gt; for i in mylist:\\n...    print(i)\\n0\\n1\\n4\\n</code></pre>\\n\\n<p>Everything you can use \"<code>for... in...</code>\" on is an iterable; <code>lists</code>, <code>strings</code>, files...</p>\\n\\n<p>These iterables are handy because you can read them as much as you wish, but you store all the values in memory and this is not always what you want when you have a lot of values.</p>\\n\\n<h2>Generators</h2>\\n\\n<p>Generators are iterators, but <strong>you can only iterate over them once</strong>. It\\'s because they do not store all the values in memory, <strong>they generate the values on the fly</strong>:</p>\\n\\n<pre><code>&gt;&gt;&gt; mygenerator = (x*x for x in range(3))\\n&gt;&gt;&gt; for i in mygenerator:\\n...    print(i)\\n0\\n1\\n4\\n</code></pre>\\n\\n<p>It is just the same except you used <code>()</code> instead of <code>[]</code>. BUT, you <strong>cannot</strong> perform <code>for i in mygenerator</code> a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.</p>\\n\\n<h2>Yield</h2>\\n\\n<p><code>Yield</code> is a keyword that is used like <code>return</code>, except the function will return a generator.</p>\\n\\n<pre><code>&gt;&gt;&gt; def createGenerator():\\n...    mylist = range(3)\\n...    for i in mylist:\\n...        yield i*i\\n...\\n&gt;&gt;&gt; mygenerator = createGenerator() # create a generator\\n&gt;&gt;&gt; print(mygenerator) # mygenerator is an object!\\n&lt;generator object createGenerator at 0xb7555c34&gt;\\n&gt;&gt;&gt; for i in mygenerator:\\n...     print(i)\\n0\\n1\\n4\\n</code></pre>\\n\\n<p>Here it\\'s a useless example, but it\\'s handy when you know your function will return a huge set of values that you will only need to read once.</p>\\n\\n<p>To master <code>yield</code>, you must understand that <strong>when you call the function, the code you have written in the function body does not run.</strong> The function only returns the generator object, this is a bit tricky :-)</p>\\n\\n<p>Then, your code will be run each time the <code>for</code> uses the generator.</p>\\n\\n<p>Now the hard part:</p>\\n\\n<p>The first time the <code>for</code> calls the generator object created from your function, it will run the code in your function from the beginning until it hits <code>yield</code>, then it\\'ll return the first value of the loop. Then, each other call will run the loop you have written in the function one more time, and return the next value, until there is no value to return.</p>\\n\\n<p>The generator is considered empty once the function runs but does not hit <code>yield</code> anymore. It can be because the loop had come to an end, or because you do not satisfy an <code>\"if/else\"</code> anymore.</p>\\n\\n<hr>\\n\\n<h2>Your code explained</h2>\\n\\n<p>Generator:</p>\\n\\n<pre><code># Here you create the method of the node object that will return the generator\\ndef node._get_child_candidates(self, distance, min_dist, max_dist):\\n\\n  # Here is the code that will be called each time you use the generator object:\\n\\n  # If there is still a child of the node object on its left\\n  # AND if distance is ok, return the next child\\n  if self._leftchild and distance - max_dist &lt; self._median:\\n      yield self._leftchild\\n\\n  # If there is still a child of the node object on its right\\n  # AND if distance is ok, return the next child\\n  if self._rightchild and distance + max_dist &gt;= self._median:\\n      yield self._rightchild\\n\\n  # If the function arrives here, the generator will be considered empty\\n  # there is no more than two values: the left and the right children\\n</code></pre>\\n\\n<p>Caller:</p>\\n\\n<pre><code># Create an empty list and a list with the current object reference\\nresult, candidates = list(), [self]\\n\\n# Loop on candidates (they contain only one element at the beginning)\\nwhile candidates:\\n\\n    # Get the last candidate and remove it from the list\\n    node = candidates.pop()\\n\\n    # Get the distance between obj and the candidate\\n    distance = node._get_dist(obj)\\n\\n    # If distance is ok, then you can fill the result\\n    if distance &lt;= max_dist and distance &gt;= min_dist:\\n        result.extend(node._values)\\n\\n    # Add the children of the candidate in the candidates list\\n    # so the loop will keep running until it will have looked\\n    # at all the children of the children of the children, etc. of the candidate\\n    candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))\\n\\nreturn result\\n</code></pre>\\n\\n<p>This code contains several smart parts:</p>\\n\\n<ul>\\n<li><p>The loop iterates on a list but the list expands while the loop is being iterated :-) It\\'s a concise way to go through all these nested data even if it\\'s a bit dangerous since you can end up with an infinite loop. In this case, <code>candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))</code> exhausts all the values of the generator, but <code>while</code> keeps creating new generator objects which will produce different values from the previous ones since it\\'s not applied on the same node.</p></li>\\n<li><p>The <code>extend()</code> method is a list object method that expects an iterable and adds its values to the list.</p></li>\\n</ul>\\n\\n<p>Usually we pass a list to it:</p>\\n\\n<pre><code>&gt;&gt;&gt; a = [1, 2]\\n&gt;&gt;&gt; b = [3, 4]\\n&gt;&gt;&gt; a.extend(b)\\n&gt;&gt;&gt; print(a)\\n[1, 2, 3, 4]\\n</code></pre>\\n\\n<p>But in your code it gets a generator, which is good because:</p>\\n\\n<ol>\\n<li>You don\\'t need to read the values twice.</li>\\n<li>You may have a lot of children and you don\\'t want them all stored in memory.</li>\\n</ol>\\n\\n<p>And it works because Python does not care if the argument of a method is a list or not. Python expects iterables so it will work with strings, lists, tuples and generators! This is called duck typing and is one of the reason why Python is so cool. But this is another story, for another question...</p>\\n\\n<p>You can stop here, or read a little bit to see an advanced use of a generator:</p>\\n\\n<h2>Controlling a generator exhaustion</h2>\\n\\n<pre><code>&gt;&gt;&gt; class Bank(): # let\\'s create a bank, building ATMs\\n...    crisis = False\\n...    def create_atm(self):\\n...        while not self.crisis:\\n...            yield \"$100\"\\n&gt;&gt;&gt; hsbc = Bank() # when everything\\'s ok the ATM gives you as much as you want\\n&gt;&gt;&gt; corner_street_atm = hsbc.create_atm()\\n&gt;&gt;&gt; print(corner_street_atm.next())\\n$100\\n&gt;&gt;&gt; print(corner_street_atm.next())\\n$100\\n&gt;&gt;&gt; print([corner_street_atm.next() for cash in range(5)])\\n[\\'$100\\', \\'$100\\', \\'$100\\', \\'$100\\', \\'$100\\']\\n&gt;&gt;&gt; hsbc.crisis = True # crisis is coming, no more money!\\n&gt;&gt;&gt; print(corner_street_atm.next())\\n&lt;type \\'exceptions.StopIteration\\'&gt;\\n&gt;&gt;&gt; wall_street_atm = hsbc.create_atm() # it\\'s even true for new ATMs\\n&gt;&gt;&gt; print(wall_street_atm.next())\\n&lt;type \\'exceptions.StopIteration\\'&gt;\\n&gt;&gt;&gt; hsbc.crisis = False # trouble is, even post-crisis the ATM remains empty\\n&gt;&gt;&gt; print(corner_street_atm.next())\\n&lt;type \\'exceptions.StopIteration\\'&gt;\\n&gt;&gt;&gt; brand_new_atm = hsbc.create_atm() # build a new one to get back in business\\n&gt;&gt;&gt; for cash in brand_new_atm:\\n...    print cash\\n$100\\n$100\\n$100\\n$100\\n$100\\n$100\\n$100\\n$100\\n$100\\n...\\n</code></pre>\\n\\n<p>It can be useful for various things like controlling access to a resource.</p>\\n\\n<h2>Itertools, your best friend</h2>\\n\\n<p>The itertools module contains special functions to manipulate iterables. Ever wish to duplicate a generator?\\nChain two generators? Group values in a nested list with a one liner? <code>Map / Zip</code> without creating another list?</p>\\n\\n<p>Then just <code>import itertools</code>.</p>\\n\\n<p>An example? Let\\'s see the possible orders of arrival for a 4 horse race:</p>\\n\\n<pre><code>&gt;&gt;&gt; horses = [1, 2, 3, 4]\\n&gt;&gt;&gt; races = itertools.permutations(horses)\\n&gt;&gt;&gt; print(races)\\n&lt;itertools.permutations object at 0xb754f1dc&gt;\\n&gt;&gt;&gt; print(list(itertools.permutations(horses)))\\n[(1, 2, 3, 4),\\n (1, 2, 4, 3),\\n (1, 3, 2, 4),\\n (1, 3, 4, 2),\\n (1, 4, 2, 3),\\n (1, 4, 3, 2),\\n (2, 1, 3, 4),\\n (2, 1, 4, 3),\\n (2, 3, 1, 4),\\n (2, 3, 4, 1),\\n (2, 4, 1, 3),\\n (2, 4, 3, 1),\\n (3, 1, 2, 4),\\n (3, 1, 4, 2),\\n (3, 2, 1, 4),\\n (3, 2, 4, 1),\\n (3, 4, 1, 2),\\n (3, 4, 2, 1),\\n (4, 1, 2, 3),\\n (4, 1, 3, 2),\\n (4, 2, 1, 3),\\n (4, 2, 3, 1),\\n (4, 3, 1, 2),\\n (4, 3, 2, 1)]\\n</code></pre>\\n\\n<h2>Understanding the inner mechanisms of iteration</h2>\\n\\n<p>Iteration is a process implying iterables (implementing the <code>__iter__()</code> method) and iterators (implementing the <code>__next__()</code> method).\\nIterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.</p>\\n\\n<p>More about it in this article about <a href=\"http://effbot.org/zone/python-for-statement.htm\">how does the for loop work</a>.</p>\\n',\n",
       " 'Question:\\nWhat is a metaclass in Python?\\n\\nAnswer:\\n<h1>Classes as objects</h1>\\n\\n<p>Before understanding metaclasses, you need to master classes in Python. And Python has a very peculiar idea of what classes are, borrowed from the Smalltalk language.</p>\\n\\n<p>In most languages, classes are just pieces of code that describe how to produce an object. That\\'s kinda true in Python too:</p>\\n\\n<pre><code>&gt;&gt;&gt; class ObjectCreator(object):\\n...       pass\\n... \\n\\n&gt;&gt;&gt; my_object = ObjectCreator()\\n&gt;&gt;&gt; print(my_object)\\n&lt;__main__.ObjectCreator object at 0x8974f2c&gt;\\n</code></pre>\\n\\n<p>But classes are more than that in Python. Classes are objects too.</p>\\n\\n<p>Yes, objects. </p>\\n\\n<p>As soon as you use the keyword <code>class</code>, Python executes it and creates\\nan OBJECT. The instruction</p>\\n\\n<pre><code>&gt;&gt;&gt; class ObjectCreator(object):\\n...       pass\\n... \\n</code></pre>\\n\\n<p>creates in memory an object with the name \"ObjectCreator\". </p>\\n\\n<p><strong>This object (the class) is itself capable of creating objects (the instances), \\nand this is why it\\'s a class</strong>. </p>\\n\\n<p>But still, it\\'s an object, and therefore:</p>\\n\\n<ul>\\n<li>you can assign it to a variable</li>\\n<li>you can copy it</li>\\n<li>you can add attributes to it</li>\\n<li>you can pass it as a function parameter</li>\\n</ul>\\n\\n<p>e.g.:</p>\\n\\n<pre><code>&gt;&gt;&gt; print(ObjectCreator) # you can print a class because it\\'s an object\\n&lt;class \\'__main__.ObjectCreator\\'&gt;\\n&gt;&gt;&gt; def echo(o):\\n...       print(o)\\n... \\n&gt;&gt;&gt; echo(ObjectCreator) # you can pass a class as a parameter\\n&lt;class \\'__main__.ObjectCreator\\'&gt;\\n&gt;&gt;&gt; print(hasattr(ObjectCreator, \\'new_attribute\\'))\\nFalse\\n&gt;&gt;&gt; ObjectCreator.new_attribute = \\'foo\\' # you can add attributes to a class\\n&gt;&gt;&gt; print(hasattr(ObjectCreator, \\'new_attribute\\'))\\nTrue\\n&gt;&gt;&gt; print(ObjectCreator.new_attribute)\\nfoo\\n&gt;&gt;&gt; ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable\\n&gt;&gt;&gt; print(ObjectCreatorMirror.new_attribute)\\nfoo\\n&gt;&gt;&gt; print(ObjectCreatorMirror())\\n&lt;__main__.ObjectCreator object at 0x8997b4c&gt;\\n</code></pre>\\n\\n<h1>Creating classes dynamically</h1>\\n\\n<p>Since classes are objects, you can create them on the fly, like any object.</p>\\n\\n<p>First, you can create a class in a function using <code>class</code>:</p>\\n\\n<pre><code>&gt;&gt;&gt; def choose_class(name):\\n...     if name == \\'foo\\':\\n...         class Foo(object):\\n...             pass\\n...         return Foo # return the class, not an instance\\n...     else:\\n...         class Bar(object):\\n...             pass\\n...         return Bar\\n...     \\n&gt;&gt;&gt; MyClass = choose_class(\\'foo\\') \\n&gt;&gt;&gt; print(MyClass) # the function returns a class, not an instance\\n&lt;class \\'__main__.Foo\\'&gt;\\n&gt;&gt;&gt; print(MyClass()) # you can create an object from this class\\n&lt;__main__.Foo object at 0x89c6d4c&gt;\\n</code></pre>\\n\\n<p>But it\\'s not so dynamic, since you still have to write the whole class yourself.</p>\\n\\n<p>Since classes are objects, they must be generated by something.</p>\\n\\n<p>When you use the <code>class</code> keyword, Python creates this object automatically. But as\\nwith most things in Python, it gives you a way to do it manually.</p>\\n\\n<p>Remember the function <code>type</code>? The good old function that lets you know what \\ntype an object is:</p>\\n\\n<pre><code>&gt;&gt;&gt; print(type(1))\\n&lt;type \\'int\\'&gt;\\n&gt;&gt;&gt; print(type(\"1\"))\\n&lt;type \\'str\\'&gt;\\n&gt;&gt;&gt; print(type(ObjectCreator))\\n&lt;type \\'type\\'&gt;\\n&gt;&gt;&gt; print(type(ObjectCreator()))\\n&lt;class \\'__main__.ObjectCreator\\'&gt;\\n</code></pre>\\n\\n<p>Well, <a href=\"http://docs.python.org/2/library/functions.html#type\"><code>type</code></a> has a completely different ability, it can also create classes on the fly. <code>type</code> can take the description of a class as parameters, \\nand return a class.</p>\\n\\n<p>(I  know, it\\'s silly that the same function can have two completely different uses according to the parameters you pass to it. It\\'s an issue due to backwards \\ncompatibility in Python)</p>\\n\\n<p><code>type</code> works this way:</p>\\n\\n<pre><code>type(name of the class, \\n     tuple of the parent class (for inheritance, can be empty), \\n     dictionary containing attributes names and values)\\n</code></pre>\\n\\n<p>e.g.:</p>\\n\\n<pre><code>&gt;&gt;&gt; class MyShinyClass(object):\\n...       pass\\n</code></pre>\\n\\n<p>can be created manually this way:</p>\\n\\n<pre><code>&gt;&gt;&gt; MyShinyClass = type(\\'MyShinyClass\\', (), {}) # returns a class object\\n&gt;&gt;&gt; print(MyShinyClass)\\n&lt;class \\'__main__.MyShinyClass\\'&gt;\\n&gt;&gt;&gt; print(MyShinyClass()) # create an instance with the class\\n&lt;__main__.MyShinyClass object at 0x8997cec&gt;\\n</code></pre>\\n\\n<p>You\\'ll notice that we use \"MyShinyClass\" as the name of the class\\nand as the variable to hold the class reference. They can be different,\\nbut there is no reason to complicate things.</p>\\n\\n<p><code>type</code> accepts a dictionary to define the attributes of the class. So:</p>\\n\\n<pre><code>&gt;&gt;&gt; class Foo(object):\\n...       bar = True\\n</code></pre>\\n\\n<p>Can be translated to:</p>\\n\\n<pre><code>&gt;&gt;&gt; Foo = type(\\'Foo\\', (), {\\'bar\\':True})\\n</code></pre>\\n\\n<p>And used as a normal class:</p>\\n\\n<pre><code>&gt;&gt;&gt; print(Foo)\\n&lt;class \\'__main__.Foo\\'&gt;\\n&gt;&gt;&gt; print(Foo.bar)\\nTrue\\n&gt;&gt;&gt; f = Foo()\\n&gt;&gt;&gt; print(f)\\n&lt;__main__.Foo object at 0x8a9b84c&gt;\\n&gt;&gt;&gt; print(f.bar)\\nTrue\\n</code></pre>\\n\\n<p>And of course, you can inherit from it, so:</p>\\n\\n<pre><code>&gt;&gt;&gt;   class FooChild(Foo):\\n...         pass\\n</code></pre>\\n\\n<p>would be:</p>\\n\\n<pre><code>&gt;&gt;&gt; FooChild = type(\\'FooChild\\', (Foo,), {})\\n&gt;&gt;&gt; print(FooChild)\\n&lt;class \\'__main__.FooChild\\'&gt;\\n&gt;&gt;&gt; print(FooChild.bar) # bar is inherited from Foo\\nTrue\\n</code></pre>\\n\\n<p>Eventually you\\'ll want to add methods to your class. Just define a function\\nwith the proper signature and assign it as an attribute.</p>\\n\\n<pre><code>&gt;&gt;&gt; def echo_bar(self):\\n...       print(self.bar)\\n... \\n&gt;&gt;&gt; FooChild = type(\\'FooChild\\', (Foo,), {\\'echo_bar\\': echo_bar})\\n&gt;&gt;&gt; hasattr(Foo, \\'echo_bar\\')\\nFalse\\n&gt;&gt;&gt; hasattr(FooChild, \\'echo_bar\\')\\nTrue\\n&gt;&gt;&gt; my_foo = FooChild()\\n&gt;&gt;&gt; my_foo.echo_bar()\\nTrue\\n</code></pre>\\n\\n<p>And you can add even more methods after you dynamically create the class, just like adding methods to a normally created class object.</p>\\n\\n<pre><code>&gt;&gt;&gt; def echo_bar_more(self):\\n...       print(\\'yet another method\\')\\n... \\n&gt;&gt;&gt; FooChild.echo_bar_more = echo_bar_more\\n&gt;&gt;&gt; hasattr(FooChild, \\'echo_bar_more\\')\\nTrue\\n</code></pre>\\n\\n<p>You see where we are going: in Python, classes are objects, and you can create a class on the fly, dynamically.</p>\\n\\n<p>This is what Python does when you use the keyword <code>class</code>, and it does so by using a metaclass.</p>\\n\\n<h1>What are metaclasses (finally)</h1>\\n\\n<p>Metaclasses are the \\'stuff\\' that creates classes.</p>\\n\\n<p>You define classes in order to create objects, right?</p>\\n\\n<p>But we learned that Python classes are objects.</p>\\n\\n<p>Well, metaclasses are what create these objects. They are the classes\\' classes,\\nyou can picture them this way:</p>\\n\\n<pre><code>MyClass = MetaClass()\\nMyObject = MyClass()\\n</code></pre>\\n\\n<p>You\\'ve seen that <code>type</code> lets you do something like this:</p>\\n\\n<pre><code>MyClass = type(\\'MyClass\\', (), {})\\n</code></pre>\\n\\n<p>It\\'s because the function <code>type</code> is in fact a metaclass. <code>type</code> is the \\nmetaclass Python uses to create all classes behind the scenes.</p>\\n\\n<p>Now you wonder why the heck is it written in lowercase, and not <code>Type</code>?</p>\\n\\n<p>Well, I guess it\\'s a matter of consistency with <code>str</code>, the class that creates\\nstrings objects, and <code>int</code> the class that creates integer objects. <code>type</code> is\\njust the class that creates class objects.</p>\\n\\n<p>You see that by checking the <code>__class__</code> attribute. </p>\\n\\n<p>Everything, and I mean everything, is an object in Python. That includes ints, \\nstrings, functions and classes. All of them are objects. And all of them have\\nbeen created from a class:</p>\\n\\n<pre><code>&gt;&gt;&gt; age = 35\\n&gt;&gt;&gt; age.__class__\\n&lt;type \\'int\\'&gt;\\n&gt;&gt;&gt; name = \\'bob\\'\\n&gt;&gt;&gt; name.__class__\\n&lt;type \\'str\\'&gt;\\n&gt;&gt;&gt; def foo(): pass\\n&gt;&gt;&gt; foo.__class__\\n&lt;type \\'function\\'&gt;\\n&gt;&gt;&gt; class Bar(object): pass\\n&gt;&gt;&gt; b = Bar()\\n&gt;&gt;&gt; b.__class__\\n&lt;class \\'__main__.Bar\\'&gt;\\n</code></pre>\\n\\n<p>Now, what is the <code>__class__</code> of any <code>__class__</code> ?</p>\\n\\n<pre><code>&gt;&gt;&gt; age.__class__.__class__\\n&lt;type \\'type\\'&gt;\\n&gt;&gt;&gt; name.__class__.__class__\\n&lt;type \\'type\\'&gt;\\n&gt;&gt;&gt; foo.__class__.__class__\\n&lt;type \\'type\\'&gt;\\n&gt;&gt;&gt; b.__class__.__class__\\n&lt;type \\'type\\'&gt;\\n</code></pre>\\n\\n<p>So, a metaclass is just the stuff that creates class objects.</p>\\n\\n<p>You can call it a \\'class factory\\' if you wish.</p>\\n\\n<p><code>type</code> is the built-in metaclass Python uses, but of course, you can create your\\nown metaclass.</p>\\n\\n<h1>The <a href=\"http://docs.python.org/2/reference/datamodel.html?highlight=__metaclass__#__metaclass__\"><code>__metaclass__</code></a> attribute</h1>\\n\\n<p>You can add a <code>__metaclass__</code> attribute when you write a class:</p>\\n\\n<pre><code>class Foo(object):\\n  __metaclass__ = something...\\n  [...]\\n</code></pre>\\n\\n<p>If you do so, Python will use the metaclass to create the class <code>Foo</code>.</p>\\n\\n<p>Careful, it\\'s tricky.</p>\\n\\n<p>You write <code>class Foo(object)</code> first, but the class object <code>Foo</code> is not created\\nin memory yet.</p>\\n\\n<p>Python will look for <code>__metaclass__</code> in the class definition. If it finds it,\\nit will use it to create the object class <code>Foo</code>. If it doesn\\'t, it will use\\n<code>type</code> to create the class.</p>\\n\\n<p>Read that several times.</p>\\n\\n<p>When you do:</p>\\n\\n<pre><code>class Foo(Bar):\\n  pass\\n</code></pre>\\n\\n<p>Python does the following:</p>\\n\\n<p>Is there a <code>__metaclass__</code> attribute in <code>Foo</code>?</p>\\n\\n<p>If yes, create in memory a class object (I said a class object, stay with me here), with the name <code>Foo</code> by using what is in <code>__metaclass__</code>.</p>\\n\\n<p>If Python can\\'t find <code>__metaclass__</code>, it will look for a <code>__metaclass__</code> at the MODULE level, and try to do the same (but only for classes that don\\'t inherit anything, basically old-style classes). </p>\\n\\n<p>Then if it can\\'t find any <code>__metaclass__</code> at all, it will use the <code>Bar</code>\\'s (the first parent) own metaclass (which might be the default <code>type</code>) to create the class object.</p>\\n\\n<p>Be careful here that the <code>__metaclass__</code> attribute will not be inherited, the metaclass of the parent (<code>Bar.__class__</code>) will be. If <code>Bar</code> used a <code>__metaclass__</code> attribute that created <code>Bar</code> with <code>type()</code> (and not <code>type.__new__()</code>), the subclasses will not inherit that behavior.</p>\\n\\n<p>Now the big question is, what can you put in <code>__metaclass__</code> ?</p>\\n\\n<p>The answer is: something that can create a class.</p>\\n\\n<p>And what can create a class? <code>type</code>, or anything that subclasses or uses it.</p>\\n\\n<h1>Custom metaclasses</h1>\\n\\n<p>The main purpose of a metaclass is to change the class automatically,\\nwhen it\\'s created.</p>\\n\\n<p>You usually do this for APIs, where you want to create classes matching the\\ncurrent context.</p>\\n\\n<p>Imagine a stupid example, where you decide that all classes in your module\\nshould have their attributes written in uppercase. There are several ways to \\ndo this, but one way is to set <code>__metaclass__</code> at the module level.</p>\\n\\n<p>This way, all classes of this module will be created using this metaclass, \\nand we just have to tell the metaclass to turn all attributes to uppercase.</p>\\n\\n<p>Luckily, <code>__metaclass__</code> can actually be any callable, it doesn\\'t need to be a\\nformal class (I know, something with \\'class\\' in its name doesn\\'t need to be \\na class, go figure... but it\\'s helpful).</p>\\n\\n<p>So we will start with a simple example, by using a function.</p>\\n\\n<pre><code># the metaclass will automatically get passed the same argument\\n# that you usually pass to `type`\\ndef upper_attr(future_class_name, future_class_parents, future_class_attr):\\n  \"\"\"\\n    Return a class object, with the list of its attribute turned \\n    into uppercase.\\n  \"\"\"\\n\\n  # pick up any attribute that doesn\\'t start with \\'__\\' and uppercase it\\n  uppercase_attr = {}\\n  for name, val in future_class_attr.items():\\n      if not name.startswith(\\'__\\'):\\n          uppercase_attr[name.upper()] = val\\n      else:\\n          uppercase_attr[name] = val\\n\\n  # let `type` do the class creation\\n  return type(future_class_name, future_class_parents, uppercase_attr)\\n\\n__metaclass__ = upper_attr # this will affect all classes in the module\\n\\nclass Foo(): # global __metaclass__ won\\'t work with \"object\" though\\n  # but we can define __metaclass__ here instead to affect only this class\\n  # and this will work with \"object\" children\\n  bar = \\'bip\\'\\n\\nprint(hasattr(Foo, \\'bar\\'))\\n# Out: False\\nprint(hasattr(Foo, \\'BAR\\'))\\n# Out: True\\n\\nf = Foo()\\nprint(f.BAR)\\n# Out: \\'bip\\'\\n</code></pre>\\n\\n<p>Now, let\\'s do exactly the same, but using a real class for a metaclass:</p>\\n\\n<pre><code># remember that `type` is actually a class like `str` and `int`\\n# so you can inherit from it\\nclass UpperAttrMetaclass(type): \\n    # __new__ is the method called before __init__\\n    # it\\'s the method that creates the object and returns it\\n    # while __init__ just initializes the object passed as parameter\\n    # you rarely use __new__, except when you want to control how the object\\n    # is created.\\n    # here the created object is the class, and we want to customize it\\n    # so we override __new__\\n    # you can do some stuff in __init__ too if you wish\\n    # some advanced use involves overriding __call__ as well, but we won\\'t\\n    # see this\\n    def __new__(upperattr_metaclass, future_class_name, \\n                future_class_parents, future_class_attr):\\n\\n        uppercase_attr = {}\\n        for name, val in future_class_attr.items():\\n            if not name.startswith(\\'__\\'):\\n                uppercase_attr[name.upper()] = val\\n            else:\\n                uppercase_attr[name] = val\\n\\n        return type(future_class_name, future_class_parents, uppercase_attr)\\n</code></pre>\\n\\n<p>But this is not really OOP. We call <code>type</code> directly and we don\\'t override\\nor call the parent <code>__new__</code>. Let\\'s do it:</p>\\n\\n<pre><code>class UpperAttrMetaclass(type): \\n\\n    def __new__(upperattr_metaclass, future_class_name, \\n                future_class_parents, future_class_attr):\\n\\n        uppercase_attr = {}\\n        for name, val in future_class_attr.items():\\n            if not name.startswith(\\'__\\'):\\n                uppercase_attr[name.upper()] = val\\n            else:\\n                uppercase_attr[name] = val\\n\\n        # reuse the type.__new__ method\\n        # this is basic OOP, nothing magic in there\\n        return type.__new__(upperattr_metaclass, future_class_name, \\n                            future_class_parents, uppercase_attr)\\n</code></pre>\\n\\n<p>You may have noticed the extra argument <code>upperattr_metaclass</code>. There is\\nnothing special about it: <code>__new__</code> always receives the class it\\'s defined in, as first parameter. Just like you have <code>self</code> for ordinary methods which receive the instance as first parameter, or the defining class for class methods.</p>\\n\\n<p>Of course, the names I used here are long for the sake of clarity, but like\\nfor <code>self</code>, all the arguments have conventional names. So a real production\\nmetaclass would look like this:</p>\\n\\n<pre><code>class UpperAttrMetaclass(type): \\n\\n    def __new__(cls, clsname, bases, dct):\\n\\n        uppercase_attr = {}\\n        for name, val in dct.items():\\n            if not name.startswith(\\'__\\'):\\n                uppercase_attr[name.upper()] = val\\n            else:\\n                uppercase_attr[name] = val\\n\\n        return type.__new__(cls, clsname, bases, uppercase_attr)\\n</code></pre>\\n\\n<p>We can make it even cleaner by using <code>super</code>, which will ease inheritance (because yes, you can have metaclasses, inheriting from metaclasses, inheriting from type):</p>\\n\\n<pre><code>class UpperAttrMetaclass(type): \\n\\n    def __new__(cls, clsname, bases, dct):\\n\\n        uppercase_attr = {}\\n        for name, val in dct.items():\\n            if not name.startswith(\\'__\\'):\\n                uppercase_attr[name.upper()] = val\\n            else:\\n                uppercase_attr[name] = val\\n\\n        return super(UpperAttrMetaclass, cls).__new__(cls, clsname, bases, uppercase_attr)\\n</code></pre>\\n\\n<p>That\\'s it. There is really nothing more about metaclasses.</p>\\n\\n<p>The reason behind the complexity of the code using metaclasses is not because\\nof metaclasses, it\\'s because you usually use metaclasses to do twisted stuff\\nrelying on introspection, manipulating inheritance, vars such as <code>__dict__</code>, etc.</p>\\n\\n<p>Indeed, metaclasses are especially useful to do black magic, and therefore\\ncomplicated stuff. But by themselves, they are simple:</p>\\n\\n<ul>\\n<li>intercept a class creation</li>\\n<li>modify the class</li>\\n<li>return the modified class</li>\\n</ul>\\n\\n<h1>Why would you use metaclasses classes instead of functions?</h1>\\n\\n<p>Since <code>__metaclass__</code> can accept any callable, why would you use a class\\nsince it\\'s obviously more complicated?</p>\\n\\n<p>There are several reasons to do so:</p>\\n\\n<ul>\\n<li>The intention is clear. When you read <code>UpperAttrMetaclass(type)</code>, you know\\nwhat\\'s going to follow</li>\\n<li>You can use OOP. Metaclass can inherit from metaclass, override parent methods. Metaclasses can even use metaclasses.</li>\\n<li>You can structure your code better. You never use metaclasses for something as\\ntrivial as the above example. It\\'s usually for something complicated. Having the\\nability to make several methods and group them in one class is very useful\\nto make the code easier to read.</li>\\n<li>You can hook on <code>__new__</code>, <code>__init__</code> and <code>__call__</code>. Which will allow\\nyou to do different stuff. Even if usually you can do it all in <code>__new__</code>, \\nsome people are just more comfortable using <code>__init__</code>.</li>\\n<li>These are called metaclasses, damn it! It must mean something!</li>\\n</ul>\\n\\n<h1>Why would you use metaclasses?</h1>\\n\\n<p>Now the big question. Why would you use some obscure error prone feature?</p>\\n\\n<p>Well, usually you don\\'t:</p>\\n\\n<blockquote>\\n  <p>Metaclasses are deeper magic that\\n  99% of users should never worry about.\\n  If you wonder whether you need them,\\n  you don\\'t (the people who actually\\n  need them know with certainty that\\n  they need them, and don\\'t need an\\n  explanation about why). </p>\\n</blockquote>\\n\\n<p><em>Python Guru Tim Peters</em></p>\\n\\n<p>The main use case for a metaclass is creating an API. A typical example of this is the Django ORM.</p>\\n\\n<p>It allows you to define something like this:</p>\\n\\n<pre><code>class Person(models.Model):\\n  name = models.CharField(max_length=30)\\n  age = models.IntegerField()\\n</code></pre>\\n\\n<p>But if you do this:</p>\\n\\n<pre><code>guy = Person(name=\\'bob\\', age=\\'35\\')\\nprint(guy.age)\\n</code></pre>\\n\\n<p>It won\\'t return an <code>IntegerField</code> object. It will return an <code>int</code>, and can even take it directly from the database.</p>\\n\\n<p>This is possible because <code>models.Model</code> defines <code>__metaclass__</code> and \\nit uses some magic that will turn the <code>Person</code> you just defined with simple statements\\ninto a complex hook to a database field. </p>\\n\\n<p>Django makes something complex look simple by exposing a simple API\\nand using metaclasses, recreating code from this API to do the real job\\nbehind the scenes.</p>\\n\\n<h1>The last word</h1>\\n\\n<p>First, you know that classes are objects that can create instances.</p>\\n\\n<p>Well in fact, classes are themselves instances. Of metaclasses.</p>\\n\\n<pre><code>&gt;&gt;&gt; class Foo(object): pass\\n&gt;&gt;&gt; id(Foo)\\n142630324\\n</code></pre>\\n\\n<p>Everything is an object in Python, and they are all either instances of classes\\nor instances of metaclasses.</p>\\n\\n<p>Except for <code>type</code>.</p>\\n\\n<p><code>type</code> is actually its own metaclass. This is not something you could\\nreproduce in pure Python, and is done by cheating a little bit at the implementation\\nlevel.</p>\\n\\n<p>Secondly, metaclasses are complicated. You may not want to use them for \\nvery simple class alterations. You can change classes by using two different techniques:</p>\\n\\n<ul>\\n<li><a href=\"http://en.wikipedia.org/wiki/Monkey_patch\">monkey patching</a></li>\\n<li>class decorators</li>\\n</ul>\\n\\n<p>99% of the time you need class alteration, you are better off using these.</p>\\n\\n<p>But 99% of the time, you don\\'t need class alteration at all.</p>\\n',\n",
       " 'Question:\\nHow do I check whether a file exists using Python?\\n\\nAnswer:\\n<p>You can also use <a href=\"https://docs.python.org/2/library/os.path.html#os.path.isfile\"><code>os.path.isfile</code></a></p>\\n\\n<blockquote>\\n  <p>Return <code>True</code> if path is an existing regular file. This follows symbolic links, so both <a href=\"https://docs.python.org/2/library/os.path.html#os.path.islink\">islink()</a> and <a href=\"https://docs.python.org/2/library/os.path.html#os.path.isfile\">isfile()</a> can be true for the same path.</p>\\n</blockquote>\\n\\n<pre><code>import os.path\\nos.path.isfile(fname) \\n</code></pre>\\n\\n<p>if you need to be sure it\\'s a file.</p>\\n\\n<p>Starting with Python 3.4, the <a href=\"https://docs.python.org/3/library/pathlib.html#pathlib.Path.is_file\"><code>pathlib</code> module</a> offers an object-oriented approach:</p>\\n\\n<pre><code>from pathlib import Path\\n\\nmy_file = Path(\"/path/to/file\")\\nif my_file.is_file():\\n    # file exists\\n</code></pre>\\n',\n",
       " 'Question:\\nDoes Python have a ternary conditional operator?\\n\\nAnswer:\\n<p>Yes, it was <a href=\"https://mail.python.org/pipermail/python-dev/2005-September/056846.html\">added</a> in version 2.5.<br>\\nThe syntax is:</p>\\n\\n<pre><code>a if condition else b\\n</code></pre>\\n\\n<p>First <code>condition</code> is evaluated, then either <code>a</code> or <code>b</code> is returned based on the <a href=\"https://en.wikipedia.org/wiki/Boolean_data_type\">Boolean</a> value of <code>condition</code><br>\\nIf <code>condition</code> evaluates to <em>True</em> <code>a</code> is returned, else <code>b</code> is returned. </p>\\n\\n<p>For example:</p>\\n\\n<pre><code>&gt;&gt;&gt; \\'true\\' if True else \\'false\\'\\n\\'true\\'\\n&gt;&gt;&gt; \\'true\\' if False else \\'false\\'\\n\\'false\\'\\n</code></pre>\\n\\n<p>Keep in mind that it\\'s frowned upon by some Pythonistas for several reasons:</p>\\n\\n<ul>\\n<li>The order of the arguments is different from many other languages (such as C, Ruby, Java, etc.), which may lead to bugs when people unfamiliar with Python\\'s \"surprising\" behaviour use it (they may reverse the order).</li>\\n<li>Some find it \"unwieldy\", since it goes contrary to the normal flow of thought (thinking of the condition first and then the effects).</li>\\n<li>Stylistic reasons.</li>\\n</ul>\\n\\n<p>If you\\'re having trouble remembering the order, then remember that if you read it out loud, you (almost) say what you mean. For example, <code>x = 4 if b &gt; 8 else 9</code> is read aloud as <code>x will be 4 if b is greater than 8 otherwise 9</code>.</p>\\n\\n<p>Official documentation:</p>\\n\\n<ul>\\n<li><a href=\"https://docs.python.org/3/reference/expressions.html#conditional-expressions\">Conditional expressions</a></li>\\n<li><a href=\"https://docs.python.org/3.3/faq/programming.html#is-there-an-equivalent-of-c-s-ternary-operator\">Is there an equivalent of Câ\\x80\\x99s â\\x80\\x9d?:â\\x80\\x9d ternary operator?</a></li>\\n</ul>\\n',\n",
       " 'Question:\\nCalling an external command in Python\\n\\nAnswer:\\n<p>Look at the <a href=\"https://docs.python.org/2/library/subprocess.html\">subprocess module</a> in the stdlib:</p>\\n\\n<pre><code>from subprocess import call\\ncall([\"ls\", \"-l\"])\\n</code></pre>\\n\\n<p>The advantage of subprocess vs system is that it is more flexible (you can get the stdout, stderr, the \"real\" status code, better error handling, etc...). I think os.system is deprecated, too, or will be:</p>\\n\\n<p><a href=\"https://docs.python.org/2/library/subprocess.html#replacing-older-functions-with-the-subprocess-module\">https://docs.python.org/2/library/subprocess.html#replacing-older-functions-with-the-subprocess-module</a></p>\\n\\n<p>For quick/dirty/one time scripts, <code>os.system</code> is enough, though.</p>\\n']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d9c83c6",
   "metadata": {
    "papermill": {
     "duration": 0.013284,
     "end_time": "2024-03-03T20:20:28.978877",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.965593",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Fine-tuning Gemma\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eea09dea",
   "metadata": {
    "papermill": {
     "duration": 0.012263,
     "end_time": "2024-03-03T20:20:29.003550",
     "exception": false,
     "start_time": "2024-03-03T20:20:28.991287",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Gemma Language Model Backbone Configuration:\n",
    "\n",
    "- Enable Long-Range Attention (LoRA) mechanism with a rank of 4 for the Gemma Language Model backbone.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "5ce115c8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:29.030360Z",
     "iopub.status.busy": "2024-03-03T20:20:29.030012Z",
     "iopub.status.idle": "2024-03-03T20:20:29.155744Z",
     "shell.execute_reply": "2024-03-03T20:20:29.154996Z"
    },
    "papermill": {
     "duration": 0.141139,
     "end_time": "2024-03-03T20:20:29.157854",
     "exception": false,
     "start_time": "2024-03-03T20:20:29.016715",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "gemma_lm.backbone.enable_lora(rank=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "7e70aa99",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:29.184676Z",
     "iopub.status.busy": "2024-03-03T20:20:29.183990Z",
     "iopub.status.idle": "2024-03-03T20:20:29.188427Z",
     "shell.execute_reply": "2024-03-03T20:20:29.187647Z"
    },
    "papermill": {
     "duration": 0.019631,
     "end_time": "2024-03-03T20:20:29.190266",
     "exception": false,
     "start_time": "2024-03-03T20:20:29.170635",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_prompt(query:str)->str:\n",
    "    template = \"Python Query:\\n{python_query}\\n\\nResponse:\\n{response}\"\n",
    "    prompt = template.format(\n",
    "        python_query=query,\n",
    "        response=\"\",\n",
    "    )\n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d2625df",
   "metadata": {
    "papermill": {
     "duration": 0.01233,
     "end_time": "2024-03-03T20:20:29.215310",
     "exception": false,
     "start_time": "2024-03-03T20:20:29.202980",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Gemma Language Model Configuration and Training:\n",
    "\n",
    "- **Limiting Input Sequence Length**: Set the input sequence length to 128 to control memory usage.\n",
    "\n",
    "- **Optimizer Configuration**:\n",
    "  - Use AdamW optimizer, a common choice for transformer models.\n",
    "  - Set learning rate to 5e-6 and weight decay to 0.01.\n",
    "  - Exclude layernorm and bias terms from weight decay.\n",
    "\n",
    "- **Model Compilation**:\n",
    "  - Compile the Gemma Language Model using SparseCategoricalCrossentropy loss function (from logits).\n",
    "  - Use the configured optimizer.\n",
    "  - Track model performance using SparseCategoricalAccuracy as the weighted metric.\n",
    "\n",
    "- **Model Training**:\n",
    "  - Fit the model using the formatted data with one epoch and a batch size of 1.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6789b505",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:20:29.241291Z",
     "iopub.status.busy": "2024-03-03T20:20:29.240989Z",
     "iopub.status.idle": "2024-03-03T20:38:14.218007Z",
     "shell.execute_reply": "2024-03-03T20:38:14.217117Z"
    },
    "papermill": {
     "duration": 1064.992068,
     "end_time": "2024-03-03T20:38:14.219859",
     "exception": false,
     "start_time": "2024-03-03T20:20:29.227791",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0000 00:00:1709497271.254890      81 graph_launch.cc:671] Fallback to op-by-op mode because memset node breaks graph update\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m5000/5000\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1064s\u001b[0m 205ms/step - loss: 1.4846 - sparse_categorical_accuracy: 0.6553\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.src.callbacks.history.History at 0x7a2880131fc0>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Limit the input sequence length to 128 (to control memory usage).\n",
    "gemma_lm.preprocessor.sequence_length = 128\n",
    "# Use AdamW (a common optimizer for transformer models).\n",
    "optimizer = keras.optimizers.AdamW(\n",
    "    learning_rate=5e-5,\n",
    "    weight_decay=0.01,\n",
    ")\n",
    "# Exclude layernorm and bias terms from decay.\n",
    "optimizer.exclude_from_weight_decay(var_names=[\"bias\", \"scale\"])\n",
    "\n",
    "gemma_lm.compile(\n",
    "    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "    optimizer=optimizer,\n",
    "    weighted_metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
    ")\n",
    "gemma_lm.fit(train, epochs=1, batch_size=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57d3ac27",
   "metadata": {
    "papermill": {
     "duration": 0.44075,
     "end_time": "2024-03-03T20:38:15.085455",
     "exception": false,
     "start_time": "2024-03-03T20:38:14.644705",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Finetuned Model Prediction "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f7053093",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:38:15.979877Z",
     "iopub.status.busy": "2024-03-03T20:38:15.979035Z",
     "iopub.status.idle": "2024-03-03T20:38:37.975342Z",
     "shell.execute_reply": "2024-03-03T20:38:37.974202Z"
    },
    "papermill": {
     "duration": 22.416917,
     "end_time": "2024-03-03T20:38:37.977537",
     "exception": false,
     "start_time": "2024-03-03T20:38:15.560620",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0000 00:00:1709498317.688331      26 graph_launch.cc:671] Fallback to op-by-op mode because memset node breaks graph update\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python Query:\n",
      "I'm facing issue with smtblib can you help me I couldnt authorize my email account\n",
      "\n",
      "Response:\n",
      "\n"
     ]
    }
   ],
   "source": [
    "prompt = get_prompt(\"I'm facing issue with smtblib can you help me I couldnt authorize my email account\")\n",
    "print(gemma_lm.generate(prompt, max_length=512))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78989e53",
   "metadata": {
    "papermill": {
     "duration": 0.415136,
     "end_time": "2024-03-03T20:38:38.864868",
     "exception": false,
     "start_time": "2024-03-03T20:38:38.449732",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Save Fine-Tuned Gemma Language Model:\n",
    "\n",
    "- Save the fine-tuned Gemma Language Model to a file named \"version_finetuned.keras\" inorder to use it directly from notebook.\n",
    "- If model is not saved then you will have to again finetune the model i.e. run all code blocks after running the session again "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "1b92513d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-03-03T20:38:39.691593Z",
     "iopub.status.busy": "2024-03-03T20:38:39.691238Z",
     "iopub.status.idle": "2024-03-03T20:39:38.821748Z",
     "shell.execute_reply": "2024-03-03T20:39:38.820701Z"
    },
    "papermill": {
     "duration": 59.54878,
     "end_time": "2024-03-03T20:39:38.824227",
     "exception": false,
     "start_time": "2024-03-03T20:38:39.275447",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "gemma_lm.save(\"version_finetuned.keras\")"
   ]
  }
 ],
 "metadata": {
  "kaggle": {
   "accelerator": "gpu",
   "dataSources": [
    {
     "databundleVersionId": 7669720,
     "sourceId": 64148,
     "sourceType": "competition"
    },
    {
     "datasetId": 262,
     "sourceId": 726715,
     "sourceType": "datasetVersion"
    },
    {
     "isSourceIdPinned": true,
     "modelInstanceId": 5171,
     "sourceId": 10260,
     "sourceType": "modelInstanceVersion"
    },
    {
     "isSourceIdPinned": true,
     "modelInstanceId": 5383,
     "sourceId": 11217,
     "sourceType": "modelInstanceVersion"
    },
    {
     "modelInstanceId": 5171,
     "sourceId": 11371,
     "sourceType": "modelInstanceVersion"
    },
    {
     "isSourceIdPinned": true,
     "modelInstanceId": 8318,
     "sourceId": 11382,
     "sourceType": "modelInstanceVersion"
    }
   ],
   "dockerImageVersionId": 30646,
   "isGpuEnabled": true,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 1336.756268,
   "end_time": "2024-03-03T20:39:45.616119",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2024-03-03T20:17:28.859851",
   "version": "2.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}