Task2-Information_Retrieval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "clear-contest",
   "metadata": {},
   "source": [
    "# Task 2: Information Retrieval"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "technological-thong",
   "metadata": {},
   "source": [
    "## Document Indexing\n",
    "\n",
    "Document indexing is the method of using specific attributes of a document to create an index. This index is used to be able to retrieve the information from the document. In this case we will be the TF.IDF value which will be stored using a term by document matrix.\n",
    "\n",
    "A term-document matrix is a mathematical matrix that describes the frequency of terms that occur in a collection of documents. This is a matrix where. Each row represents one document. Each column represents one term (word)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "armed-amendment",
   "metadata": {},
   "source": [
    "Importing all necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "athletic-model",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Libraries for Pre-Processing\n",
    "import os\n",
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem import PorterStemmer \n",
    "from bs4 import BeautifulSoup as bs\n",
    "\n",
    "#Libraries for TF-IDF and Cosine Similarity\n",
    "import math\n",
    "import numpy as np\n",
    "from collections import Counter "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "heard-volleyball",
   "metadata": {},
   "source": [
    "### Pre-Processing\n",
    "\n",
    "Pre-Processing is the process of preparing data so that it could be further analysed. This includes tokenization, case folding, stop-word removal and stemming. This makes the data much more manageable and so easier to analyse."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unlimited-edmonton",
   "metadata": {},
   "source": [
    "Reading from File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "covered-hayes",
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_file(fileName):\n",
    "\n",
    "    content = []\n",
    "    with open(fileName, 'r', encoding=\"utf8\") as file:\n",
    "        # Read each line in the file, readlines() returns a list of lines\n",
    "        content = file.readlines()\n",
    "\n",
    "        # Combine the lines in the list into a string\n",
    "        content = \"\".join(content)\n",
    "        bs_content = bs(content, \"xml\")\n",
    "\n",
    "    #returns \"bs4.element.Tag\" object\n",
    "    result = bs_content.find(\"raw\")\n",
    "    #print(result.prettify())\n",
    "\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brave-shield",
   "metadata": {},
   "source": [
    "Changing from a bs4.element.Tag object to a String."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "literary-farming",
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_Text(data):\n",
    "\n",
    "    text = data.get_text()\n",
    "\n",
    "    return text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cheap-cosmetic",
   "metadata": {},
   "source": [
    "Tokenizing the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "associate-photography",
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenize(data):\n",
    "    \n",
    "    tokens = nltk.word_tokenize(data)\n",
    "\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "chemical-setup",
   "metadata": {},
   "source": [
    "Changing all the words to lowercase"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "similar-cleaners",
   "metadata": {},
   "outputs": [],
   "source": [
    "def change_case(data):\n",
    "\n",
    "    for i in range(len(data)):\n",
    "        data[i] = data[i].casefold()\n",
    "\n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "after-happiness",
   "metadata": {},
   "source": [
    "Removing all 'extra' words, such as “the”, “is” and “and”. These do not give us any meaning and so can be ignored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "indian-forest",
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_stop_words(data):\n",
    "    \n",
    "    stop_words = set(stopwords.words('english'))\n",
    "\n",
    "    filtered_list = []\n",
    "\n",
    "    for word in data: \n",
    "        if word not in stop_words:\n",
    "            filtered_list.append(word)\n",
    "\n",
    "    return filtered_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suitable-cancer",
   "metadata": {},
   "source": [
    "Stemming reduces the word to its essence, rather than grammatical correctnes. For example \"waiting\" and \"waits\" would both be reduced to \"wait\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "whole-surgery",
   "metadata": {},
   "outputs": [],
   "source": [
    "def stemming(data):\n",
    "    \n",
    "    ps = PorterStemmer() \n",
    "\n",
    "    for i in range(len(data)):\n",
    "        data[i] = ps.stem(data[i])\n",
    "\n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "south-violence",
   "metadata": {},
   "source": [
    "Removing Symbols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "chubby-investment",
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_symbols(data):\n",
    "    \n",
    "    temp = []\n",
    "\n",
    "    symbols = \"“”‘’!\\\"#$€%&()*'+-,./:;<=>?@[\\]^_`{|}~\\n\"\n",
    "    for t in data:\n",
    "        if t not in symbols:\n",
    "            temp.append(t)\n",
    "            \n",
    "    return temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "legal-prompt",
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocessing(filename):\n",
    "    data = read_file(filename)\n",
    "    data = convert_Text(data)\n",
    "    data = tokenize(data)\n",
    "    data = change_case(data)\n",
    "    data = remove_stop_words(data)\n",
    "    data = stemming(data)\n",
    "    data = remove_symbols(data)\n",
    "    \n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "selected-restoration",
   "metadata": {},
   "source": [
    "### Printing and Data managment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rural-reserve",
   "metadata": {},
   "source": [
    "Returns the query selected as text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "molecular-quality",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_query(filename):\n",
    "    data = read_file(filename)\n",
    "    data = convert_Text(data)\n",
    "    \n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "level-morrison",
   "metadata": {},
   "source": [
    "Given the id of a document, meaning the index it is located in an array, returns the text found in that particular document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "contrary-banking",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_doc(id):\n",
    "    text = read_file(file_dataset[id][0])\n",
    "    print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dominant-queue",
   "metadata": {},
   "source": [
    "Given a dictionary, returns a matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "altered-deadline",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_Matrix(dictionary):\n",
    "\n",
    "    data = list(dictionary.items())\n",
    "    matrix = np.array(data)\n",
    "    \n",
    "    return matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "knowing-yemen",
   "metadata": {},
   "source": [
    "Prints out a matrix in an organised way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "informal-cheese",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_Matrix(matrix):\n",
    "    print('\\n'.join([''.join(['{:15}'.format(item) for item in row]) \n",
    "      for row in matrix]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "thermal-sussex",
   "metadata": {},
   "source": [
    "### Calculating for TF-IDF\n",
    "\n",
    "Term Frequency-Inverse Document Frequency, or in short, TF-IDF is a technique to quantify a word in documents, we compute a weight to each word which signifies the importance of the word in the document and the dataset. This weight is then stored in a term by document matrix. The image below goes over the basic formulas of how the TF-IDF is worked out.\n",
    "\n",
    "![TFID Formula Explenation](https://miro.medium.com/max/538/1*dug-uXDMOD6H5JMnYNpgfQ.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "speaking-sullivan",
   "metadata": {},
   "source": [
    "Calculating the Document Frequency. Returns a Dictionary of the number of documents in which each word can be found"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "virtual-minimum",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_DF(dataset):\n",
    "    \n",
    "    dfDict = {}\n",
    "\n",
    "    for i in range(len(dataset)):\n",
    "        tokens = dataset[i]\n",
    "        for w in tokens:\n",
    "            try:\n",
    "                dfDict[w].add(i)\n",
    "            except:\n",
    "                dfDict[w] = {i}\n",
    "\n",
    "    for i in dfDict:\n",
    "        dfDict[i] = len(dfDict[i])\n",
    "\n",
    "    return dfDict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "peaceful-definition",
   "metadata": {},
   "source": [
    "Calculate the Term Frequency. Meaning the the number of times a word appears in a document divded by the total number of words in the document. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "later-compilation",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_TF(dfDict):\n",
    "    tfDict = {}\n",
    "    doc_words = get_doc_words(dfDict)\n",
    "    doc_words_count = len(doc_words)\n",
    "    for word, count in dfDict.items():\n",
    "        tfDict[word] = count / float(doc_words_count)\n",
    "    return tfDict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "regional-geology",
   "metadata": {},
   "source": [
    "Calculate the Inverse Data Frequency. Meaning the log of the number of documents divided by the number of documents that contain a specific word.\n",
    "Although this function calculates the IDF, it was easier to not use this function when working out the TF-IDF."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "literary-violence",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_IDF(dataset):\n",
    "    \n",
    "    #total documents\n",
    "    N = len(dataset)\n",
    "    \n",
    "    dfDict = calculate_DF(dataset)\n",
    "    \n",
    "    idfDict = {}\n",
    "    \n",
    "    for word, val in dfDict.items():\n",
    "        idfDict[word] = math.log(N / float(val))\n",
    "\n",
    "    return idfDict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "normal-helicopter",
   "metadata": {},
   "source": [
    "Returns a list of all the unique words in a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "increased-syria",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_doc_words(dfDict):\n",
    "    doc_words = [x for x in dfDict]\n",
    "    return doc_words"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cosmetic-hearts",
   "metadata": {},
   "source": [
    "Return a list of all the unique words from all of the files in the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "continuing-disco",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_unique_words(dataset):\n",
    "    \n",
    "    unique_terms = []\n",
    "    \n",
    "    for f in dataset:\n",
    "        words = get_doc_words(f)\n",
    "        unique_terms.extend(words)\n",
    "    \n",
    "    unique_terms = set(unique_terms)\n",
    "    return unique_terms"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "experimental-vulnerability",
   "metadata": {},
   "source": [
    "Calculates the TF-IDF. This function ended up not being used when working out the term by document matrix, since it was simpler to use another method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "established-assist",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_TFIDF(dataset):\n",
    "    \n",
    "    \n",
    "    N = len(dataset)\n",
    "    df_dict = calculate_DF(dataset)\n",
    "    \n",
    "    tf_idf = {}\n",
    "    \n",
    "    for i in range(N):\n",
    "        \n",
    "        tokens = dataset[i]\n",
    "        counter = Counter(tokens)\n",
    "        \n",
    "        for token in np.unique(tokens):\n",
    "\n",
    "            tf = counter[token]/len(tokens)\n",
    "            df = df_dict[token]\n",
    "            idf = np.log(N/(df))\n",
    "            tf_idf[i, token] = tf*idf\n",
    "            #tf_idf[filenames[i], token] = tf*idf\n",
    "        \n",
    "    return tf_idf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "restricted-spelling",
   "metadata": {},
   "source": [
    "## Creating Dataset and calculating document weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "chief-tuner",
   "metadata": {},
   "source": [
    "Creating a list of all the directories of the files in the dataset.\n",
    "\n",
    "* os.walk gives us the files in the directory\n",
    "* os.getcwd gives us the current directory and title. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "democratic-sequence",
   "metadata": {},
   "outputs": [],
   "source": [
    "filenames = []\n",
    "for (root,dirs,files) in os.walk(str(os.getcwd())+'/docs-raw-texts/'):\n",
    "    filenames.extend(files)\n",
    "    path = root\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "developmental-radiation",
   "metadata": {},
   "source": [
    "Creating a list of all the titles of the articles and their directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "available-investigator",
   "metadata": {},
   "outputs": [
    {
     "output_type": "error",
     "ename": "FeatureNotFound",
     "evalue": "Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFeatureNotFound\u001b[0m                           Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-67-5cc820f5d601>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      7\u001b[0m     \u001b[0mfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 9\u001b[1;33m     \u001b[0mbs_content\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbs\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"xml\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     10\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     11\u001b[0m     \u001b[1;31m#returns \"bs4.element.Tag\" object\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\bs4\\__init__.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, markup, features, builder, parse_only, from_encoding, exclude_encodings, element_classes, **kwargs)\u001b[0m\n\u001b[0;32m    241\u001b[0m             \u001b[0mbuilder_class\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbuilder_registry\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlookup\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mfeatures\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    242\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mbuilder_class\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 243\u001b[1;33m                 raise FeatureNotFound(\n\u001b[0m\u001b[0;32m    244\u001b[0m                     \u001b[1;34m\"Couldn't find a tree builder with the features you \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    245\u001b[0m                     \u001b[1;34m\"requested: %s. Do you need to install a parser library?\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFeatureNotFound\u001b[0m: Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?"
     ]
    }
   ],
   "source": [
    "file_dataset = []\n",
    "for f in filenames:\n",
    "    temp = []\n",
    "    \n",
    "    file = open(path+f, 'r', encoding=\"utf8\")\n",
    "    text = file.read().strip()\n",
    "    file.close()\n",
    "\n",
    "    bs_content = bs(text, \"xml\")\n",
    "\n",
    "    #returns \"bs4.element.Tag\" object\n",
    "    file_title = (bs_content.find(\"fileDesc\")).get(\"title\")\n",
    "    \n",
    "    temp.append(str(path + f))\n",
    "    temp.append(str(file_title))\n",
    "    \n",
    "    file_dataset.append(temp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "floating-phoenix",
   "metadata": {},
   "source": [
    "Creating a list conataining all pre-processed files. This is the dataset which we would work on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "anticipated-watson",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = []\n",
    "\n",
    "for f in file_dataset:\n",
    "    data = preprocessing(f[0])\n",
    "    dataset.append(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "incorrect-estate",
   "metadata": {},
   "source": [
    "Building the term by document matrix with the TF-IDF weight for each term within each document. This is stored in a dictionary called \"tf_idf\". The decision to use a dictionary was made because they give the ability to set a key pair and a corresponding value. This allows us to store the tf-idf values in a more organised and straightforward way using the file and word as the key pair, with the value obviosly being the corresponding tfidf value.\n",
    "\n",
    "A Counter is a subclass of dictionary used for counting hashable objects. Essencially, this means that elements are stored as dictionary keys and their counts are stored as dictionary values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "loose-access",
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_words = list(get_unique_words(dataset))\n",
    "\n",
    "N = len(dataset)\n",
    "df_dict = calculate_DF(dataset)\n",
    "    \n",
    "tf_idf = {}\n",
    "    \n",
    "for i in range(N):\n",
    "        \n",
    "    tokens = dataset[i]\n",
    "    counter = Counter(tokens)\n",
    "        \n",
    "    for token in np.unique(tokens):\n",
    "\n",
    "        tf = counter[token]/len(tokens)\n",
    "        df = df_dict[token]\n",
    "        idf = np.log(N/(df))\n",
    "        tf_idf[i, token] = tf*idf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "british-contractor",
   "metadata": {},
   "source": [
    "## TF-IDF Cosine Similarity Ranking\n",
    "\n",
    "Observe the graph below, the left vector is the query and the right vector is the document. Imagine the vectors of all the documents are plotted, by considering the angle between those two vectors, the document with the smallest angle (cosine distance) would be the document which matches mostly with the query.\n",
    "\n",
    "![Cosine Similarity Explenation](https://www.oreilly.com/library/view/statistics-for-machine/9781788295758/assets/2b4a7a82-ad4c-4b2a-b808-e423a334de6f.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "psychological-signature",
   "metadata": {},
   "source": [
    "Getting the document frequency for a particular word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "angry-scratch",
   "metadata": {},
   "outputs": [],
   "source": [
    "def doc_freq(word):\n",
    "    c = 0\n",
    "    try:\n",
    "        c = dfDict[word]\n",
    "    except:\n",
    "        pass\n",
    "    return c"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "german-surge",
   "metadata": {},
   "source": [
    "Creating a Vectors from the tf_idf dictionary, using the current index of the word as the index of the vector.\n",
    "By converting all the data to vectors, we would be able to calculate the cosine similarity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "classified-continuity",
   "metadata": {},
   "outputs": [],
   "source": [
    "D = np.zeros((N, len(unique_words)))\n",
    "for i in tf_idf:\n",
    "    try:\n",
    "        ind = unique_words.index(i[1])\n",
    "        D[i[0]][ind] = tf_idf[i]\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "lonely-clone",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gen_vector(tokens):\n",
    "\n",
    "    Q = np.zeros((len(unique_words)))\n",
    "    \n",
    "    counter = Counter(tokens)\n",
    "    words_count = len(tokens)\n",
    "\n",
    "    query_weights = {}\n",
    "    \n",
    "    for token in np.unique(tokens):\n",
    "        \n",
    "        tf = counter[token]/words_count\n",
    "        df = doc_freq(token)\n",
    "        idf = math.log((N+1)/(df+1))\n",
    "\n",
    "        try:\n",
    "            ind = unique_words.index(token)\n",
    "            Q[ind] = tf*idf\n",
    "        except:\n",
    "            pass\n",
    "    return Q"
   ]
  },
  {
   "attachments": {
    "870ac3bd-3b8b-4ed7-a9ef-5022fdd7dd75.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdoAAACMCAYAAADbRGjxAAAgAElEQVR4Ae3dB9QsRfE2cBQDRoKKCUXMgqKYcwbFLIpiQDGCIuI1IoKCYCaZMOA1Y0QFI+acA4oJFQwYMEfMYf7n199X9/SdO7O773t3dvflrT5nzu7OzvT0PB2equrq6k2aTIlAIpAIJAKJQCIwGAKbDJZzZpwIJAKJQCKQCCQCTRJtNoJEIBFIBBKBRGBABJJoBwQ3s04EEoFEIBFIBJJosw0kAolAIpAIJAIDIpBEOyC4mXUikAgkAolAIpBEm20gEUgEEoFEIBEYEIEk2gHBzawTgUQgEUgEEoEk2mwDiUAikAgkAonAgAgk0Q4IbmadCCQCiUAikAgk0WYbSAQSgUQgEUgEBkQgiXZAcDPrRCARSAQSgUQgiTbbQCKQCCQCiUAiMCACSbQDgptZJwKJQCKQCCQCSbTZBhKBRCARSAQSgQERSKIdENzMOhFIBBKBRCARSKLNNpAIJAKJQCKQCAyIQBLtgOBm1olAIpAIJAKJQBJttoFEIBFIBBKBRGBABJJoBwQ3s04EEoFEIBFIBJJosw0kAolAIpAIJAIDIpBEOyC4mXUikAgkAolAIpBEm20gEUgEEoFEIBEYEIEk2gHBzawTgUQgEVhJCPz1r39tfvOb3zR//OMfV1KxF76sSbQLX0VZwEQgEUgEhkfg7LPPbk466aRmr732aj73uc8N/8DqCQj+lFNOKc8/9thjm2OOOaZ54Qtf2Bx99NHlu9/t40UvelGzdu3a5vOf/3wRDP73v/9VOS7W1yTaxaqPLE0ikAgkAnNB4Ctf+Uqz8847NzvttFPz3ve+d6Zl+Mc//tH8+Mc/bpRh//33b3bcccfm0pe+dDm22WabUqa99967cdzrXvdqbnGLW5Rrdthhh+ZWt7pV89znPrcQ7l/+8peZlnvShyXRTopUXpcIJAKJwDkUAST3uMc9rrnUpS7VbL/99jMn2oCVVvqzn/2skOi5znWuxoFoDzvssELEyvnFL36xOe6445pnP/vZzc1udrPmIhe5SHPxi1+8uc1tbtN86UtfamjHi5aSaBetRrI8iUAikAjMEIF//vOfzaGHHtpc9rKXbc597nM3tMSTTz55hiVY/1H/+te/mhvc4AalLOc5z3maO97xjp1zxn/729+aj33sY831r3/9QshIebfddmt+8IMfrJ/hAvxKol2ASsgiJAKJQCIwDwT+/e9/N6eddlpzhStcoZAVor3RjW7UfPnLX55HcRoa7U9+8pOi0SrLFlts0Tz84Q/vLQuTs/lkJOt6mu2szd69hav+SKKtwMiviUAikAisJgT+9Kc/Nde+9rWbzTbbbB1ZPfCBD2z++9//zgUGz33Vq15VzMXI01zthz70oZFl+c53vrOu7JtuumlzwgknjLx+Hn8m0c4D9XxmIpAIJAJzRuC3v/1tw3N32223bS5wgQsUskJue+6559xK9p///Kd5znOe01ziEpco5bnJTW5S5l1HFei73/3uOqKl1b797W8fdflc/kuinQvs+dBEIBFIBOaHAJI9/vjji+PTmjVrmq233rqYXudNtEzZtNjzn//8hTwf9ahHNX/4wx96gTK/zAEqTMdJtL1Q5R+JQCKQCCQCs0LAvOYXvvCFZtddd21ud7vbNUyvV7rSleZOtMzGv/vd7wrpI05a9sEHHzwSFo5P9773vUvZkewVr3jFsabmkRkO9GdqtAMBm9kmAolAIrCICHz7298uQSk4PX3mM59pmGuvcY1rFLI63/nO1zzsYQ+bS7F5Eb/zne9sttpqq6KhIv+3vvWtnWWhyZ5xxhkliIXlPeZmL3ShC5X3sgRo0VIS7aLVSJYnEUgEEoGBEPj973/fPOMZzyhLeA444IDyFKR19atfvRCtIBHPf/7zB3r66GyFftxnn33WzRff6U53ar71rW+Vm2i7f/7zn8vxq1/9qvna177WPOYxjylLkpDslltuWYJtfPOb32xo7IuWkmgXrUayPIlAIpAIDIAAAnrHO95RAjuIvoR0abOnn356MbkyvdJyP/nJTw7w9PFZWtaD6BEn0/Hznve85swzzyzm5J///OfNe97znrJ0R2jG61znOs3mm29etNjLXe5yze67716WBVmDu4gpiXYRayXLlAgkAonAlBH49Kc/XYI7mJelEUp///vfSzzh8PK94Q1vODeiZQoWoGKTTTYpRHuVq1ylufnNb17CLV7vetdrmLXPe97zriNiUawe8IAHlHchMCxySqJd5NrJsiUCiUAisJEICALBy9j6WFGWPvCBDxSClW2baGm0n/rUpzbyiUu/XfmOOOKIdQ5ZNFa/3/KWt5Rwi3vssUfxkKbx8kgOx6enPOUpzS9/+culP3DGdyTRzhjwfFwikAgkArNEgJPRox/96OZqV7ta88Y3vnG9cIaI9qijjirrVsN0vBSiFfCCeZc2ao51ucn9zL+xTOcOd7hD2WDAdn3y/eEPf9h8/etfL7v7WPJz+ctfvrngBS/YXPnKV26e/vSnlyVAuXvPctHP+xKBRCARSASWjYCt78QDRkiPeMQjCmHVmSHaI488soQuRHJ2xbEuddL08pe/vOG0dPvb37552tOeVuZJJ723vs5Wd8hTGczR2uCgS1O1YYAlPZymmI6Zki3pee1rX9tw6lrUlBrtotZMlisRSAQSgY1AAMkiTbvaOMzR/uIXvyga4q9//evyyQFpv/32ay560YuWuVGar91zJk3Xuta1ytwpbfi6171uecak98Z1glS8733vW2c2tlzns5/97EjitJ2e7fEQM832lre8ZfFIjjwX7TOJdtFqJMuTCCQCicBGIsA5SGjC+9znPkVbvctd7tK84AUvaF760peud9BmaYackJCW8IdLSZYF0UARrYhOH/3oR5dye7mWafjwww8vecjHbjzf+973RuZDiBDMQpkdBIVREaRGZjaDP5NoZwByPiIRSAQSgVkiIMLSS17ykmIyFjnJZukO3+O3z7ve9a7F/Bqevksl2nve856N5TW8lkWaMo+61PSNb3yjmJ6RLFMwDfuss84am439aN2TRDsWqrwgEUgEEoFEYNoIMBNbGnPrW9+6bD3HUajrQMhMtYgWaT33uc9dUlGYoE899dRiol5uRCbzs7H/LKIVUIMX8rhEKAii9Q6p0Y5DLP9PBBKBRCARmAoCX/ziF4vJmNl4nAlW0AokRStcDtEyUXNCEgzDXOtyEnO28Imeb+kOD+Nx62JFiTrooIOSaJcDeN6TCCQCiUAisHwEEBDvYs5P9nEd54mLaM1vIjkBIma9xdyPfvSjUl7P59RkWY8yjUtiNN/4xjcuAsKFL3zhxhz0X/7yl3G3ze3/nKOdG/T54EQgEUgEpoMAszCSffWrX108cJmAmXVHJUtlmJhptIiOqXkpS3tG5T3pf54X3sNCKloi5D1GJY5Q++67b3Oxi12sEO0222xTNiNY1PCL3iWJdlSNLuE/kqOGa11apg0RsGh+kedQNixxnpkEAfWq3S9nkGNupL0ghPpomw3juknKsxqvEXBfYAcke9WrXrXhoGTru3GJt69gD7ajQ7TWwp5yyinjbpvq/yeddFIJpMF0fclLXrLskds3hnpPpnD3XPva1y7ezltssUVxwkLOGbBiqlWzeJlxKOAQYIunn/70p4MW0Do4C7kNcEMlDVr+or44dOL4Xn+2z5M0+wZcW3OtXbu2dBQdaZE7xVC4npPyRYbWWzLhCfY+ibnP+6t3xClI/Oc+97myzZklJsLtHXbYYWWZx/e///31dmDRp17xilcU8tDGtM9M/w8BWHAcev3rX18CPogHjHAnGR949oqyFJus0yytT51VIng/8pGPLPOziFbACg5VBLeoY23FmEMo4NH80Ic+tASooP3ydGZq1gYXPaVGuxE1ZLCJRs5V/klPetKy3NsnKYKGZw7ikEMOaZ7whCeUBjnJfcu5Riflcv+Rj3ykOfHEE0u80Te96U3Nm9/85san+KOECoc5nTjv2tNOO62YfjhG1GT6iU98ornf/e5XosjwMtR5ojMtp4x5z3wQUKcEJQOi9n6Pe9yjrL2cRMBU3zQP9X/ooYeW9ZLXvOY1y/ya6EIXv/jFi5Zy3/vet7SjcK6x9Zng8THviNTbWu980JjfU2GpnxJY7Nm69dZbl6UxNNPjjjuu4NzVv5wjDKsH2qvYx7GGdocddmje9ra3lTHNWDNtjLUddYo8EefJJ59c1t4qM6L1DsYWISCFdbQd3pe//OUy5ohApXzmk2mxNHf75mpLKyEtNNGqmDgWDUwNViO/+93vXvZyJJXT8IZKGv6xxx5bNkW+293uVhajD/UsHVjEGBrHzW52syJxcv+PtXbmdLjj2+lju+22azgjWLTu06J16/d0lBgoo5zwsofkFa5whdLJhsQrnpmf00WAtqFdCCog9J1dYPpMfe0nq+8PfvCDzWUuc5myj6h1nSF0+U+w+wh+sOeee64nTCJXyz7E6yXgGYRXc4KX7eyMBUyucItDvbAQdE3VOEcYRsZ1sAlk535rYmmJJ5xwQrGcGX+nlRCscYXWTGhSTsQZdR5lEECDAGbHHuUhCLiGti761IMe9KBCwIvs/NTGbGGJlkYl3iVN0TzCIiUSIRd0phaeei972ctKx++SIKdRbvO/NkA2L6GxDU20IXkaQGns3jGIFrF6XxKp/5iE3/ve9xYhgBfgZpttVjQTA7Ey61yRSMjMVTShiL0qbmmmlYEAs60BWDu0u8oXvvCFxrlJBmPmYN6wSAHRvv/97y99RvvQb0JwFVjenCEzIgtJJP8jCUEKtt9++2IdMV2zWhOipe3B8V3velfZZ9ZesyxM5jCFMOwjWkTHWuU69zjUqwPmwiHSKgnGk9TtpHWgrpG8vD3D8wlNnluXwbl3v/vdZVxRHv97L+dNnbGGRbuZ9Nnzvm5hiVZDsGUTMwHX7UVKpDKmrm233bZ50YteVBrktM0s9fsiMwvPub+T+oYm2vrZvl/jGtdYR7Q0+NqbkdBhsHVO57nJTW5SJFCE+5CHPGSDIONw0oH9R6Ie2gzefpf8vXwEDHb6pDoWao/lY5KBWChAMXSZBs2rIQZE0RZMEYNtz2g5BMrXvOY16xXWs/gnEMCRrfm61SqosRaxLtQ+E/V3/7UtSsAcd1/kQVsctzRovcqZ4If6JrzHM5b6uegOT6MgWFiiJfXwgmNmNGezKInJ6lWvelVxLedIgHSHJFmkdPTRRxezbMxlHHDAAZ07W0wbI+9Fa1AHNFraiPiifclASeqMOZ/LXvayRbLuut6cLfMzU7M5XgNDpsVFgCbCZMekp76Q7CRJvfIr0BYsx9CXnesiaOTLesXRRVu31KMrFN9Xv/rV5ra3vW0RAM3drSQT4iSY5TXnPAQWlmgRmAXXJufNRSxCMk9ksp7ZzMChw086P7Wc8pMoDWqkd3MUHAYctr3q8+5dznP67mGe4e1pzsRz7dTBgWFUgkkQrXs+/OEPd15OCyaw0GppSN4p02IiwLuYUHmlK12pRBzSNydJyFj9m2vj2UqwMqfbl9pEa67QFFI7IWr9kJCmL9KQMyUCi4zAwhLtIoImtJn9GjkD7bHHHuvNP067vIjUHCfzWHhj0ipJ+ubGZpEMaEx5W265ZSFa5utx6+wQLaEgHKf6iFb5zdsxgzOJs16Y8820eAi84Q1vKE5vAgNwyJs06S+c6dSveVnzq6MSi8iTn/zkYjompPGB6AugIKLQwx/+8OITwEHG1EXbFD3qWflfIjBLBJJoJ0Sb5soEZv7IgGOj4a45kAmzG3sZr9299tqraHuch4K8EK0BbBbJHAqzsUDfiHNc1BYaOJNwmLhptuO2zTJHy/mFk5WlQ+ekxCKg3dSHc0tJ2lh9f3xf6nRFX1ki/y6SYt5lluVdro7s+dlHfO13kq+5VHOy2gNT7zjnJWTJB8CztDeeqQS3vnTMMccUEjel8eIXv3hic3Zffnk+ERgKgUGIVqfV0RwGBIdzOq7D+fYcTfzn07XucV17AOi6LvKvQWqXof6v/l4/Sz7tcsW1OjxtlqTNOchcbd+1cc9yP5UJkTO5PfWpTy2msXkQLVM57T2001EaCSxo4JzEYGRwvfSlL907RxvYmB5gAjS4et92fcd1K+kz2p42YymMw6J6S1l4fHa1//b7wVN7tD417udJKh+fvC9HtdfITz4EIMJZlMX9vvNaZQb2nSm/nZAzhyT1qD7NnxK+xiXlIihamuE+bZcz1Lhkba62E8duu+1WvFT77lNu5miOU5aKCByTKRFYRASmTrSk0mc+85klDJi9Dkmo++yzT1k/J3zW6aefXv6r18GZyzGgGKgtTj/++OObBz7wgeVezg51Ym50jcNaMAvmXcs9PLQF3mnMXZ6tDJYIWY7TntdEJM973vPKPo2uEz9Tvl3p8Y9//LrYmpY3DEkI5qUtzrZu0IBKKzRgITyD1iw0WvNldtXgPey5nGBGmYF5g9pYOnbhUF71zplrVDJYMi/Sfs0FL8XRZlS+8/oPYakfYfBYPnbaaaey9o+HOmcgpKXN6gujvDqZRgV1QCTuIdyZq+e5ayrBUioL9kdFxdEf9Kk73/nOxZHNkipew/KRh3yVj0bYtcsLU65lWoQg9amvTNLukbEpBysG3Gce3hKNUQkW5m+DZH1aRtbus3UeIgXtvPPO5RnM01YAaLeZEoFFQ2CqRIsskYMBhsmRRGrO5Xa3u906xwUmJB2w3ruQ5G+5B7IT2Nq9BiXzOrGGVkckgUeEGE4QJGamXIOQTkdTMMgxR0YUEf+bY9Qhka2E2F2HCDg18XJ0nUHM4NVeMmDgMDgiA9cYRIZIBjFSucHNfCgHIdqBTwOWAynxAB06IXhzYGE2NlhbZtROBjaaGmEnNB9ka4suQtEoMpEXcyKnFwKEetBeVupgSYszvUAQE1pOEBPYaLesAdqpOtTWWEeYYZmC20lQB/1E+9eOeXpb7mZt4Qtf+MIi9GjTzLLaycc//vF2FuW352ov+hLiJ5iaY7fumTBsDpSTkjK161ZfIhQHWQoUYd3juIQYkTahSX+Rt34Gm1GJd/GaNWvWI1rOcqOS9mWMQcraqTFknGA3Kr/8LxEYCoGpES2NRjQSJGmgZSoykFhDZ96O85BOyxRpkKg9FwU/YFKjMRncRRjSQUneosVICMd1CNKAQ9IPk6aACjQJJjFONQYnA4z7dUKDuDIZ1JEsc6XB3XWuJ93HoMADlpZVJztcGDzlxfOSxjntZGAzGImcpAwWotO4vTNJHR7ewxKLcYPWNMpmsCTIeKb3fv7zn1+0ffVG6+cU40ACtCuDPgKBE0FHUPOwMIwqj3dEQp7DBKj9aEuTJgO7PJRrWsdynLJCA9WmhLVDXN6DoKGM/jdnCUvtlgZmDr7uB/DSvuGpn8CfVusajmkOZXvjG99YAn5oE4QTbV0fq4UaliX9EcmaW6Ut0lD1I/nwJOZB7n7Yt4nWtXaAQcTKLA99dFxiqXriE59Y+pP79GWWkbpsXXmcccYZxbLkHgdTMIFgVCIAv+51ryvXewcCvvFm0gQL76k/Tavt1Ja6ScuR153zEZga0SKxXXbZpTR6RIqskJpkACF1G4SZoUR/6RpMaXMPfvCD1xEtj1RSazu57v73v38ZsHRk0r9Ot+uuuxYzneDkCJopU6c1IDlI80xppGBa4ytf+cqyTRTyCnJ33qBVJ1pELHFhBkP0004I1aBqcD3wwAPXhZhDeLQR5ScM7L333oNL7WEZiEEPKZh7JTzR1JjkldPApq5pE7QvYfNoXrSTpcxfMxEGoXtPbQcZTJIQDyEARuo1Pn2P387F+fgev+O6+lr5LSXRSrU15MCpi5m0bWJFtgTJaIs+tVftMZJ2bRlNmN/b/8d1SEGUpXA0cj0ihDvykBAnCxHCRuhda02RLcsDMm0TLYuT/qfNaQcsQqxG4xIhTBzaqE+Cqeg++vuog/BLYI42p71NsgONICmwjPsIqKaOJkmuozUbj6IdtNtH/TvaTJyr20x8J2RmSgTaCEyNaJmpdHYN3uArikw9WDLHim1Kgj7ooIM6w4MxV9IuDdw6D9NbkHVdcCYtmijNgOmX5M5pA5GTng0gIVkrj06PKJlgmfYIBDqYjiZ/ZlEDkmfydGwvYaFhhmQ/BNEqh3lY+BkUaT8GZsnARUOK9zAQD20eoyHSZmLw4qzE4xjOTI6sAZZURKxSgzHs3dMVYKCuu67vbaLlZd0liHXd63lM+eYaTSf4rI84F5/t/5yPI/7z/KUkpnykCC+49Jm+TZHQ2LfaaqtyECrVtUQY1e78p71qb9pyV/t3vfbiWVFHrudAFyQjL31EX9JHEHrbwsAKRCDV9ttEi+Row/LXzyYhWv1dfjXxwXb//fcvVguWi/bxnOc8p+zaQ0iL+3yKl00rH5eCaMO61Y6RPOp+7dy8M+tY1H20hfiM8/EZ5306F5/x3dRWpkSgjcAgRKvTk0iRK+1IMiDqVIiKxkADrZPraG9hwqVh9s3RcHzi2KFzkZjN6xnARFCKzmmgoenqtEySOgCHJudosgYk0j9yN7jFdbx8Y7CK8nEEigHN93FLVuK+ST5pPhxazGMLCEGzDa3E/W2ipVUq85DJIIs4YuAzeNWE4PkGbiZMkX7MW6s3xEs46tKeRpVXUAx1BGPPpDVPshuMPJkktS1lZjb0Oenh+vqI+5YqyBBAxOb1DuZM+xLhwTSA92OZYIoNIiUcOh/tjLXA9nN9iaWHVSeuR86INeopiBae+qOdkwR/IAREn5S3tm7uuO2XwEIlT33MwYoxzjeA+ZeGHGVC4HwbaN9dh/d1EMBjfl95TUFMYqZWfsQOd2V0r+d1WcG6cIQDgdx7Rd3Xn9E26nPjvtdTAV3PzHOrE4GpEa2OYZDR2DV6nZQkW5OCAVgHNjDWnR30CJLJNIiWFtm3Zk/sY+YyHRrBMpEhwHpujfmXxB8DzWMf+9gyGCLZIAKDjkFSXsrLFEeSbicEKB/P05FJwtNIzKvyoiUSHBBXOyFaGqRnK6M53BrT9vXT+G1uDHnGO9MY1VlXgjnNjKVCvasP1oy26bTr3jjHy9zzYrA0+E5KtJHHPD9Dm+VZzNt9OYm5nDUnSGoS0ylzddRR9Lkw7/pkudFm4hqEqn2zCOl/sYRIG6utT8qPlN0rX4c66loCVL+rQCq0w3gHlqJxpKcMfCBM2Singym3dpasn9H+LmocklbG6J/jntnOY16/vbt+ksfiYKBOhkhTI1qEoRPTDjV6Hcagy7QTndhLMInWGlu8FEmcNBwDA02JhNyVOH9EpyStmxNi+qrJm3brP9chb/M/SJImFmAiD85XsXyBJyhnlHaqiZZmsBQSaedV/0b0vLQtuSAAhHZTX0PY4BxjEPEeBuQuT9X6no39bo4xzPfwM+h21ZnnwIITmndQRvO56qeui3HlkT9LQwyW6mQlEi1rzSgtdBQOBENtP0hqEqJlWibguCewo8lK2pbpFOZfdegadRoOa4QnhCvpk9Enyon/T7SRr7y9V/uauDY+PU+fi3eYxIxLo2a+DYuGNk7gDstU5N336R3Mj8b707xXCtFyxFJ+ykcchJ74Xn86X/8Xv8edq/Oov096f9d1kU/97Ppcfb7v/vqauNdn1/l2HvXv+vo4X+e31DwnbXd97bHv/NSI1kCM7DT0mgSRlLWR46Rhc7wkevcacDg1tAlFR7dcgmSuY7mWiQ1ZMSHGQICgmYpjgJEfZx0m2prMmMs8M8i9b1cc2nLkxVw3rWQZB62bNkcr2m+//cpBa3X4TSsxOHpfgxDT1LSIvus9eGB6rvc1+DFHjvNyZiFgMXAPLJlRY4656xntc4Qf89AxWK5UoiVkWou9nMRZMAjK5yRES4MkhMEtsAui1R8JktahI1ttJ/JHhtbO3vSmNy3rz5lP24IR4cf1kfc4D2DCNKfBeIa2wLLRnoZpY0NIY71ynzLSThHlpO3Hkr3ahL6SiJaAQUClBMTBstX+Xp+L/3w6H0d9Pr7HffU1cS6uic+4pu//+rqu73EuPrvyiXPxrLg2PuvzcW381/6Ma/uuG/d/5Fdf5zuP+Un9Q9ptedTvqRGthyBTnd/6vNCIaIvmHxFpX0KSSCfIzDKgrni+iJTjDOcOA4BgCpw02mYm99JOIz/XmReuTcs6sjLFNT7NFbbnjpXZko0YQKZFtOZ6kIuBQeVyEOMtXR9MyszkMeAhvqHNxiTAMFUbkM0JT7JkgYYUWBosJx0o4XtOIVptXrtbTmoTrWmYMAP35YcMwymwTbTuCbKl+fKU1W9qjZP1Qdt+0IMeVOZo6zqTd9Sn9jeOaM1zElRd6z6Hc32WkHgnPhGcJ91HO7czVVcfjOvbnyuZaFmrxAmw7Mthqs0R3+O89cXxPa5xLq6N/+KzPj/J97jPp+sj7/re9jVxbVzTLmNcH//H7/jsOx//+6yv6fruXH2+vrf+Pu6aOh+Wm5habLe1jfk9VaJVEBIsZ6GQonU40jNPzlqbrAuNPMIRxIDBjMgxqp1ocgJacJRyHecbu3i0E0maxKLzIlkesUyRdafndFSTg+UqfTvT1ERr3mmpzjJ1+QgLBBLza0ysggjIr+941rOetY5oDZJDE61lFtYfw9fcNcFmnFbi/QyQtFmYL5VoaWGEsyCLpWi0sCQc6CAOhFB/97s+4r+4tuu/0Arrehv1PeZovTsPbdMgYV0ZdZ+BNoTENtGa7+X0NyqNI1r36jPMyNbR8komRKkfZXXon5yWWCTqttUmWlMbo3wTQriVp3YqwEWdX9d7aPPW10cZ9G3kXBN+1331OdqvKaF4n6U4Q7GYEXg5VEU7qNtHfPdftJe+z7ifxW3SZDw0DvX1/TzfPy4Ohc0QJKs9bDTRGoTbi9EVlgktvFF1Astm+uZOaJZMKDHQknLbtnIDF/MUE3BcZ063PY+rLCLRhA7sug0AABk5SURBVLMUByeenm2SZ+amDejkykdz1Mm7EvO3azzX943Z0k05LLJnthMIfZxZ1jWeHYNRn1NSV7mXes4Ad/jhh69bU6z+eMPWAkpXngZg9a2MyNbc/FIGS+ssCUZRr0txhoIHkmeCn9bBi30pqSZaZnP1O868b4A1txgbKZijhR8MHHDs87qPsrWJlobqnMQUjEjqJT36KiHCWk94E0K1LQfhyh7QkeQTgpP/lbVL+I3ra6IlCCNu64JHJaRlGoilxvz2W9/61lGXd/7nuQThaDtLIVrtlsBr7JlW2zEvnSkRaCOwbKJFfEw8pGQT+vXA4jszLTOr+UUdVYcy79pOBnEDE3OxzmKwoaG0ydtv0mJ4pyJSThPthLiYZOVjzodkrSzt8ilL7Siiw5HAEUSbWHRCg47yyW+5ASvkjcxpK9bLhjbTfof4rRx2KIEfLcFclgF6qOT9mRE9j8mf9hOObKOeSRuzIQDMaUdiS7fn/EbdT2sOkvHJXE1inSSpW+3HnDayN7/sM+a543yc81l/j+vqc69+9asnefS6ayxPCdIyT6tumdvb7WjdDU1TvHjN/a9du7acNlcZc/HRDzgXtv0U6jxoxAS2uF4dhDALF5altiDqfm2Iwxvtu8a93teVY4k2oC049Kkg8boM8b0mWiZgJlGadF9CcvLUj/VpFq9RGnNfPvU6Wu+i7RmPJkkEDzgwHUa7iM9oN9E+4nf8H591e3PO8rZMiUAbgWUTLeJ7/etfXzRMofi6zIskVOZdHbWPaHnemYt0jQHDgIUs5V8P1q5DrAZy11ko3rWelZSMDOVHsuZU0k4EBPNWOqbD3JqBxeBE0213eFKyAcFzl0u0iJ5JUSxl2gNhotY22mX02zsLFOFdzGOZvx3KtOF5NHVChefF1mNtgacuJ2GL8MATG0kQbGhKTHG1YFPf0/W9HbCCk8iQAkVXGTbmnME6pipgp40avINs4eSACfJFnkiRRSXWiyIH86jamEM+NGXtsi/pdzG/qd8QyqLt0lytXeWI2JWUwbIa2qTn6Qc10dJe9V3nlYXQNYpoeeubn3btOKKFBccp2r++Z6po1IbwXeWPc4g2hGDPpq1PKqRFHov0CZuu1He+69ppn+t7dvu83+1zytJ3ftrlHJVfV7mibKPum9Z/yyZaHdUSGtKoAQFxtBMNydybDoBcujTaWH/rGqYv2oBOzvyLNCPpPDq+wdzAgEA50bRThJ1zjWU/5k7aSVlD6zXQGOCQu05LQm2/i8EqBAZEO0lw9fYzaRpM3SJUwY2031f5cS+NRbQogx1NiVl3SKJlwmXCUxe8ub13nwkYaTDvmw+HicGOYIM8RpFzvFv92SZaAslSiLrOax7fkUx4zsJOfWnLNHOaKqHBlIE25jdiIZjYiCE2umA5YLp1bxCtviWSUl+iPekPyIq5nhk92hSi5ZzEWbAruY4HO23c/eqvJlp5CarhP+80LjIUAVWfVP5RREvQIIAINKPvMdsS8Gqhuqu8fedqjVY5TUOtpLZTv1dg07aEEMjHmeHrfKb93bPbSkGUta43bZxy0E6UsC5FrH3dUL9jrGqPZX7rm9Fnhnq+fJdNtAYG2haJ2hKDLicAaz5pnjqAEGUk6HailfrPNToorZVkjnBrT2WDQpjnXNs17ypvZQkJl7dzlwejcung8tHZOWSYKzNQ0KbaS5FoG7xAXe99lW0pySAkqANTNc9PldvuTF35KSeHKc8dRbQasfk8pivzm5Y7Cfe3lMaN+OGAIDzPvJdO09UIdSgesfBiBncPLY7mP0oD63pHWAsaYoBmquSFzbKwkpJ3YL62PMB7wM+hvhEgQRSpCubgN4GE74C2HwOYwQC5aVvhX4BEtWECVzvRLhGp51miZuqjNvMjWuRIa7QJRpfwo10TgOWhfdWCsDoWmzn6nHwErehLrudPIC/l1xbb5GBgs6SOduwaZm+rDZbSTuvneyarmWfq8/xAQnCpr1sJ31kirHqATVgllJuPBEsWS9i8kmcrg7JEUkZltXwsyqsume7bWzeyvBiXOJ7NOlFMjO3KStCNREk46qijSnmjD8Z/Q3wum2gNzLQfRKWji8FrcDdYIBHESDs0oJBeSd9d5kCkEOYvA611o8yXOn1IR8gSWDGAGdCYJ+uEEAwuoZExfVoy05XMfSmz/HRQ5EcgYHYSdq2dVATzqPJ53xve8IbtSzp/G0CUyeBJkzUg+DRv1pau2hmQ+mm+iEw5PduAbYCFYz1w8qimVXpn7xWDe7vBt5/htwFeA6TNMO97Fo2KZyoCpOXSxs0nszAQRMy/MX0iDHPrBCUORASjpTZaePPkho12IF7ycgferveb1TkDPHKhqXqXaKs+kQqckJb25j2FI22v19MmWHM41CA+96pPjnrqUp/TJ1iBWEfUt/YU8a9r4U27Q9LajblgZCtvQgFrFKsB8606VB51WgcJ0S5YkbQDhM9HwMA0KtFMtQXX8+I3120cUGYESxjkVKW/GRto995puQnm4bWsXwqxqt+sxITEWPG0n7pdENIJVOpwXsmzlUFZIimjsrJ6BAGrb1Mo7Qh3plEImXw5Zp0I7Zxr9b96bDcGG9M51xLYhk7LJlpSjMKTiBEpoiI1CBShU9NunKcZWcOqw3UlA3eQkIHIYK9i6k6v8wALKA6m3rZ0ZGB45zvfWUjaNeaL+jYqN2AY/EIS9nyDG22sJrC6vJZaaETu4Xnc1nrraw14Gh+Skq/Bx+DikIe5M422TUru895MkZYekdCRpsGOKZcQwVkJFgbbICQDjrowiCufwwBNiuxLBlsEysrAxKnumNo9y4HghcWjtRIUdCgDPoL1/sqljg2YduxRv+336Xt2fZ63recYKLUXZDBOCKnvX5Tv3h25qR99wSBU14c6UUemUvgkmFapibF+D+0QkSIr1gKasSkE99FcEah2ZPCyBrvdF+SlLKZHECnzrzboN0sQ7VuYQ8ve1LP2RJCqzYDyoA1YH8nSpPymLkbVMS3BtIpneab8OUTSaAxq2rOpHfkg3lF51Xj0fSfY6edw1t79nsWg2VeejTmv/7B6wY2yEolgzTJBuZhX8mxlqOfolVFZtfcY2y0BM061veWNL8iaxjvrpE1aU0tQrAUFY5/xi0BbW4KGKt+yiVYn1HGYbsR3RaYGawOJTmzOyvwP6SYqouslOEHohEy5Oo35OmRTd3oarZimOqwDgba1YxqtuVUOWq6hofWZkeRl0KGZKieTFwmnj2SVmwaAIAkDGp75NOTelQyg8lMGjc47MbH4jMN7tyvYOzPnIS7e3PG+7U/vSMMJk43GhFSZ4uDoMIAa1PoSQQFeLAdMVu1nxG9OY/E9yu5TuD1lJTFuDDGydDBLIhPCxUpO2g/rAK902m1dH+qEgEiwoeH1tZ14f6Z5OBvgtFN5sWg49BMWBANXnwanbpAkIo42IR/WA+Z5fddcMa9n7aeL9LVH7RSpG1SZBcd5ymuT2r0pD33LnL0yE9JYaLTbSd4/cBj1aWwhnBEmkFQInqPuWdT/kmiHqZkVT7Q6JqmUZiQhDVqc5Su0Egfzwijycp8BP+5DTrNIpF5mMWUl2XQNMl3l0LGRAnM409/GEExX/ss9p/wED+/inWgzNM1RRLvcZ03zPnVPQGBqpP1YInJOSEiUAKJ/qI84aivNJO8JH/fE/dGvfI4zuWrjyqAPxv3tz3GkGWVk0mReVkcCmoxL+oW2WJfXs7r8Jcbl1fe/8YaGr+3YA5hZctxY05fXIpxPoh2mFlY80Q4Dy2LnauB86EMfWhx/mLgNdl3OQvN8C+UxV06ToEEtcuLsRcsyiDNNZ1pMBGjhzNTMs9aTLoKAqY2znsW67cVEbvJSJdFOjtVSrkyiXQpaC3ItzdGSImYwc3AHH3zwOvPtghSxlIfZkpf0POZEloKDKQPzluaxzVlmWkwEmJA5H3JSNP+8CO2K45P5Ne18qZaCRUQ5iXaYWkmiHQbXwXNlKuctaY0jJypzWItgsqLJmpcTJ5jJmCPWNE110wbW3DBHOoO3stZz8tN+Vua38QhoS+GQaH7XXOy8rDnmeS3jY1Xir3BOaDuLQLScyTjLGUfqNCtnqPYUWHuqo++38lpK1uX3kERb1+QK+65SOQnxzuVUZW5xYz0oNwYCg57ABjRtB89P83OLmAgqgohw0BFE3qYQbce2RSx3lqkpyyOY+A28PDm1sUn9G6aBn4HUXCwPbETLwbBeCjONZ8wrj3kTLY9cU03Gj7avxKyIlsDEg974oJ3xrje1FIc652DLI185TR34j+MdBYNjLn+fWvEJouXEuiK9jufVIBfhuSF5WYeo4i1hsIZwnIPKUGVHtNYAH3LIIcUbelFJllcoj3EdSOfgnaojZFoZCBjAeKpbrmHZFy9xTk/hEDnkW3DuYkkywPJm5s3PofGckuZFtKwSVpBYZmlZJAG43lwCvrMiWoKUshCelIGfSSxX5FlOwLKEUzQwcRWsAiEccNLj52FZmeWlliEF2SbRrvAeooEiVmteeWVaa2o5xjy0M84pvDoR7qKa0TR483z3ve99y9IXpuN25KAV3iRWRfENYAhOXSI9pKsPGCCHSto1TVZwCpqLwfacNqc/L6JliWN+F/DGWvZ5Em27/TzkIQ8pRCtwixgE1twHgfI6R8i0VJpsLEHjO8NhNTzqk2jbqK7g3wYea1Et3h63WfcKfs2NKjpNVlQgy6III7M0OW5UwfPmTgQIdLzwbYfHmlKHt+u8YSNO0lAEFxC9ynMWwet5I16n89Z5EC0czW9ac49kERrnMh7ddZqVRls/k/WLsySNVrAJXu99icIj7gCt1/UC78RmGkm0fait0PMkesEDSN+ZNkSA9toXQGTDq/PMSkFAvWr37eAr0yw/Ryykfk5OsyZaFjlWAXGMhelEskKvil/fdqKcB9GKnc9REnHSZvnEjErG3whb6h7TaFIS7SjUVuB/Gm4cK7D4gxcZNl1egYM/OB8wKALR5n0OlVZD25k10ZpXt6mFEJ+ifgXRcqps1+U8iNbUBK9ypGnntNBQ+9oYohVdznsk0fahlOcTgUQgEVjFCMySaFkfbBAilKd5b6ZZBOUwtdNO8yBau13RUJGm1QnjrGE10XqP1GjbtZi/E4FEIBFY5QjMkmh5jvPWtVxGXG5zm0G0ds9qp3kQLXINTVsM/VHLJ83nCo8rSpj38D7xHmk6btdm/k4EEoFEYJUiMCuiFU9eVC2bVVhvL6oWgjI/S3sMgqqrYZZEy2wtRrZd35Amc7A1233Jsi87mYlYFsRsN7BYopRE24dcnk8EEoFEYJUhMAuipRWK7iXIg01HmFs5bwbRimXd5XQ0S6Llx2FfW1tEIlrb65188skbtAbLfCzvEQ7UMjObSxAUbN9pjXUEMgmiXbHb5G3w5nkiEUgEEoFEYFkIDE20lvIIsSgIhDX/liFaokXDZWql0SJgG3200yyJ1rI/kZ/EQFcmG85bbkQoYCJWXkFSaLGWe1n2Y2mSgBWW9QghKTpUpCTaQCI/E4FEIBFY5QgMSbS0RKENRWNDtBFi0dKsZz3rWWWPbaQmYpstNttplkRL6xZDPkzZ1k+/7W1va97+9rc3xx9/fJlX3nHHHYvmKjgFTVUoXMEtEHJ7mVkSbbs283cikAgkAqsUgSGJ1lIe4TJt4mGZTKyTZV5ds2ZNIatFIFoaNu9i+30rz+abb17CK4ppfNOb3rQs86G9brrppsVMzFx8rWtdq3nDG95Qgph0LR9Mol2lHSpfOxFIBBKBNgJDEa05WHGMaX00Q8QThIRo999//3VES9udp0YrwhNnLJoqomXKFqqVKVhIRWZt8dGPPPLIhlZrbplGK3zka17zmnXhGWtsk2hrNPJ7IpAIJAKrGIEhiJbDEOI0d8kES1usQ58iWiE0aYmIbd5EK1IV83aEg9x1113L/t+aBW9kmjky5imNdIW8Nb9sZx7BLU444YTm7LPPXq8VJdGuB0f+SAQSgURg9SIwbaJFsuYsBaRgerXxg6UwdRKjXQB+y2IQLU9kJNZOs5qj/dGPflS0Wd7GPIiPOuqo3u0+zeXaaILW61rkfPe7332DmNtJtO3azN+JQCKQCKxSBKZNtEytD37wg8vWdz5t6mH3G+Tk8J0p2XpVc57IzS44XYEhZkG0hIBPfvKThTSVhanbLk2jEhK1HAnROszpcvqqUxJtjUZ+TwQSgURgFSMwTaK1FOa4444rS184Cwm1eNe73rUc1pzGwUyL1GizPkWJ6kqzIFpmYxo1wlQWWrhdm8YlZQ5Bwb2iXtUpibZGI78nAolAIrCKEZgW0XJ0svONpToI9Zvf/GbvwbGoJlrxj7vSLIjW/PHuu+9eyoMwfZ9kx6Yk2q4ay3OJQCKQCCQCGyAwLaL97W9/2+y5557NLrvsUnbn4UTUdwhagdRotD7nSbSf/vSnm6233roQLQ31Qx/60AZrYjcArWmKFq7scaRG24VSnksEEoFEIBFopkW04fxE02sHb2jDzDQbRCumMHLrSkNrtOZnadcRRhHhEgJiGVJXmZw788wzS5QrWjlytv72u9/97nqXp+l4PTjyRyKQCCQCqxeBjSVawR7e9773NQLqH3bYYZ3ewzW6vJI/85nPFKJFVDTgb3/72/Ul674PTbTCKobZGGEKwciZa1zizGUNLWHBetrHPOYxzVlnnbXebUm068GRPxKBRCARWL0IbAzRWjsqsINt78QGFgdYbONRCSHtu+++60yuNGGE15WGJlrltS0ewqfVPulJT2p+9atfdRWlnKOpn3baac3DHvaw5gIXuEBZnsRL2Xx022s6ibYXxvwjEUgEEoHVhcByiVY4xY9+9KMl0IPADSeeeGLzhz/8YSx4AvPbkzbmNudFtDRru/Vsu+22hWit6bXFXTvwhBcy1yw+s+39dtttt2a77bYr62e32Wab5vDDD99gnbB7kmjHNoW8IBFIBBKB1YHAUomWqRihWh9rKQyTq4PH8Thtltb3pS99qbntbW+7jmjveMc7rgvKL+86DaXR0kx5FiN5hE+jRbRM2kzHHLsIEiJYWR/LtH3wwQeXmM3CNJpXFr8ZyQo1iYjbKYm2jUj+TgQSgURglSKwFKJFpNadWiuLLC94wQuWgxnV/CwTcB1qMSB1H01RlKgnPvGJJSiEexxMr/aqFSRC3jVZT5NokSEt1lpfDlAHHXRQ2XNWGWjkPq39vde97tUcc8wxzete97rm0EMPba53ves1O+20U4lvbN9cYSWFjxR8A8n2pSTaPmTyfCKQCCQCqwyBpRDt9773vebAAw8s5lPxiR3WzTp23nnnouEhy3aiFdpgQIxg19X3+e4ck6y8PSPSNIlWvGLezTRTZBrlj7IQHOJd4lz8FvuY5m0++vTTTy/abpcWG+X2mURbo5HfE4FEIBFYxQgshWhphEyqv/nNb4rTEMeh+rBBetv8C9rQJJlk6+vr7/KUt2sjTZNoESPT9ajy1+WpvyNNB62cxj1u+Y/yJ9FGLeZnIpAIJAKrHIGlEO2soZom0c667Em0s0Y8n5cIJAKJwIIikEQ7TMUk0Q6Da+aaCCQCicCKQyCJdpgqS6IdBtfMNRFIBBKBFYdAEu0wVZZEOwyumWsikAgkAisOgSTaYaosiXYYXDPXRCARSARWHAJJtMNUWRLtMLhmrolAIpAIrDgEkmiHqbIk2mFwzVwTgUQgEVhxCCTRDlNlSbTD4Jq5JgKJQCKw4hBIoh2mypJoh8E1c00EEoFEYMUhkEQ7TJUl0Q6Da+aaCCQCicCKQyCJdpgqS6IdBtfMNRFIBBKBFYdAEu0wVZZEOwyumWsikAgkAisOgTPPPLPZa6+9ylZx9l+NdOqppzZ3uMMdyrZwcW7Wn7ak22WXXRpliaSMtrVTZkKCZNu+y1/+8s3atWvjsvK57777NjvuuGPZtWe9P2bwA9GuWbOmbL9n96JI9sG18b0t9+yLO3TaZOgHZP6JQCKQCCQCoxEw2H/jG98o28fZfSfSH//4x+ZjH/vYBuQV/8/iE3Eqg7JEUkZb3SlzEJXv9rQlNNTJnrEvfelLm7POOqs+PZPvdgo67bTTmgMOOKBsIB8PtR/uV77yleaII45Yb+/d+H/an0m000Y080sEEoFEYIkI2PLtb3/7W9l0vd603XekQDObV/JsxNoulz1vkWxsV+e7bfbqTeOVGUHbnL1r676h38m2fPbARfJ1uZTZdns083F72k6jjEm000Ax80gEEoFEIBFIBHoQSKLtASZPJwKJQCKQCCQC00AgiXYaKGYeiUAikAgkAolADwJJtD3A5OlEIBFIBBKBRGAaCCTRTgPFzCMRSAQSgUQgEehBIIm2B5g8nQgkAolAIpAITAOBJNppoJh5JAKJQCKQCCQCPQgk0fYAk6cTgUQgEUgEEoFpIJBEOw0UM49EIBFIBBKBRKAHgSTaHmDydCKQCCQCiUAiMA0EkmingWLmkQgkAolAIpAI9CCQRNsDTJ5OBBKBRCARSASmgUAS7TRQzDwSgUQgEUgEEoEeBJJoe4DJ04lAIpAIJAKJwDQQSKKdBoqZRyKQCCQCiUAi0INAEm0PMHk6EUgEEoFEIBGYBgL/B19rSiwxjM9EAAAAAElFTkSuQmCC"
    }
   },
   "cell_type": "markdown",
   "id": "concrete-phase",
   "metadata": {},
   "source": [
    "Cosine Similarity is worked out using the below formula. 'A' being the query vector, and 'B' being the document vector\n",
    "\n",
    "![image.png](attachment:870ac3bd-3b8b-4ed7-a9ef-5022fdd7dd75.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "insured-ethics",
   "metadata": {},
   "outputs": [],
   "source": [
    "def cosine_sim(a, b):\n",
    "    cos_sim = np.dot(a, b)/(np.linalg.norm(a)*np.linalg.norm(b))\n",
    "    return cos_sim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "fuzzy-publisher",
   "metadata": {},
   "outputs": [],
   "source": [
    "def cosine_similarity(queryFile):\n",
    "    files = len(file_dataset)\n",
    "\n",
    "    #pre-processing the query\n",
    "    query = print_query(queryFile)\n",
    "    tokens = preprocessing(queryFile)\n",
    "    \n",
    "    print(\"Query:\", query)\n",
    "    \n",
    "    d_cosines = []\n",
    "    \n",
    "    query_vector = gen_vector(tokens)\n",
    "    \n",
    "    for d in D:\n",
    "        print(\"query vetor\", query_vector)\n",
    "        print(\"d\", d)\n",
    "        d_cosines.append(cosine_sim(query_vector, d))\n",
    "        \n",
    "    #the sorted list of files by cosine similarty\n",
    "    coSim = np.array(d_cosines).argsort()[-files:][::-1]\n",
    "        \n",
    "    return list(coSim)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "complicated-intranet",
   "metadata": {},
   "outputs": [
    {
     "output_type": "error",
     "ename": "FeatureNotFound",
     "evalue": "Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFeatureNotFound\u001b[0m                           Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-24-9b32002a939f>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mquery\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m\"queries-raw-texts/wes2015.q01.naf\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mQ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcosine_similarity\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mquery\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      3\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"\\nRanked List of Documents:\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mQ\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"\\nBest Match:\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-23-b4c32abc2175>\u001b[0m in \u001b[0;36mcosine_similarity\u001b[1;34m(queryFile)\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m     \u001b[1;31m#pre-processing the query\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m     \u001b[0mquery\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mprint_query\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mqueryFile\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      6\u001b[0m     \u001b[0mtokens\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpreprocessing\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mqueryFile\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-11-430807e69b39>\u001b[0m in \u001b[0;36mprint_query\u001b[1;34m(filename)\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mprint_query\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfilename\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m     \u001b[0mdata\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mread_file\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfilename\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      3\u001b[0m     \u001b[0mdata\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mconvert_Text\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-3-69376cab2abe>\u001b[0m in \u001b[0;36mread_file\u001b[1;34m(fileName)\u001b[0m\n\u001b[0;32m      8\u001b[0m         \u001b[1;31m# Combine the lines in the list into a string\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m         \u001b[0mcontent\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m\"\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcontent\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 10\u001b[1;33m         \u001b[0mbs_content\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbs\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcontent\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"xml\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     11\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     12\u001b[0m     \u001b[1;31m#returns \"bs4.element.Tag\" object\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\bs4\\__init__.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, markup, features, builder, parse_only, from_encoding, exclude_encodings, element_classes, **kwargs)\u001b[0m\n\u001b[0;32m    241\u001b[0m             \u001b[0mbuilder_class\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbuilder_registry\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlookup\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mfeatures\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    242\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mbuilder_class\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 243\u001b[1;33m                 raise FeatureNotFound(\n\u001b[0m\u001b[0;32m    244\u001b[0m                     \u001b[1;34m\"Couldn't find a tree builder with the features you \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    245\u001b[0m                     \u001b[1;34m\"requested: %s. Do you need to install a parser library?\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFeatureNotFound\u001b[0m: Couldn't find a tree builder with the features you requested: xml. Do you need to install a parser library?"
     ]
    }
   ],
   "source": [
    "query = \"queries-raw-texts/wes2015.q01.naf\"\n",
    "Q = cosine_similarity(query)\n",
    "print(\"\\nRanked List of Documents:\")\n",
    "print(Q)\n",
    "print(\"\\nBest Match:\")\n",
    "print_doc(Q[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}