4_Example.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "4_Example.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "B3OkhhW2zo-0"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/titsitits/Python_Data_Science/blob/master/4_Example.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DfQ-d2onvooo",
        "colab_type": "text"
      },
      "source": [
        "Mickaël Tits\n",
        "CETIC\n",
        "mickael.tits@cetic.be"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ug1Bj8_tAVWq",
        "colab_type": "text"
      },
      "source": [
        "# Chapitre 4 - Un exemple concret: analysons quelques bien immobiliers...\n",
        "\n",
        "Le dataset simulé contient des maisons à vendre. Les informations sur ces maisons sont leur adresse, le site de référence (Immoweb ou Immovlan), le prix de vente, la surface du bien, et le nombre de pièces.\n",
        "\n",
        "* Quelle est le \"meilleur\" site de référence ? (immoweb ou immovlan ?)\n",
        "* Les maisons sont-elles plus chères sur un des sites (pour savoir si une plateforme est plus utilisée pour les biens de luxe) ?\n",
        "* Quelle maison a le meilleur ratio rooms/price et surface/price ?\n",
        "* Quelle ville est la plus chère ?\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U-3impyQwfAd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Un inventaire de biens immobiliers\n",
        "\n",
        "houses_dataset = {\"address\":[\"Rue de Fer 26, 5000 Namur\",\n",
        "                             \"Rue de Bruxelles 42, 5000 Namur\",\n",
        "                      \"Porte de Namur 25, Bruxelles\",\n",
        "                      \"Rue de L'Eglise 42, Charleroi\",\n",
        "                      \"Rue Saint-ghislain 30, 6224 Fleurus\",\n",
        "                     \"Rue de la Closière 20, Fleurus\",\n",
        "                     \"Rue de la Closière 20, Fleurus\",\n",
        "                     \"Rue de Fer 25, 5000 Namur\",\n",
        "                     \"Rue du Luxembourg 15, 1000 Bruxelles\",\n",
        "                     \"NaN\",\n",
        "                     \"Rue de Bruxelles 42, 5000 Namur\",\n",
        "                     \"Rue de la Loi 50, Bruxelles\"],\n",
        "          \"website\":[\"immoweb\",\"immoweb\",\"immoweb\",\"immoweb\",\"immoweb\",\"immoweb\",\"immoweb\",\"immovlan\",\"immovlan\",\"immovlan\",\"immovlan\",\"immovlan\"],\n",
        "           \"price\": [400000,\n",
        "                     350000,\n",
        "                     400000,\n",
        "                     150000,\n",
        "                     \"330000\",\n",
        "                     230000,\n",
        "                     230000,\n",
        "                     0,\n",
        "                     -100,\n",
        "                     \"cent mille\",\n",
        "                     350000,\n",
        "                     700000],\n",
        "           \"surface\":[150,\n",
        "                      200,\n",
        "                      120,\n",
        "                      150,\n",
        "                      320,\n",
        "                      175,\n",
        "                      170,\n",
        "                      170,\n",
        "                      100,\n",
        "                      100,\n",
        "                      200,\n",
        "                      220],\n",
        "                 \"rooms\":[4,5,3,5,5,2,3,3,\"two\",0,4,3]}\n",
        "\n",
        "print(houses_dataset)\n",
        "\n",
        "#Note: Pour corser l'exercice, on peut orthographier des addresses identiques différemment (avec ou sans code postal, Rue de la Closière = Rue des Closières,  Boulevard = Bd, Saint = St, ...), pour rendre le nettoyage de données plus compliqué (ce qui est plus proche de la réalité)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M3sYySxyMghV",
        "colab_type": "text"
      },
      "source": [
        "## Représentation des données"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JVKPPOeHLfGw",
        "colab_type": "text"
      },
      "source": [
        "Pour mettre en pratique le principe d'encapsulation, nous allons créer une classe représantant un bien immobilier. Le code est ainsi plus modulaire et pourrait facilement être réutilisé dans un autre projet. \n",
        "\n",
        "Néanmoins, on pourrait tout à fait se passer d'utiliser une classe, et simplament appliquer des fonctions sur les éléments du dictionnaire ou d'une autre représentation des données. Nous verrons en l'occurrence une manière plus pratique de représenter un dataset: le DataFrame.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tV7-AbjP0i2_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class House:\n",
        "  \n",
        "  def __init__(self, address, website, price, surface, rooms):\n",
        "    \n",
        "    self.address = address\n",
        "    self.website = website\n",
        "    self.price = price\n",
        "    self.surface = surface\n",
        "    self.rooms = rooms\n",
        "    \n",
        "    #On vérifie la validité des données, et on les rend valides si possible, sinon on renvoie False (pas valide)\n",
        "    self.is_valid = self.validate()\n",
        "    \n",
        "  def price_m2(self):\n",
        "        \n",
        "    return self.price/self.surface\n",
        "  \n",
        "  \n",
        "  def price_rooms(self):\n",
        "    \n",
        "    return self.price/self.rooms  \n",
        "  \n",
        "  \n",
        "  def get_street(self):\n",
        "    \n",
        "    address = self.address\n",
        "    \n",
        "    #Le nom de la rue est la partie avant le premier nombre (le numéro de la rue).\n",
        "    for i in range(len(address)):\n",
        "      c = address[i]\n",
        "      if c in \"0123456789\":\n",
        "        #c est un nombre, on peut sortir de la boucle\n",
        "        break\n",
        "    \n",
        "    #On extrait le nom de la rue de l'adresse\n",
        "    street = address[:i-1]\n",
        "    \n",
        "    return street\n",
        "  \n",
        "  \n",
        "  def get_city(self):\n",
        "    \n",
        "    address = self.address\n",
        "    \n",
        "    #Le nom de la rue est la partie après le dernier nombre, ou une virgule si aucun code postal n'est renseigné. On cherche donc un nombre en commençant par la fin ( range(len(address),0,-1) )\n",
        "    for i in range(len(address)-1,0,-1):\n",
        "      c = address[i]\n",
        "      if c in \"0123456789,\":\n",
        "        #c est un nombre (ou une virgule), on peut sortir de la boucle\n",
        "        break\n",
        "    \n",
        "    #On extrait le nom de la ville\n",
        "    city = address[i+2:]\n",
        "    \n",
        "    return city  \n",
        "  \n",
        "  \n",
        "  def validate(self):\n",
        "    \n",
        "    \"\"\"\n",
        "    Vérifie si l'objet est valide\n",
        "    \"\"\"    \n",
        "    \n",
        "    #Si le prix, la surface, ou le nombre de chambre ne sont pas des int, on essaye de les convertir, sinon pas valide\n",
        "    try:\n",
        "      self.price = int(self.price)\n",
        "      self.surface = int(self.surface)\n",
        "      self.rooms = int(self.rooms)\n",
        "    except:\n",
        "      return False\n",
        "    \n",
        "    #Si le prix <= 0, pas valide\n",
        "    if self.price <= 0:\n",
        "      return False\n",
        "    \n",
        "    #si l'addresse n'est pas un string ou si c'est \"NaN\", pas valide\n",
        "    adr = self.address\n",
        "    if (type(adr) is not str) or (adr == \"NaN\"):\n",
        "      return False\n",
        "    \n",
        "    #si rooms n'est pas un entier, ou si c'est <= 0, pas valide\n",
        "    if type(self.rooms) is not int or self.rooms <= 0:\n",
        "      return False\n",
        "    \n",
        "    #si surface n'est pas un entier, ou si c'est <= 0, pas valide\n",
        "    if type(self.surface) is not int or self.surface <= 0:\n",
        "      return False    \n",
        "    \n",
        "    #Si aucun des \"return\" précédent n'a été réalisé, c'est que l'objet est valide\n",
        "    return True\n",
        "  \n",
        "  \n",
        "  def display(self):\n",
        "    \n",
        "    if self.is_valid:\n",
        "      print(\"%s: %s, %d €, %d m2, %d rooms, %d €/m2, %d €/room (%s)\" % (self.get_street(), self.get_city(), self.price, self.surface, self.rooms, self.price_m2(), self.price_rooms(), self.website) )\n",
        "    else:\n",
        "      print(\"L'élément n'est pas un élément valide:\", self.address)\n",
        "    \n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cPz1Jdb1MQSN",
        "colab_type": "text"
      },
      "source": [
        "Créons maintenant une liste d'objets de type House:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8RU-6Zb_VGv4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Nombre de maisons\n",
        "n = len(houses_dataset[\"address\"])\n",
        "\n",
        "#Pour chaque élément du dataset, on crée un objet de type House avec ses propres attributs\n",
        "houses = [House( houses_dataset[\"address\"][i], houses_dataset[\"website\"][i], houses_dataset[\"price\"][i], houses_dataset[\"surface\"][i], houses_dataset[\"rooms\"][i]  ) for i in range(n)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WYzZqAKmMc09",
        "colab_type": "text"
      },
      "source": [
        "## Correction des données\n",
        "\n",
        "Avant de pouvoir concrètement analyser les données, il est important de traiter le dataset. En effet, des données réelles contiennent souvent des données invalides, manquantes ou non-pertinentes. Dans cet exemple, nous allons corriger ou éliminer les données invalides, et les doublons."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vz-TdVspK364",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for item in houses:\n",
        "  item.display()\n",
        "  \n",
        "#On ne garde que les éléments valides\n",
        "valid_houses = [h for h in houses if h.is_valid]\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-k1cVJqdWhBn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Pour supprimer les doublons, on peut convertir la liste en dictionnaire d'objets House, en prenant comme clé l'adresse. En effet, dans un dictionnaire les clés sont uniques!\n",
        "unique_houses_dict = {h.address:h for h in valid_houses}\n",
        "\n",
        "print(\"Affichage des valeurs du dictionnaire:\")\n",
        "for key in unique_houses_dict:\n",
        "  unique_houses_dict[key].dicplay()\n",
        "\n",
        "\n",
        "#On peut reconvertir les valeurs du dictionnaire en liste:\n",
        "unique_houses = list(unique_houses_dict.values())\n",
        "\n",
        "print(\"Affichage de la liste:\")\n",
        "for item in unique_houses:\n",
        "  item.display()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RAlC6b8lNc6F",
        "colab_type": "text"
      },
      "source": [
        "## Exploration des données"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d6nedl7QQe-3",
        "colab_type": "text"
      },
      "source": [
        "### Quelle est la maison la moins chère au mètre carré ?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P5LnQVXzXBU6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "price_per_m2 = [h.price_m2() for h in unique_houses]\n",
        "#Minimum\n",
        "min_price = min(price_per_m2)\n",
        "#Indice correspondant à ce prix\n",
        "best = price_per_m2.index(min_price)\n",
        "#Maison correspondante\n",
        "best_house = unique_houses[best]\n",
        "\n",
        "best_house.display()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wDc5zn6-QhUj",
        "colab_type": "text"
      },
      "source": [
        "### Quel est le \"meilleur\" site web immobilier ?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2i-Fra6TX4QI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Attention, si on retire la doublons, on crée un biais, puisqu'on a retiré des maisons valides d'une plateforme\n",
        "\n",
        "#si on garde les éléments non-valides:\n",
        "immovlan = [h for h in houses if h.website == \"immovlan\"]\n",
        "immoweb = [h for h in houses if h.website == \"immoweb\"]\n",
        "\n",
        "print(\"Avec les non-valide - immovlan:\", len(immovlan), \"immoweb:\", len(immoweb))\n",
        "\n",
        "#Si on ne garde que les éléments valides\n",
        "immovlan = [h for h in valid_houses if h.website == \"immovlan\"]\n",
        "immoweb = [h for h in valid_houses if h.website == \"immoweb\"]\n",
        "\n",
        "print(\"Elements valides - immovlan:\", len(immovlan), \"immoweb:\", len(immoweb))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NSXEkSBiQuY0",
        "colab_type": "text"
      },
      "source": [
        "Si le dataset était réel et suffisamment grand pour extraire des statistiques fiables, on en déduirait peut-être que immoweb est plus gros et de meilleure qualité."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UyPlZhw1Q0aE",
        "colab_type": "text"
      },
      "source": [
        "### Quelle est la plateforme la plus branchée maisons de luxe ?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y5l34-3-cnaH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def meanprice(house_list):\n",
        "  \n",
        "  prices = [h.price for h in house_list]\n",
        "  return int(sum(prices)/len(prices))\n",
        "\n",
        "price_immovlan = meanprice(immovlan)\n",
        "price_immoweb = meanprice(immoweb)\n",
        "\n",
        "print(\"immovlan:\", price_immovlan)\n",
        "print(\"immoweb:\", price_immoweb)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HUDqy-9HQxqL",
        "colab_type": "text"
      },
      "source": [
        "Si le dataset était réel et suffisamment grand pour extraire des statistiques fiables, on en déduirait peut-être que immovlan propose des maisons généralement plus chères que immoweb."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SBuIkN36Q2fb",
        "colab_type": "text"
      },
      "source": [
        "### Quelle est la ville la plus chère ?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mQnURGvhcPy4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Il faut d'abord extraire la liste des villes\n",
        "cities = [h.get_city() for h in unique_houses]\n",
        "\n",
        "#Pour extraire simplement les villes unique, on peut utiliser une autre collection d'objets pas encore vue: le \"set\". C'est une collection non-ordonnée d'éléments uniques. \n",
        "#Remarque: ca n'aurait pas fonctionné sur la classe House puisqu'on ne sait pas directement utiliser d'opérateur de test d'égalité sur les objets de type House\n",
        "cities = list(set(cities))\n",
        "print(cities)\n",
        "\n",
        "#Pour chaque ville, on extrait la liste des maisons. On obtient donc une liste de listes!\n",
        "cities_houses = [[h for h in unique_houses if h.get_city() == c] for c in cities]\n",
        "\n",
        "#On calcule ensuite le prix moyen pour chaque liste:\n",
        "cities_prices = [meanprice(l) for l in cities_houses]\n",
        "\n",
        "print(cities_prices)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8mq2xuDLQ7BT",
        "colab_type": "text"
      },
      "source": [
        "Si le dataset était réel et suffisamment grand pour extraire des statistiques fiables, on en déduirait peut-être que Bruxelles est la ville la plus chère."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tF2zEbteRMcT",
        "colab_type": "text"
      },
      "source": [
        "Dans le Chapitre suivant, nous allons découvrir différentes librairies Python, qui nous permettront d'analyser bien plus efficacement un ensemble de données. C'est par ici: [Chapitre 5: Les librairies Python pour l'analyse de données](https://colab.research.google.com/github/titsitits/UNamur_Python_Analytics/blob/master/5_Python_packages.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B3OkhhW2zo-0",
        "colab_type": "text"
      },
      "source": [
        "# Bonus - Exemple 2: un peu nutrition et de Natural Language Processing (NLP)\n",
        "Dans cet exemple, nous avons une collection de denrées alimentaires, et quelques informations sur ces aliments. Chaque aliment a le même type de propriété (un nom, un poids, des valeurs énergétiques)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HSTP6ffZzrY3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "#extract singular versions of words in text (defaults plurals and exceptions are used, but you can override default rules)\n",
        "def singulize(string, pluriels = None, exceptions = None):\n",
        "  \n",
        "  \"\"\"\n",
        "  Extract singular versions of words in text (defaults plurals and exceptions are used, but you can override default rules)\n",
        "  Default plurals: {'ois':'ois','s':'','eaux':'eau','aux':'al','x':''}\n",
        "  Default exceptions: {'os':'os','chacals':'chacal','souris':'souris','rabais':'rabais','prix':'prix', 'taux':'taux','rhinoceros':'rhinoceros','jus':'jus','noix':'noix','mais':'mais'}\n",
        "\n",
        "  \"\"\"\n",
        "\n",
        "  #singulize (french ad-hoc method...)\n",
        "  #règles du pluriel\n",
        "  if pluriels is None:\n",
        "      pluriels = {'ois':'ois','s':'','eaux':'eau','aux':'al','x':''}\n",
        "  #exceptions\n",
        "  if exceptions is None:\n",
        "      exceptions = {'os':'os','chacals':'chacal','souris':'souris','rabais':'rabais','prix':'prix', 'taux':'taux','rhinoceros':'rhinoceros','jus':'jus','noix':'noix','mais':'mais', 'chips':'chips'}\n",
        "\n",
        "  #Une fonction dans une fonction! Pourquoi diable ? Elle n'est pas appelée ailleurs que dans la fonction singulize. Par soucis de clarté, on la déclare donc uniquement dans la portée de cette fonction.\n",
        "  def singulize_word(word):\n",
        "\n",
        "      if word in exceptions:\n",
        "\n",
        "          return exceptions[word]\n",
        "\n",
        "      isplural = [word.endswith(k) for k in pluriels]\n",
        "\n",
        "      if any(isplural):\n",
        "\n",
        "          #take first key\n",
        "          keyid = isplural.index(True)\n",
        "          key = list(pluriels)[keyid]\n",
        "\n",
        "          #to replace last occurence, reverse all strings and replace first occurrence\n",
        "          return word[::-1].replace(key[::-1], pluriels[key][::-1], 1)[::-1]\n",
        "\n",
        "      return word\n",
        "\n",
        "  return ' '.join([singulize_word(word) for word in string.split()])\n",
        "\n",
        "\n",
        "\n",
        "#remove stopwords from text (defaults stopwords are used but you can override default rules)\n",
        "def remove_stopwords(string, stopwords = None):\n",
        "      \n",
        "  \"\"\"\n",
        "  Remove stopwords from text (defaults stopwords are used but you can override default rules)\n",
        "  Default stopwords: ['de','du','le','les','aux','la','des', 'a', 'une', 'un', 'au','d','l']\n",
        "  \"\"\"\n",
        "  #remove stopwords\n",
        "  if stopwords is None:\n",
        "      stopwords = ['de','du','le','les','aux','la','des', 'a', 'à', 'une', 'un', 'au','d','l','et']\n",
        "\n",
        "  string = string.replace(\"d'\",\"\")\n",
        "  string = string.replace(\"l'\",\"\")        \n",
        "  return ' '.join([word for word in string.split() if word not in (stopwords)])\n",
        "\n",
        "\n",
        "\n",
        "#Un petit test\n",
        "sentence1 = \"cake à la banane noix de cajou\"\n",
        "sentence2 = \"cakes aux bananes et aux noix de cajou\"\n",
        "\n",
        "s1 = singulize( remove_stopwords(sentence1) )\n",
        "s2 = singulize( remove_stopwords(sentence2) )\n",
        "\n",
        "\n",
        "\n",
        "print(s1)\n",
        "print(s2)\n",
        "\n",
        "if s1 == s2:\n",
        "  print(\"Les phrases sont identiques\")\n",
        "  \n",
        "\n",
        "  \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UZMWGhdiZcWw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Pour éviter de redéfinir à chaque fois toutes les règles de traitement (pluriels, stopwords, exceptions) quand on appelle une fonction, on peut à la place créer une classe et définir les règles comme des attributs de cette classe.\n",
        "\n",
        "\n",
        "class NLP:\n",
        "  \n",
        "  def __init__(self, plurals, stopwords, exceptions):\n",
        "    \n",
        "    self.pluriels = plurals\n",
        "    self.stopwords = stopwords\n",
        "    self.exceptions = exceptions\n",
        "    \n",
        "  def singulize(self, string):\n",
        "    \n",
        "    \"\"\"\n",
        "    Extract singular versions of words in text\n",
        "    \"\"\"    \n",
        "    \n",
        "    #Une fonction dans une fonction! Pourquoi diable ? Elle n'est pas appelée ailleurs que dans la fonction singulize. Par soucis de clarté, on la déclare donc uniquement dans la portée de cette fonction.\n",
        "    def singulize_word(word):\n",
        "\n",
        "        if word in self.exceptions:\n",
        "\n",
        "            return self.exceptions[word]\n",
        "\n",
        "        isplural = [word.endswith(k) for k in self.pluriels]\n",
        "\n",
        "        if any(isplural):\n",
        "\n",
        "            #take first key\n",
        "            keyid = isplural.index(True)\n",
        "            key = list(self.pluriels)[keyid]\n",
        "\n",
        "            #to replace last occurence, reverse all strings and replace first occurrence\n",
        "            return word[::-1].replace(key[::-1], self.pluriels[key][::-1], 1)[::-1]\n",
        "\n",
        "        return word\n",
        "\n",
        "    return ' '.join([singulize_word(word) for word in string.split()])\n",
        "  \n",
        "  def remove_stopwords(self, string):\n",
        "\n",
        "    \"\"\"\n",
        "    Remove stopwords from text\n",
        "    \"\"\"\n",
        "\n",
        "    string = string.replace(\"d'\",\"\")\n",
        "    string = string.replace(\"l'\",\"\")        \n",
        "    return ' '.join([word for word in string.split() if word not in (stopwords)])\n",
        "  \n",
        "  def simplify(self, string):\n",
        "    \n",
        "    string = self.remove_stopwords(string)\n",
        "    return self.singulize(string)\n",
        "\n",
        "\n",
        "#Définition unique des règles  \n",
        "pluriels = {'ois':'ois','s':'','eaux':'eau','aux':'al','x':''}\n",
        "exceptions = {'os':'os','chacals':'chacal','souris':'souris','rabais':'rabais','prix':'prix', 'taux':'taux','rhinoceros':'rhinoceros','jus':'jus','noix':'noix','mais':'mais', 'chips':'chips'}\n",
        "stopwords = ['de','du','le','les','aux','la','des', 'a', 'à', 'une', 'un', 'au','d','l','et']\n",
        "\n",
        "textprocessor = NLP(pluriels, stopwords, exceptions)\n",
        "\n",
        "print(textprocessor.pluriels)\n",
        "\n",
        "\n",
        "#Test (le résultat devrait être le même)\n",
        "s1 = textprocessor.simplify(sentence1)\n",
        "s2 = textprocessor.simplify(sentence2)\n",
        "\n",
        "print(s1)\n",
        "print(s2)\n",
        "\n",
        "if s1 == s2:\n",
        "  print(\"Les phrases sont identiques\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LC9y2TzqckJ4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Food:\n",
        "  \n",
        "  def __init__(self, nom, poids, calories_per_100g):\n",
        "    \n",
        "    self.nom = nom\n",
        "    self.poids = poids\n",
        "    self.cal = calories_per_100g\n",
        "    \n",
        "    self.simplify_name()\n",
        "\n",
        "  def simplify_name(self):\n",
        "    \n",
        "    self.nom = singulize( remove_stopwords(self.nom) )\n",
        "    self.nom = self.nom.capitalize()\n",
        "    \n",
        "  def totcal(self):\n",
        "    \n",
        "    return self.poids*self.cal/100\n",
        "  \n",
        "  def display(self):\n",
        "    \n",
        "    print(\"%s: %d cal/100g, %d g, calories totales: %d\" % (self.nom, self.cal, self.poids, self.totcal()) )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K0g1G58afU3e",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Un inventaire d'aliments\n",
        "\n",
        "dataset = {\"noms\":[\"des pommes\",\"chips\",sentence1, sentence2, \"pomme\"], \n",
        "           \"poids\": [150, 120, 75, 80, 145], \n",
        "           \"calories_per_100g\":[52, 536, 320, 310, 53]}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IzRWmR8XDHd6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "\n",
        "n = len(dataset[\"noms\"])\n",
        "\n",
        "#Je peut utiliser la classe Food pour représenter mes aliments de manière structurée:\n",
        "aliments = [Food(dataset[\"noms\"][i], dataset[\"poids\"][i], dataset[\"calories_per_100g\"][i]) for i in range(0,n)]\n",
        "\n",
        "for item in aliments:\n",
        "  item.display()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uhiqwYs9eqCJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Lors de la création d'un aliment, son nom est déjà traité (voir la méthode __init__). La liste contient donc deux aliments qui ont le même nom: \"Pomme\"\n",
        "#retirons les duplicatas\n",
        "#On peut pour cela passer par un dictionnaire: chaque clé doit être unique. On utilise donc le nom de l'aliment comme clé:\n",
        "aliments_dict = {item.nom:item for item in aliments}\n",
        "unique_aliments = list(aliments_dict.values())\n",
        "\n",
        "for item in unique_aliments:\n",
        "  item.display()\n",
        "\n",
        "\n",
        "def find_lightest(foods):\n",
        "  \n",
        "  \"\"\"\n",
        "  Arguments d'entrée:\n",
        "  foods: liste d'aliments (objets de la classe Food)\n",
        "  \n",
        "  Résultats:\n",
        "  un tuple contanent le meilleur aliment, et son indice dans la liste\n",
        "  \"\"\"\n",
        "\n",
        "  allcals = [item.totcal() for item in aliments]\n",
        "  mincals = min(allcals)\n",
        "  best = allcals.index(mincals)\n",
        "  \n",
        "  return (aliments[best], best)\n",
        "\n",
        "meilleur_aliment = find_lightest(aliments)[0]\n",
        "\n",
        "print(\"Aliment le plus léger:\", meilleur_aliment.nom)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xCmVa8NnJmVy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}