hacking_texts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logos](https://i.ytimg.com/vi/a3tl5jcvGIU/maxresdefault.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Processamento de Linguagem Natural (em inglês Natural Language Processing)** é a parte central de nossas vidas no dia a dia e pode ser usado para resolver problemas relacionados a linguagem. O Processamento de Linguagem Natural (PLN daqui em diante) é um campo da ciência da computação, inteligência artificial e computação linguística cujo foco está na interação entre computadores e linguagem (natural) humana. <p>São muitos os desafios envolvidos em fazer o computador compreender o significado na geração de linguagem natural. Essencialmente, PLN é a aplicação de aspectos teóricos da linguagem a aplicações de computador.<p>\n",
    "    [Fonte](http://datascienceacademy.com.br/blog/processamento-de-linguagem-natural-python-nltk/)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#http://minerandodados.com.br/index.php/2017/06/22/mineracao-de-textos-7-tecnicas/\n",
    "#http://datascienceacademy.com.br/blog/processamento-de-linguagem-natural-python-nltk/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting wordcloud\n",
      "  Downloading https://files.pythonhosted.org/packages/23/4e/1254d26ce5d36facdcbb5820e7e434328aed68e99938c75c9d4e2fee5efb/wordcloud-1.5.0-cp37-cp37m-win_amd64.whl (153kB)\n",
      "Requirement already satisfied: numpy>=1.6.1 in c:\\programdata\\anaconda3\\lib\\site-packages (from wordcloud) (1.15.4)\n",
      "Requirement already satisfied: pillow in c:\\programdata\\anaconda3\\lib\\site-packages (from wordcloud) (5.3.0)\n",
      "Installing collected packages: wordcloud\n",
      "Successfully installed wordcloud-1.5.0\n"
     ]
    }
   ],
   "source": [
    "!pip install wordcloud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# iniciando como sempre ... com o import das bibliotecas que usaremos\n",
    "import nltk\n",
    "import re\n",
    "from nltk.corpus import wordnet\n",
    "from wordcloud import WordCloud"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Operações com Python**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Python eh uma linguagem fenomenal'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase1 = 'Python eh uma linguagem fenomenal'; frase1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' para qualquer cientista de dados!'"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase2 = ' para qualquer cientista de dados!';frase2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' Além do aprendizado ser simples, é fácil.'"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase3 = ' Além do aprendizado ser simples, é fácil.';frase3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Python eh uma linguagem fenomenal para qualquer cientista de dados! Além do aprendizado ser simples, é fácil.'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase1 + frase2 + frase3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' para qualquer '"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase2[0:15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Além do aprendizado'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase3[1:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'linguagem fenomenal'"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase1[14:35]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Objeto String**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "33"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(frase1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(frase2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' para qualquer cientista de dados!'"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase2 = frase2.lower()\n",
    "frase2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(frase3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' PARA QUALQUER CIENTISTA DE DADOS!'"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase3 = frase2.upper()\n",
    "frase3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Varrer STRINGS**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase1.index('linguagem')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'linguagem fenomenal'"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase1[14:35]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logos](https://upload.wikimedia.org/wikipedia/commons/thumb/9/92/Metallica_Live_at_The_O2%2C_London%2C_England%2C_22_October_2017.jpg/800px-Metallica_Live_at_The_O2%2C_London%2C_England%2C_22_October_2017.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "banda = ['James Hetfield','Lars Ulrich'] + ['Kirk Hammett','Robert Trujillo']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['James Hetfield', 'Lars Ulrich', 'Kirk Hammett', 'Robert Trujillo']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "banda"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "musicas = ['Enter Sandman','Nothing Else Matters','Master of Puppets'] + ['Fight Fire With Fire']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Enter Sandman',\n",
       " 'Nothing Else Matters',\n",
       " 'Master of Puppets',\n",
       " 'Fight Fire With Fire']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "musicas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Join**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hot dog'"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lista = ['hot','dog']\n",
    "' '.join(lista)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Trabalhando com a NLTK**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logos](https://s3-ap-south-1.amazonaws.com/av-blog-media/wp-content/uploads/2019/01/NLP_Abstract_BG_1200X800_02.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers\\punkt.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Iron', 'Maiden', 'eh', 'uma', 'Banda', 'Incrível']"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frase = 'Iron Maiden eh uma Banda Incrível'\n",
    "palavras = nltk.word_tokenize(frase); palavras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FreqDist({'six': 3, 'Iron Maiden': 2, 'the': 1, 'Best': 1, 'forever': 1})"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from nltk import FreqDist\n",
    "frequencia = FreqDist(['six','six','six','Iron Maiden','the','Best','Iron Maiden','forever'])\n",
    "frequencia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_values([3, 2, 1, 1, 1])\n"
     ]
    }
   ],
   "source": [
    "print (frequencia.values())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'six'"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frequencia.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEICAYAAACwDehOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGxJJREFUeJzt3Xm0HHWd/vH3QxJCWDOSK0K2S0xwBpE1bAPORFkEBgWPMOKMI6JM3BhEGY7AeCDkNx70JwPIwAgBQVbZZQLCsAwEUNmSEJYQwQhIAhm9BLJBJCZ+5o/63pqi031v5ebWreTmeZ3T59by7epP9e3up+tb1VWKCMzMzAA2qrsAMzNbdzgUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51CwdZqkZZLG9NKyQtLY3lhWb5A0TdLxddfRVyTtJ+nX6X96ZN31WHMD6y7A1o6kl4FtgFWFyTtExGv1VNS7ImLzumuwXjMZuDAiflB3IdaatxT6h49HxOaF22qBIMlfADYAyqyr7+vRwOxmM9bxujco/if0U5LaU3fJFyW9Atyfpu8j6ZeSFkl6StKEwn22l/SgpKWS7pV0oaRr0rwJkuY3PMbLkg5MwxtJOlXSbyQtlHSjpPc01HKspFckvS7pXwrLGSDp9HTfpZJmSBqZ5uVdPpL+RtKTkpZImidpUjfPwSmSFkh6TdIXGua1XJakTSRdk9ZjkaQnJG3T4jFelnSapOckvSnpCkmbpHl/JukOSR1p3h2SRrRYzvsl3Z8e83VJ10oamuadKunmhvY/kHRBGp4m6TuSfgG8DYyRdJykOen5fFHSlwr3HZZqWSTpDUkPt/pATs//l1O3z5uSLpKk7mpuspzfAGOA21P30eDerFvSdpJuSc/1S5JObFaHlRARvq3HN+Bl4MAm09uBAK4CNgOGAMOBhcBhZF8IDkrjbek+jwDnAoOBvwKWAtekeROA+a0eGzgJeBQYke5/CfCThlouTXXsArwD/EWafwrwDPABQGn+1mleAGMLNXwo1b4z8DvgyBbPyyFp/k5p/a8ruyzgS8DtwKbAAGAPYMsunv9ngZHAe4BfAP+a5m0NfCotZwvgJuC2wn2nAcen4bHp/zEYaAMeAs5P80aTfWhumcYHAAuAfQrLeQX4IFmX8CDgb4D3p+fzr9P9d0/tzwYuTu0GAR8G1GL9ArgDGAqMAjqAQ7qrucxrtbfqTv/DGcAZwMZk4fMi8LG635/r4632Anxby39g9kZbBixKt9vS9Pb0hh5TaPst4OqG+98NHJve8CuBzQrzrqN8KMwBDijM2xb4Y3qzd9YyojD/ceCYNPw8cESL9cs/yJvMOx84r8W8y4HvFsZ3KLss4AvAL4GdSz7/Xy6MHwb8pkXbXYE3C+PTSKHQpO2RwJOF8Z8Dn0vDBxUfIy1ncjd13gZ8PQ1PBv6z1XPR5PnfvzB+I3BqmZpbPFeNobDWdQN7A680TDsNuKKn76sN+ebuo/7hyIgYmm6NR3XMKwyPBo5Om9+LJC0C9if7AN+O7APrrUL7365BDaOBnxaWO4ds53ex2+V/CsNvA507kUcCv+nuASTtLemB1EWwGPgyMKxF8+1497q/a126WdbVZGF5fep6+v+SBnVRWuPjbJceY1NJl0j6raQlZN+kh0oa0GTd3ivpekmvprbXNKzbdcBn0vDfpfFWNSDpUEmPpm6WRWRh1bm87wNzgXtSF82pXawbtPi/lai5jN6oezSwXcPr+nTe/dqzkhwK/V/xNLjzyLYUhhZum0XEd8m6I/5M0maF9qMKw2+RdYMA2X4Asi6D4rIPbVj2JhHxaoka55F1GXTnOmAqMDIitiLrSlCLtgvIwqbTqIb5LZcVEX+MiLMiYkfgL4HDgc91UVfj43Tu6D+ZrEts74jYkqxLjhY1n032v9o5tf1sQ7ubgAlpn8QnWT0U8v+zpMHALcA5wDYRMRS4s7B+SyPi5IgYA3wc+KakA7pYv1a6q7mM3qh7HvBSw2tvi4g4rAfrtMFzKGxYrgE+LuljynbubqJsB/KIiPgtMB04S9LGkvYne+N1egHYRNkO2kHAt8n6kjtdDHxH0mgASW2SjihZ12XA/5M0TpmdJW3dpN0WwBsR8QdJe5F9Y27lRuDzknaUtClwZtllSfqIpA+l4FtC1g22ita+JmmEsh3rpwM3FB5jObAozWusobGeZantcLL9LLmI6CDrbrmC7ANwThfL2pjsf9MBrJR0KHBwYf0OlzQ27TBektatq/XrUc090NO6HweWSPqWpCHptb2TpD3Xsp4NkkNhAxIR84AjyD64Osi+YZ3C/70O/o6sf/YNsg+wqwr3XQx8lewD/FWyLYfi0Ug/IPvmfY+kpWQ7nfcuWdq5ZB/i95C92X9EtkO60VeByWn5Z6T7tFrXu8j2E9xP1uVw/xos633AzamWOcCDZIHaynWp9hfT7V/T9PPTerxO9nz8VxfLOAvYHVgM/Ay4tcXjHMjqWwnvEhFLgRPTOr1J9n+dWmgyDriP7AP9EeA/ImJaV8tci5pL62ndEbGK7AvMrsBLZM/3ZcBWa1PPhkoRvsiONafsMM2xEfHZumtZVyn78eDxEXFf3bWY9QZvKZiZWc6hYGZmOXcfmZlZzlsKZmaWW+9OkjZs2LBob2+vuwwzs/XKjBkzXo+Itu7arXeh0N7ezvTp0+suw8xsvSKp1BkK3H1kZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUqC4V0WubHlV0HeLaks5q0GSzpBklzJT0mqb2qeszMrHtVbim8A3w0InYhO6XtIZL2aWjzRbKrfY0FzgO+V2E9ZmbWjcpCITLL0mjnhbYbT7R0BHBlGr4ZOCBdQMPMzGpQ6S+a05WrZgBjgYsi4rGGJsNJ12iNiJXpWrlbk10ko7icicBEgFGjGq+qWN55977Q4/uua75x0A51l2Bm/VClO5ojYlVE7AqMAPaStFNDk2ZbBaudtjUipkTE+IgY39bW7ak7zMysh/rk6KOIWER2fdlDGmbNJ130XNJAssvnvdEXNZmZ2eqqPPqoTdLQNDyE7Nqyv2poNhU4Ng0fBdwfvsCDmVltqtynsC1wZdqvsBFwY0TcIWkyMD0ippJdoP1qSXPJthCOqbAeMzPrRmWhEBFPA7s1mX5GYfgPwNFV1WBmZmvGv2g2M7OcQ8HMzHIOBTMzyzkUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51AwM7OcQ8HMzHIOBTMzyzkUzMws51AwM7NcZaEgaaSkByTNkTRb0tebtJkgabGkWel2RlX1mJlZ9wZWuOyVwMkRMVPSFsAMSfdGxHMN7R6OiMMrrMPMzEqqbEshIhZExMw0vBSYAwyv6vHMzGzt9ck+BUntwG7AY01m7yvpKUl3Sfpgi/tPlDRd0vSOjo4KKzUz27BVHgqSNgduAU6KiCUNs2cCoyNiF+DfgduaLSMipkTE+IgY39bWVm3BZmYbsEpDQdIgskC4NiJubZwfEUsiYlkavhMYJGlYlTWZmVlrVR59JOBHwJyIOLdFm/eldkjaK9WzsKqazMysa1UefbQf8A/AM5JmpWmnA6MAIuJi4CjgK5JWAsuBYyIiKqzJzMy6UFkoRMTPAXXT5kLgwqpqMDOzNeNfNJuZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmucpCQdJISQ9ImiNptqSvN2kjSRdImivpaUm7V1WPmZl1b2CFy14JnBwRMyVtAcyQdG9EPFdocygwLt32Bn6Y/pqZWQ0q21KIiAURMTMNLwXmAMMbmh0BXBWZR4GhkratqiYzM+talVsKOUntwG7AYw2zhgPzCuPz07QFDfefCEwEGDVqVFVl9nvn3ftC3SX0mm8ctEPdJZj1S5XvaJa0OXALcFJELGmc3eQusdqEiCkRMT4ixre1tVVRppmZUXEoSBpEFgjXRsStTZrMB0YWxkcAr1VZk5mZtVbl0UcCfgTMiYhzWzSbCnwuHYW0D7A4Iha0aGtmZhWrcp/CfsA/AM9ImpWmnQ6MAoiIi4E7gcOAucDbwHEV1mNmZt0oFQqSxgFnAzsCm3ROj4gxre4TET+n+T6DYpsAvlaqUjMzq1zZ7qMryH5DsBL4CHAVcHVVRZmZWT3KhsKQiPhvQBHx24iYBHy0urLMzKwOZfcp/EHSRsCvJZ0AvAq8t7qyzMysDmW3FE4CNgVOBPYg24F8bFVFmZlZPUptKUTEE2lwGT5CyMys3+oyFCSdHxEnSbqd5r80/kRllZmZWZ/rbkuh8wijc6ouxMzM6tdlKETEjDQ4HVgeEX8CkDQAGFxxbWZm1sfK7mj+b7IdzZ2GAPf1fjlmZlansqGwSUQs6xxJw5t20d7MzNZDZUPhreKlMiXtASyvpiQzM6tL2R+vnQTcJKnztNbbAp+upiQzM6tL6d8pSPpz4ANkJ7n7VUT8sdLKzMysz63JqbP3BNrTfXaTRERcVUlVZmZWi7Knzr4aeD8wC1iVJgfZ2VLNzKyfKLulMB7YMV3/wMzM+qmyRx89C7yvykLMzKx+ZbcUhgHPSXoceKdzos99ZGbWv5QNhUlVFmFmZuuGsoekPihpNDAuIu6TtCkwoNrSzMysr5XapyDpH4GbgUvSpOHAbVUVZWZm9Si7o/lrwH7AEoCI+DW+HKeZWb9TNhTeiYgVnSOSBtLkojtmZrZ+KxsKD0o6HRgi6SDgJuD26soyM7M6lA2FU4EO4BngS8CdwLerKsrMzOpR9uijPwGXplspki4HDgd+HxE7NZk/AfhP4KU06daImFx2+WZm1vvKnvvoJZrsQ4iIMV3c7cfAhXR9fqSHI+LwMjWYmVn11uTcR502AY4G3tPVHSLiIUntPSvLzMzqUGqfQkQsLNxejYjzgY/2wuPvK+kpSXdJ+mCrRpImSpouaXpHR0cvPKyZmTVTtvto98LoRmRbDlus5WPPBEZHxDJJh5H9GG5cs4YRMQWYAjB+/HgfCmtmVpGy3Uf/VhheCbwM/O3aPHBELCkM3ynpPyQNi4jX12a5ZmbWc2WPPvpIbz+wpPcBv4uIkLQX2RbIwt5+HDMzK69s99E3u5ofEec2uc9PgAnAMEnzgTOBQan9xcBRwFckrQSWA8f4Ij5mZvVak6OP9gSmpvGPAw8B81rdISI+09UCI+JCskNWzcxsHbEmF9nZPSKWAkiaBNwUEcdXVZiZmfW9sqe5GAWsKIyvANp7vRozM6tV2S2Fq4HHJf2U7JfNn6TrXyqbmdl6qOzRR9+RdBfw4TTpuIh4srqyzMysDmW7jwA2BZZExA+A+ZK2r6gmMzOrSdnLcZ4JfAs4LU0aBFxTVVFmZlaPslsKnwQ+AbwFEBGvsfanuTAzs3VM2VBYkX5YFgCSNquuJDMzq0vZULhR0iXAUEn/CNzHGlxwx8zM1g9ljz46J12beQnwAeCMiLi30srMzKzPdRsKkgYAd0fEgYCDwMysH+u2+ygiVgFvS9qqD+oxM7Malf1F8x+AZyTdSzoCCSAiTqykKjMzq0XZUPhZupmZWT/WZShIGhURr0TElX1VkJmZ1ae7fQq3dQ5IuqXiWszMrGbdhYIKw2OqLMTMzOrXXShEi2EzM+uHutvRvIukJWRbDEPSMGk8ImLLSqszM7M+1WUoRMSAvirEzMzqtybXUzAzs37OoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZrnKQkHS5ZJ+L+nZFvMl6QJJcyU9LWn3qmoxM7NyqtxS+DFwSBfzDwXGpdtE4IcV1mJmZiVUFgoR8RDwRhdNjgCuisyjZNd/3raqeszMrHtlr6dQheHAvML4/DRtQWNDSRPJtiYYNWpUnxRn/ct5975Qdwm95hsH7bDG9+kv678hrzv0bP3XVJ07mtVkWtOT7kXElIgYHxHj29raKi7LzGzDVWcozAdGFsZHAK/VVIuZmVFvKEwFPpeOQtoHWBwRq3UdmZlZ36lsn4KknwATgGGS5gNnAoMAIuJi4E7gMGAu8DZwXFW1mJlZOZWFQkR8ppv5AXytqsc3M7M15180m5lZzqFgZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUcCmZmlnMomJlZzqFgZmY5h4KZmeUcCmZmlnMomJlZrtJQkHSIpOclzZV0apP5n5fUIWlWuh1fZT1mZta1gVUtWNIA4CLgIGA+8ISkqRHxXEPTGyLihKrqMDOz8qrcUtgLmBsRL0bECuB64IgKH8/MzNZSlaEwHJhXGJ+fpjX6lKSnJd0saWSzBUmaKGm6pOkdHR1V1GpmZlQbCmoyLRrGbwfaI2Jn4D7gymYLiogpETE+Isa3tbX1cplmZtapylCYDxS/+Y8AXis2iIiFEfFOGr0U2KPCeszMrBtVhsITwDhJ20vaGDgGmFpsIGnbwugngDkV1mNmZt2o7OijiFgp6QTgbmAAcHlEzJY0GZgeEVOBEyV9AlgJvAF8vqp6zMyse5WFAkBE3Anc2TDtjMLwacBpVdZgZmbl+RfNZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVnOoWBmZjmHgpmZ5RwKZmaWcyiYmVmu0lCQdIik5yXNlXRqk/mDJd2Q5j8mqb3KeszMrGuVhYKkAcBFwKHAjsBnJO3Y0OyLwJsRMRY4D/heVfWYmVn3qtxS2AuYGxEvRsQK4HrgiIY2RwBXpuGbgQMkqcKazMysCwMrXPZwYF5hfD6wd6s2EbFS0mJga+D1YiNJE4GJaXSZpOcrqbj3DKNhHXrbN6tc+NqpfN1hw15/r/s6aX143Y8u06jKUGj2jT960IaImAJM6Y2i+oKk6RExvu466rAhrzts2Ovvde8f615l99F8YGRhfATwWqs2kgYCWwFvVFiTmZl1ocpQeAIYJ2l7SRsDxwBTG9pMBY5Nw0cB90fEalsKZmbWNyrrPkr7CE4A7gYGAJdHxGxJk4HpETEV+BFwtaS5ZFsIx1RVTx9bb7q6KrAhrzts2Ovvde8H5C/mZmbWyb9oNjOznEPBzMxyDoVeJOmyJr/arpWkZRUtd5KkkDS2MO0baVqXh+a1ep4kfV7ShVXU21skDZX01TQ8QdIddddUNUmrJM2S9JSkmZL+sofLOUnSpr1dX2+QdKKkOZKurbuWujkUelFEHB8Rz9VdR3fSKUh6wzO8++CAo4Bu1399eZ5aGAp8te4i+tjyiNg1InYBTgPO7uFyTgLWyVAg+58eFhF/313DdPh8r+vF9+VacSj0kKTNJP0sfXt6VtKnJU2TNF7SaEm/ljRM0kaSHpZ0cM31TpD0gKTryD7MkfTNVPuzkk5K09rTN6ZLJc2WdI+kIS0Wexvp1CWSxgCLgY7CY/5Q0vS0nLMK06d1bk1IOk7SC5IeBPYrtGmTdIukJ9JtvzR9kqTL0zJelHRibz5PJXwXeL+kWcD3gc0l3SzpV5Ku7TxNi6Q9JD0oaYakuyVt28d1VmVL4M3OEUmnpP/P053/4xbvjROB7YAHJD1QU+1NSboYGANMlXSypNvS+jwqaefUZpKkKZLuAa6SNEDS9wvr/qXU7gZJhxWW/WNJn+qi/Wrvy9pFhG89uAGfAi4tjG8FTAPGp/Hjyc7ndApwSY11Lkt/JwBvAdun8T3IXoSbAZsDs4HdgHZgJbBrancj8Nkmy50E/DNwK7AT8C9kvzkpPgfvSX8HpOk7p/FpwHhgW+AVoA3YGPgFcGFqcx2wfxoeBcwpPO4vgcFkpxZYCAzqw+ezHXi28JwuJvth5kbAI8D+wKBUY1tq92myQ7Jrf932cJ1XAbOAX6X13SNNP5jsUEyl9b8D+Ktm743092VgWN3r02IdX06vp38HzkzTPgrMKrzuZgBD0vhE4NtpeDAwHdge+CRwZZq+MdlpfIZ00X4ChfflunCr8jQX/d0zwDmSvgfcEREPq3Auv4i4TNLRwJeBXWuqsdHjEfFSGt4f+GlEvAUg6Vbgw2Q/KHwpImaldjPIPghbuZ6sC+ljwAHAcYV5f6vsvFUDyQJgR+Dpwvy9gWkR0ZFquAHYIc07ENix8JxuKWmLNPyziHgHeEfS74FtyH4dX4fHI2I+QNp6aAcWkQXlvan+AcCCmurrDcsjYlcASfuSfVPeiSwUDgaeTO02B8YBD9Pw3qih5p7anyzUiIj7JW0taas0b2pELE/DBwM7SzoqjW9Ftu53ARdIGgwcAjwUEctTT0Gz9it49/uydg6FHoqIFyTtARwGnJ02K3Nph9qINLo5sLSPS2zmrcJwV2ejfacwvIrsm04rt5N1o0yPiCWdH+KStifbktgzIt6U9GNgkyb3b/VDmY2AfQtvws7lNquvztdxs1oEzI6IfespqToR8YikYWRbdwLOjohLGts1vjciYnIfl9pTXZ2PrfH9808RcfdqC5CmkX1J+jTwk67aS5rQsNzaeZ9CD0naDng7Iq4BzgF2b2jyPeBa4Azg0j4ur4yHgCMlbSppM7LN3jX+Rpc+tL8FfKdh1pZkL/bFkrYhu65Go8eACenb2CDg6MK8e4ATOkckrStbW0uBLbpp8zzQlr5VI2mQpA9WXlkfkPTnZFs+C8nOVvAFSZunecMlvbeL90aZ565uDwF/D/kH9usRsaRJu7uBr6TXLZJ2SO8jyLaejyPb8r67RPt1ircUeu5DwPcl/Qn4I/AVsjcAkv4a2BPYLyJWpR1Nx0XEFfWV+24RMTN9e388TbosIp5UD65+FxHXN5n2lKQnyfZVvEi2v6CxzQJJk8j64hcAM8k+cABOBC6S9DTZ6/Qhsq64WkXEQkm/kPQssBz4XZM2K1I3wQWp62EgcD7Zc7E+GpK6xiD7xntsRKwC7pH0F8AjaQtuGfBZYCyrvzcg2/9wl6QFEfGRPl2D8iYBV6TX3dv837nZGl1G1lU4Mx1c0AEcmebdA1xF1t20okT7dYpPc2FmZjl3H5mZWc6hYGZmOYeCmZnlHApmZpZzKJiZWc6hYGZmOYeCmZnl/he6dD6leWljDgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "palavras = frequencia.keys()\n",
    "y_pos = np.arange(len(palavras))\n",
    "contagem = frequencia.values()\n",
    "\n",
    "plt.bar(y_pos, contagem, align='center', alpha=0.5)\n",
    "plt.xticks(y_pos, palavras)\n",
    "plt.ylabel('Frequencia')\n",
    "plt.title('Frequencia das palavras na frase')\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Pre Processing**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Expressões reculares**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Av. Paulitsta nº    , andar   cjto    '"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texto = 'Av. Paulitsta nº 900, andar 4 cjto 402'\n",
    "re.sub('[-|0-9]',' ', texto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'i m back baby     Data Science        '"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texto = \"i'm back baby!!!, Data Science ??;;;()\"\n",
    "re.sub(r'[-./?!,\":;()\\']',' ',texto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Stemming**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package rslp to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping stemmers\\rslp.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('rslp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'copi'"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stemmer = nltk.stem.RSLPStemmer()\n",
    "stemmer.stem('frequentemente')\n",
    "stemmer.stem('copiar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "def Stemming(instancia):\n",
    "    stemmer = nltk.stem.RSLPStemmer()\n",
    "    palavras=[]\n",
    "    for w in instancia.split():\n",
    "        palavras.append(stemmer.stem(w))\n",
    "    return (\" \".join(palavras))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ele trabalh frequ no projet de criaca de nov produt'"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Stemming('Ele trabalhava frequentemente no projeto de criacao de novos produtos')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'iron maiden eh uma band incr'"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Stemming('Iron Maiden eh uma Banda Incrível')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Stopwords**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Stopword** nada mais é que uma palavra que possui apenas significado sintático dentro da sentença, porém não traz informações relevantes sobre o seu sentido. <p> As stopwords possuem uma frequência muito grande em todos os idiomas e por esse motivo precisamos eliminá-las das palavras que extraímos do texto. Caso contrário, nosso algoritmo poderia dar importância para palavras como: “e”, “ou”, “para”….e isso certamente atrapalharia nossa análise.<p>\n",
    "    fonte: [Link](https://medium.com/@viniljf/utilizando-processamento-de-linguagem-natural-para-criar-um-sumariza%C3%A7%C3%A3o-autom%C3%A1tica-de-textos-775cb428c84e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping corpora\\stopwords.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('stopwords')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "stopwords = nltk.corpus.stopwords.words('portuguese')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['de',\n",
       " 'a',\n",
       " 'o',\n",
       " 'que',\n",
       " 'e',\n",
       " 'do',\n",
       " 'da',\n",
       " 'em',\n",
       " 'um',\n",
       " 'para',\n",
       " 'com',\n",
       " 'não',\n",
       " 'uma',\n",
       " 'os',\n",
       " 'no',\n",
       " 'se',\n",
       " 'na',\n",
       " 'por',\n",
       " 'mais',\n",
       " 'as',\n",
       " 'dos',\n",
       " 'como',\n",
       " 'mas',\n",
       " 'ao',\n",
       " 'ele',\n",
       " 'das',\n",
       " 'à',\n",
       " 'seu',\n",
       " 'sua',\n",
       " 'ou',\n",
       " 'quando',\n",
       " 'muito',\n",
       " 'nos',\n",
       " 'já',\n",
       " 'eu',\n",
       " 'também',\n",
       " 'só',\n",
       " 'pelo',\n",
       " 'pela',\n",
       " 'até',\n",
       " 'isso',\n",
       " 'ela',\n",
       " 'entre',\n",
       " 'depois',\n",
       " 'sem',\n",
       " 'mesmo',\n",
       " 'aos',\n",
       " 'seus',\n",
       " 'quem',\n",
       " 'nas',\n",
       " 'me',\n",
       " 'esse',\n",
       " 'eles',\n",
       " 'você',\n",
       " 'essa',\n",
       " 'num',\n",
       " 'nem',\n",
       " 'suas',\n",
       " 'meu',\n",
       " 'às',\n",
       " 'minha',\n",
       " 'numa',\n",
       " 'pelos',\n",
       " 'elas',\n",
       " 'qual',\n",
       " 'nós',\n",
       " 'lhe',\n",
       " 'deles',\n",
       " 'essas',\n",
       " 'esses',\n",
       " 'pelas',\n",
       " 'este',\n",
       " 'dele',\n",
       " 'tu',\n",
       " 'te',\n",
       " 'vocês',\n",
       " 'vos',\n",
       " 'lhes',\n",
       " 'meus',\n",
       " 'minhas',\n",
       " 'teu',\n",
       " 'tua',\n",
       " 'teus',\n",
       " 'tuas',\n",
       " 'nosso',\n",
       " 'nossa',\n",
       " 'nossos',\n",
       " 'nossas',\n",
       " 'dela',\n",
       " 'delas',\n",
       " 'esta',\n",
       " 'estes',\n",
       " 'estas',\n",
       " 'aquele',\n",
       " 'aquela',\n",
       " 'aqueles',\n",
       " 'aquelas',\n",
       " 'isto',\n",
       " 'aquilo',\n",
       " 'estou',\n",
       " 'está',\n",
       " 'estamos',\n",
       " 'estão',\n",
       " 'estive',\n",
       " 'esteve',\n",
       " 'estivemos',\n",
       " 'estiveram',\n",
       " 'estava',\n",
       " 'estávamos',\n",
       " 'estavam',\n",
       " 'estivera',\n",
       " 'estivéramos',\n",
       " 'esteja',\n",
       " 'estejamos',\n",
       " 'estejam',\n",
       " 'estivesse',\n",
       " 'estivéssemos',\n",
       " 'estivessem',\n",
       " 'estiver',\n",
       " 'estivermos',\n",
       " 'estiverem',\n",
       " 'hei',\n",
       " 'há',\n",
       " 'havemos',\n",
       " 'hão',\n",
       " 'houve',\n",
       " 'houvemos',\n",
       " 'houveram',\n",
       " 'houvera',\n",
       " 'houvéramos',\n",
       " 'haja',\n",
       " 'hajamos',\n",
       " 'hajam',\n",
       " 'houvesse',\n",
       " 'houvéssemos',\n",
       " 'houvessem',\n",
       " 'houver',\n",
       " 'houvermos',\n",
       " 'houverem',\n",
       " 'houverei',\n",
       " 'houverá',\n",
       " 'houveremos',\n",
       " 'houverão',\n",
       " 'houveria',\n",
       " 'houveríamos',\n",
       " 'houveriam',\n",
       " 'sou',\n",
       " 'somos',\n",
       " 'são',\n",
       " 'era',\n",
       " 'éramos',\n",
       " 'eram',\n",
       " 'fui',\n",
       " 'foi',\n",
       " 'fomos',\n",
       " 'foram',\n",
       " 'fora',\n",
       " 'fôramos',\n",
       " 'seja',\n",
       " 'sejamos',\n",
       " 'sejam',\n",
       " 'fosse',\n",
       " 'fôssemos',\n",
       " 'fossem',\n",
       " 'for',\n",
       " 'formos',\n",
       " 'forem',\n",
       " 'serei',\n",
       " 'será',\n",
       " 'seremos',\n",
       " 'serão',\n",
       " 'seria',\n",
       " 'seríamos',\n",
       " 'seriam',\n",
       " 'tenho',\n",
       " 'tem',\n",
       " 'temos',\n",
       " 'tém',\n",
       " 'tinha',\n",
       " 'tínhamos',\n",
       " 'tinham',\n",
       " 'tive',\n",
       " 'teve',\n",
       " 'tivemos',\n",
       " 'tiveram',\n",
       " 'tivera',\n",
       " 'tivéramos',\n",
       " 'tenha',\n",
       " 'tenhamos',\n",
       " 'tenham',\n",
       " 'tivesse',\n",
       " 'tivéssemos',\n",
       " 'tivessem',\n",
       " 'tiver',\n",
       " 'tivermos',\n",
       " 'tiverem',\n",
       " 'terei',\n",
       " 'terá',\n",
       " 'teremos',\n",
       " 'terão',\n",
       " 'teria',\n",
       " 'teríamos',\n",
       " 'teriam']"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stopwords"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "def RemoviStopWords(instancia):\n",
    "    instancia = instancia.lower()\n",
    "    stopwords = set(nltk.corpus.stopwords.words('portuguese'))\n",
    "    palavras = [i for i in instancia.split() if not i in stopwords]\n",
    "    return (\" \".join(palavras))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ola mundo, ouvindo música iron maiden spotify hoje dia todo'"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RemoviStopWords('Ola mundo, estou ouvindo música do Iron Maiden no Spotify hoje o dia todo')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Trabalhando com Wordnet**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package wordnet to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping corpora\\wordnet.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#download do wordnet\n",
    "nltk.download('wordnet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Synset('music.n.01'),\n",
       " Synset('music.n.02'),\n",
       " Synset('music.n.03'),\n",
       " Synset('music.n.04'),\n",
       " Synset('music.n.05')]"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from nltk.corpus import wordnet\n",
    "wordnet.synsets('music')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'an artistic form of auditory communication incorporating instrumental or vocal tones in a structured and continuous manner'"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.01').definition()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'any agreeable (pleasing and harmonious) sounds'"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.02').definition()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Synset('antiphony.n.01'),\n",
       " Synset('bach.n.02'),\n",
       " Synset('ballet.n.02'),\n",
       " Synset('beethoven.n.02'),\n",
       " Synset('brahms.n.02'),\n",
       " Synset('chopin.n.01'),\n",
       " Synset('dance_music.n.02'),\n",
       " Synset('gilbert_and_sullivan.n.01'),\n",
       " Synset('handel.n.02'),\n",
       " Synset('harmony.n.02'),\n",
       " Synset('haydn.n.02'),\n",
       " Synset('instrumental_music.n.01'),\n",
       " Synset('monophony.n.01'),\n",
       " Synset('mozart.n.02'),\n",
       " Synset('music_genre.n.01'),\n",
       " Synset('musical_composition.n.01'),\n",
       " Synset('overture.n.01'),\n",
       " Synset('part_music.n.01'),\n",
       " Synset('pizzicato.n.01'),\n",
       " Synset('polyphony.n.01'),\n",
       " Synset('polytonality.n.01'),\n",
       " Synset('popularism.n.01'),\n",
       " Synset('prelude.n.02'),\n",
       " Synset('refrain.n.01'),\n",
       " Synset('section.n.01'),\n",
       " Synset('serialism.n.01'),\n",
       " Synset('stravinsky.n.02'),\n",
       " Synset('syncopation.n.03'),\n",
       " Synset('ta'ziyeh.n.01'),\n",
       " Synset('tune.n.01'),\n",
       " Synset('vocal_music.n.01'),\n",
       " Synset('wagner.n.03')]"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.01').hyponyms()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Synset('music_of_the_spheres.n.01')]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.02').hyponyms()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Synset('sound.n.02')]"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.02').hypernyms()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Synset('beats_per_minute.n.01')]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wordnet.synset('music.n.03').part_meronyms()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sentiwordnet**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package sentiwordnet to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping corpora\\sentiwordnet.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# download do sentiwordnet\n",
    "nltk.download('sentiwordnet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.corpus import sentiwordnet as swn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<love.n.01: PosScore=0.625 NegScore=0.0>\n"
     ]
    }
   ],
   "source": [
    "print (swn.senti_synset('love.n.01'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<sad.a.01: PosScore=0.125 NegScore=0.75>\n"
     ]
    }
   ],
   "source": [
    "print (swn.senti_synset('sad.a.01'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Part of Speech**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping taggers\\averaged_perceptron_tagger.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# downlod do averaged_perceptron_tagger\n",
    "nltk.download('averaged_perceptron_tagger')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package tagsets to\n",
      "[nltk_data]     C:\\Users\\lucasbarros\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Unzipping help\\tagsets.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('tagsets')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Hi', 'NNP'),\n",
       " (',', ','),\n",
       " ('my', 'PRP$'),\n",
       " ('name', 'NN'),\n",
       " ('is', 'VBZ'),\n",
       " ('Lucas', 'NNP'),\n",
       " (',', ','),\n",
       " ('I', 'PRP'),\n",
       " ('want', 'VBP'),\n",
       " ('to', 'TO'),\n",
       " ('print', 'VB'),\n",
       " ('the', 'DT'),\n",
       " ('ticket', 'NN'),\n",
       " ('for', 'IN'),\n",
       " ('payment', 'NN')]"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = nltk.word_tokenize(\"Hi, my name is Lucas, I want to print the ticket for payment\")\n",
    "nltk.pos_tag(text)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NNP: noun, proper, singular\n",
      "    Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos\n",
      "    Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA\n",
      "    Shannon A.K.C. Meltex Liverpool ...\n"
     ]
    }
   ],
   "source": [
    "nltk.help.upenn_tagset('NNP')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NN: noun, common, singular or mass\n",
      "    common-carrier cabbage knuckle-duster Casino afghan shed thermostat\n",
      "    investment slide humour falloff slick wind hyena override subhumanity\n",
      "    machinist ...\n"
     ]
    }
   ],
   "source": [
    "nltk.help.upenn_tagset('NN')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IN: preposition or conjunction, subordinating\n",
      "    astride among uppon whether out inside pro despite on by throughout\n",
      "    below within for towards near behind atop around if like until below\n",
      "    next into if beside ...\n"
     ]
    }
   ],
   "source": [
    "nltk.help.upenn_tagset('IN')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Minha', 'NNP'),\n",
       " ('bike', 'NN'),\n",
       " ('apesar', 'NN'),\n",
       " ('de', 'IN'),\n",
       " ('antiga', 'FW'),\n",
       " (',', ','),\n",
       " ('é', 'FW'),\n",
       " ('ótima', 'NN')]"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = nltk.word_tokenize(\"Minha bike apesar de antiga, é ótima\")\n",
    "nltk.pos_tag(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Metallica', 'NNP'),\n",
       " ('também', 'NN'),\n",
       " ('é', 'NNP'),\n",
       " ('outra', 'VBZ'),\n",
       " ('banca', 'JJ'),\n",
       " ('incrível', 'NN'),\n",
       " ('!', '.')]"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = nltk.word_tokenize(\"Metallica também é outra banca incrível!\")\n",
    "nltk.pos_tag(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logos](https://www.africarising.tv/wp-content/uploads/2017/10/Dusty-video-animation-of-complete-computer-hacking-and-security-concept.jpg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}