Wikipedia.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Wikipedia.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vitorgaboardi/network_analysis/blob/main/Wikipedia.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tirw9HnrTuau"
      },
      "source": [
        "# 1.0 Case Study: constructing a network of wikipedia pages"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ryNuGuF5aK6t"
      },
      "source": [
        "## 1.1 Introduction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K4U4cYABYxsK"
      },
      "source": [
        "So far we have learned two ways of constructing a complex network:\n",
        "- from a CSV file \n",
        "- manually\n",
        "\n",
        "What is hard for small networks may be impossible for medium-to-large scale networks; it may be impossible even for small networks if you must repeat the analysis many\n",
        "times. The case study in this lesson shows you **how to construct a large\n",
        "network in an easy way**:  \n",
        "- automatically collecting node and edge data from the Internet.\n",
        "\n",
        "The other goal of this study (aside from mastering new network construction\n",
        "techniques) is quite pragmatic. \n",
        "- Wouldn’t you want to know where the **complex\n",
        "network analysis** fits in the context of other subjects and disciplines? \n",
        "\n",
        "An answer to this question is near at hand: [on Wikipedia](https://en.wikipedia.org/wiki/Complex_network).\n",
        "\n",
        "\n",
        "Let’s start with the Wikipedia page about complex networks—the seed page.\n",
        "(Unfortunately, there is no page on complex network analysis itself.) The page\n",
        "body has external links and links to other Wikipedia pages. Those other pages\n",
        "presumably are somewhat related to complex networks, or else why would\n",
        "the Wikipedia editors provide them?\n",
        "To build a network out of the seed page and other relevant pages, **let’s treat\n",
        "the pages (and the respective Wikipedia subjects) as the network nodes and\n",
        "the links between the pages as the network edges**. You will use snowball\n",
        "sampling (a breadth-first search or [BFS algorithm](https://en.wikipedia.org/wiki/Breadth-first_search)) to discover all the nodes and edges of interest.\n",
        "\n",
        "<img width=\"200\" src=\"https://drive.google.com/uc?export=view&id=1t9ZXB5Q-wRt_qWbf8eU_1RiTwVGaStHU\">\n",
        "\n",
        "As a result, you will have a network of all pages related to complex networks\n",
        "and hopefully, you will make some conclusions about it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JnNVv00Aalcx"
      },
      "source": [
        "## 1.2 Get the Data, Build the Network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5DofZOgJa9Nf"
      },
      "source": [
        "The first half of the project script consists of:\n",
        "\n",
        "1. the initialization prologue\n",
        "2. a heavy-duty loop that retrieves the Wikipedia pages and simultaneously builds the network of nodes and edges.\n",
        "\n",
        "Let’s first import all necessary modules. We will need:\n",
        "\n",
        "- the module **wikipedia** for fetching and exploring Wikipedia pages\n",
        "- the operator **itemgetter** for sorting a list of tuples\n",
        "- and, naturally, **networkx** itself.\n",
        "\n",
        "To target the **snowballing process**, define the constant **SEED, the name of the\n",
        "starting page**. \n",
        "\n",
        "> As a side note, by changing the name of the seed page, you can\n",
        "apply this analysis to any other subject on Wikipedia.\n",
        "\n",
        "\n",
        "Last but not least, when you start the snowballing, you will eventually (and\n",
        "quite soon) bump into the pages describing ISBN and ISSN numbers, the\n",
        "arXiv, PubMed, and the like. Almost all other Wikipedia pages refer to one or\n",
        "more of those pages. This hyper-connectedness transforms any network into\n",
        "a collection of almost perfect gigantic stars, making all Wikipedia-based networks\n",
        "look similar. **To avoid the stardom syndrome**, treat the known “star”\n",
        "pages as stop words in information retrieval—in other words, ignore any links\n",
        "to them. \n",
        "\n",
        "Constructing the **black list of stop words**, STOPS, is a matter of trial\n",
        "and error. We put thirteen subjects on it; you may want to add more when you\n",
        "come across other “stars.” We also excluded pages whose names begin with\n",
        "**\"List of\"**, because they are simply lists of other subjects."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zyD6-e-34h5P",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "13c9a2d6-8683-443b-aef5-48d0c3c24503"
      },
      "source": [
        "# always check the current version in github\n",
        "!pip install networkx==2.6.2"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: networkx==2.6.2 in /usr/local/lib/python3.7/dist-packages (2.6.2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sQcOSXayXX1W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "42a6b520-4c43-4389-83e2-ddc8b71c3c24"
      },
      "source": [
        "!pip install wikipedia"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting wikipedia\n",
            "  Downloading wikipedia-1.4.0.tar.gz (27 kB)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.7/dist-packages (from wikipedia) (4.6.3)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from wikipedia) (2.23.0)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.0.0->wikipedia) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.0.0->wikipedia) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.0.0->wikipedia) (2021.5.30)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.0.0->wikipedia) (3.0.4)\n",
            "Building wheels for collected packages: wikipedia\n",
            "  Building wheel for wikipedia (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for wikipedia: filename=wikipedia-1.4.0-py3-none-any.whl size=11696 sha256=7467463c1840bd570f875a3e3daf19b8f4ec2492795dceb00e20ebb446fdbd3e\n",
            "  Stored in directory: /root/.cache/pip/wheels/15/93/6d/5b2c68b8a64c7a7a04947b4ed6d89fb557dcc6bc27d1d7f3ba\n",
            "Successfully built wikipedia\n",
            "Installing collected packages: wikipedia\n",
            "Successfully installed wikipedia-1.4.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MINWDN4gTzAC"
      },
      "source": [
        "from operator import itemgetter\n",
        "import networkx as nx\n",
        "import wikipedia\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lVqdt1994G2J",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "61b20a23-8075-4bff-8a65-1d01a201cc5a"
      },
      "source": [
        "nx.__version__"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.6.2'"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YPTw8l5iT7z9"
      },
      "source": [
        "#https://en.wikipedia.org/wiki/Complex_network\n",
        "SEED = \"Tourism in Brazil\".title()\n",
        "STOPS = (\"International Standard Serial Number\",\n",
        "         \"International Standard Book Number\",\n",
        "         \"National Diet Library\",\n",
        "         \"International Standard Name Identifier\",\n",
        "         \"International Standard Book Number (Identifier)\",\n",
        "         \"Pubmed Identifier\", \n",
        "         \"Pubmed Central\",\n",
        "         \"Digital Object Identifier\", \n",
        "         \"Arxiv\",\n",
        "         \"Proc Natl Acad Sci Usa\", \n",
        "         \"Bibcode\",\n",
        "         \"Library Of Congress Control Number\", \n",
        "         \"Jstor\",\n",
        "         \"Doi (Identifier)\",\n",
        "         \"Isbn (Identifier)\",\n",
        "         \"Pmid (Identifier)\",\n",
        "         \"Arxiv (Identifier)\",\n",
        "         \"Bibcode (Identifier)\",\n",
        "         \"Saint Pierre Et Miquelon\")"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ChlfDfm7YQNY"
      },
      "source": [
        "The next code fragment deals with setting up the **snowballing process**. A\n",
        "breadth-first search, or BFS (sometimes known to computer programmers\n",
        "as a snowballing algorithm), must remember which **pages have been already\n",
        "processed** and which have been **discovered but not yet processed**. \n",
        "- the former are stored in the set **done_set**; \n",
        "- the latter, in the list **todo_lst** and **todo_set**. \n",
        "\n",
        "You need two data structures for the unprocessed pages because you want to\n",
        "know whether a page has been already recorded (an unordered lookup, *todo_set*) and\n",
        "which page is the next to be processed (an ordered lookup, *todo_lst*). \n",
        "\n",
        "\n",
        "Snowballing an extensive network—and Wikipedia with 6,196,713 articles in\n",
        "the English segment alone can produce a huge network!—takes considerable\n",
        "time.\n",
        "\n",
        "<img width=\"300\" src=\"https://drive.google.com/uc?export=view&id=1cOpkuhQEHJ3UxQOP3hoCd3KzSlqyf1AG\">\n",
        "\n",
        "Suppose you start with one seed node, and let’s say it has $N\\approx 100$ neighbors.\n",
        "Each of them has $N$ neighbors, too, to the total of $\\approx N+N×N$ nodes. The\n",
        "third round of discovery adds $\\approx N×N×N$ more nodes. The time to shave each\n",
        "next layer of nodes grows exponentially. For this exercise, let’s process only\n",
        "the seed node itself and its immediate neighbors (layers 0 and 1). Processing\n",
        "layer 2 is still feasible, but layer 3 requires $N×N×N×N\\approx 10^8$ page downloads. To keep track of the distance from the currently processed node to the seed, store both the layer to which a node belongs and the node name together as a tuple on the **todo_lst** list."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sy1gJQxwYdNi"
      },
      "source": [
        "todo_lst = [(0, SEED)] # The SEED is in the layer 0\n",
        "todo_set = set(SEED) # The SEED itself\n",
        "done_set = set() # Nothing is done yet"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z04Y-it9g5OR"
      },
      "source": [
        "g = nx.DiGraph()\n",
        "layer, page = todo_lst[0]"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ucGHcQ4-hP4V",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1b47587f-7d13-4581-d3f5-1e67dbf43827"
      },
      "source": [
        "%%time\n",
        "while layer < 2:\n",
        "  # Remove the name page of the current page from the todo_lst, \n",
        "  # and add it to the set of processed pages. \n",
        "  # If the script encounters this page again, it will skip over it.\n",
        "  del todo_lst[0]\n",
        "  done_set.add(page)\n",
        "  \n",
        "  # Show progress\n",
        "  print(layer, page) \n",
        "  \n",
        "  # Attempt to download the selected page.\n",
        "  try:\n",
        "    wiki = wikipedia.page(page)\n",
        "  except:\n",
        "    layer, page = todo_lst[0]\n",
        "    print(\"Could not load\", page)\n",
        "    continue\n",
        "  \n",
        "  for link in wiki.links:\n",
        "    link = link.title()\n",
        "    if link not in STOPS and not link.startswith(\"List Of\"):\n",
        "      if link not in todo_set and link not in done_set:\n",
        "        todo_lst.append((layer + 1, link))\n",
        "        todo_set.add(link)\n",
        "      g.add_edge(page, link)\n",
        "  layer, page = todo_lst[0]"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0 Tourism In Brazil\n",
            "1 2007–10 Recession In South America\n",
            "1 2014 Brazilian Economic Crisis\n",
            "1 Abrolhos Marine National Park\n",
            "1 Academic\n",
            "1 Academic Conference\n",
            "1 Acre (State)\n",
            "1 Administrative Divisions Of Brazil\n",
            "1 Adventure Travel\n",
            "1 Agriculture In Brazil\n",
            "1 Alagoas\n",
            "1 Alcântara, Maranhão\n",
            "1 Alta Floresta\n",
            "1 Amapá\n",
            "1 Amazon Rainforest\n",
            "1 Amazon Basin\n",
            "1 Amazonas (Brazilian State)\n",
            "1 American Dollar\n",
            "1 Americana, São Paulo\n",
            "1 Anchieta, Espírito Santo\n",
            "1 Andorra\n",
            "1 Angra Dos Reis\n",
            "1 Animal Husbandry In Brazil\n",
            "1 Antonina, Paraná\n",
            "1 Aparados Da Serra National Park\n",
            "1 Aquidauana\n",
            "1 Aracaju\n",
            "1 Aracati\n",
            "1 Araguaia River\n",
            "1 Araraquara\n",
            "1 Araras\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/wikipedia/wikipedia.py:389: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system (\"lxml\"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.\n",
            "\n",
            "The code that caused this warning is on line 389 of the file /usr/local/lib/python3.7/dist-packages/wikipedia/wikipedia.py. To get rid of this warning, pass the additional argument 'features=\"lxml\"' to the BeautifulSoup constructor.\n",
            "\n",
            "  lis = BeautifulSoup(html).find_all('li')\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Could not load Araxá\n",
            "1 Araxá\n",
            "1 Araçatuba\n",
            "1 Argentina\n",
            "1 Argentina–Brazil Relations\n",
            "1 Aripuanã River\n",
            "1 Armação Dos Búzios\n",
            "1 Arraial D'Ajuda\n",
            "1 Arraial Do Cabo\n",
            "1 Artistic\n",
            "1 Asia\n",
            "1 Assis Brasil\n",
            "1 Atibaia\n",
            "1 Australia\n",
            "1 Austria\n",
            "1 Automotive Industry In Brazil\n",
            "1 B3 (Stock Exchange)\n",
            "1 Br-287\n",
            "1 Bahamas\n",
            "1 Bahia\n",
            "1 Balneário Camboriú\n",
            "1 Banana Production In Brazil\n",
            "1 Bananal Island\n",
            "1 Barbados\n",
            "1 Barra De São Miguel, Alagoas\n",
            "1 Barra Do Garças\n",
            "1 Barretos\n",
            "1 Barão De Melgaço\n",
            "1 Baturité\n",
            "1 Baía Da Traição\n",
            "1 Beer In Brazil\n",
            "1 Belarus\n",
            "1 Belgium\n",
            "1 Belo Horizonte\n",
            "1 Belém\n",
            "Could not load Bibliography Of Brazil\n",
            "1 Bibliography Of Brazil\n",
            "1 Birigüi\n",
            "1 Blumenau\n",
            "1 Boa Vista, Roraima\n",
            "1 Boiçucanga\n",
            "1 Bolivia\n",
            "1 Bolsa Família\n",
            "1 Bombinhas\n",
            "1 Bonito, Mato Grosso Do Sul\n",
            "1 Botucatu\n",
            "1 Bragança Paulista\n",
            "1 Brasiléia\n",
            "1 Brasília\n",
            "Could not load Brazil\n",
            "1 Brazil\n",
            "1 Brazil Cost\n",
            "1 Brazilian Antarctica\n",
            "1 Brazilian Armed Forces\n",
            "1 Brazilian Carnival\n",
            "Could not load Brazilian Gold Rush\n",
            "1 Brazilian Gold Rush\n",
            "1 Brazilian Miracle\n",
            "1 Brazilian Art\n",
            "1 Brazilian Comics\n",
            "1 Brazilian Cuisine\n",
            "1 Brazilian Disease\n",
            "1 Brazilian Literature\n",
            "1 Brazilian Mythology\n",
            "1 Brazilian Painting\n",
            "1 Brazilian Real\n",
            "1 Brazilian Science Fiction\n",
            "1 Brazilian Sculpture\n",
            "1 Brazilian Wine\n",
            "Could not load Brazilians\n",
            "1 Brazilians\n",
            "1 Bulgaria\n",
            "1 Cabedelo\n",
            "1 Cabo Frio\n",
            "1 Cachoeira\n",
            "1 Caicó\n",
            "1 Cairu\n",
            "1 Caldas Novas\n",
            "1 Camelô\n",
            "1 Camocim\n",
            "1 Campina Grande\n",
            "1 Campinas\n",
            "1 Campo Grande\n",
            "1 Campos Do Jordão\n",
            "1 Canada\n",
            "1 Cananéia\n",
            "1 Canela, Rio Grande Do Sul\n",
            "1 Canoa Quebrada\n",
            "1 Caparaó National Park\n",
            "1 Cape São Roque\n",
            "1 Caribbean\n",
            "1 Carolina (Maranhão)\n",
            "1 Caruaru\n",
            "1 Caxambu\n",
            "1 Ceará\n",
            "1 Center-West Region, Brazil\n",
            "1 Central-West Region, Brazil\n",
            "1 Central Bank Of Brazil\n",
            "1 Chapada Diamantina\n",
            "1 Chapada Diamantina National Park\n",
            "1 Chapada Dos Guimarães National Park\n",
            "1 Chapada Dos Veadeiros National Park\n",
            "1 Chile\n",
            "1 China\n",
            "1 Cinema Of Brazil\n",
            "1 Cingapura Project\n",
            "1 Citizen\n",
            "1 Climate Of Brazil\n",
            "1 Coastline Of Brazil\n",
            "1 Coffee Production In Brazil\n",
            "1 Colombia\n",
            "Could not load Colonial Brazil\n",
            "1 Colonial Brazil\n",
            "1 Competition\n",
            "1 Congonhas\n",
            "1 Constitution Of Brazil\n",
            "1 Corruption In Brazil\n",
            "1 Corumbá\n",
            "1 Costa Rica\n",
            "1 Coxim\n",
            "1 Crime In Brazil\n",
            "1 Croatia\n",
            "1 Cuba\n",
            "1 Cuiabá\n",
            "1 Cultural Tourism\n",
            "1 Culture Of Brazil\n",
            "1 Curitiba\n",
            "1 Czech Republic\n",
            "1 Cáceres, Mato Grosso\n",
            "1 Demographics Of Brazil\n",
            "1 Denmark\n",
            "1 Dependent Territory\n",
            "1 Diamantina, Minas Gerais\n",
            "1 Domingos Martins\n",
            "1 Dominican Republic\n",
            "1 Dune\n",
            "Could not load Economic History Of Brazil\n",
            "1 Economic History Of Brazil\n",
            "1 Economy Of Brazil\n",
            "1 Economy Of The Empire Of Brazil\n",
            "1 Ecotourism\n",
            "1 Ecotourism In The Amazon Rainforest\n",
            "1 Ecuador\n",
            "1 Education In Brazil\n",
            "1 Elections In Brazil\n",
            "1 Electricity Sector In Brazil\n",
            "1 Eletrobras\n",
            "1 Emas National Park\n",
            "1 Embratur\n",
            "1 Empire Of Brazil\n",
            "1 Energy Policy Of Brazil\n",
            "1 Espírito Santo\n",
            "1 Estância, Sergipe\n",
            "1 Ethanol Fuel In Brazil\n",
            "1 Europe\n",
            "1 Fazenda\n",
            "1 Federal District (Brazil)\n",
            "1 Federal Government Of Brazil\n",
            "1 Fernando De Noronha\n",
            "1 Finland\n",
            "1 First Brazilian Republic\n",
            "1 Florianópolis\n",
            "1 Fome Zero\n",
            "1 Foreign Exchange Market\n",
            "1 Foreign Relations Of Brazil\n",
            "1 Fortaleza\n",
            "1 Fourth Brazilian Republic\n",
            "1 Foz Do Iguaçu\n",
            "1 France\n",
            "1 Friend\n",
            "1 Gdp\n",
            "Could not load Galinhos\n",
            "1 Galinhos\n",
            "1 Garanhuns\n",
            "1 Geography Of Brazil\n",
            "1 Geology Of Brazil\n",
            "1 Germany\n",
            "1 Goiás\n",
            "1 Goiás, Goiás\n",
            "1 Goiânia\n",
            "1 Governance Of Brazil\n",
            "1 Government Of Brazil\n",
            "1 Gramado\n",
            "1 Greece\n",
            "1 Guajará-Mirim\n",
            "1 Guaporé, Rio Grande Do Sul\n",
            "1 Guarapari\n",
            "1 Guararema\n",
            "1 Guaratuba\n",
            "1 Guarujá\n",
            "1 Guatemala\n",
            "1 Guyana\n",
            "Could not load Health In Brazil\n",
            "1 Health In Brazil\n",
            "1 Highway Of Dinosaurs\n",
            "1 History\n",
            "1 History Of Brazil\n",
            "1 History Of Brazil Since 1985\n",
            "1 History Of Brazilian Animation\n",
            "1 History Of Ethanol Fuel In Brazil\n",
            "1 Hong Kong\n",
            "Could not load Human Rights In Brazil\n",
            "1 Human Rights In Brazil\n",
            "1 Hungary\n",
            "1 Iceland\n",
            "1 Igarassu, Pernambuco\n",
            "1 Iguape\n",
            "1 Iguazu Falls\n",
            "1 Ilha Grande\n",
            "1 Ilha Do Mel State Park\n",
            "1 Ilhabela, São Paulo\n",
            "1 Ilhéus\n",
            "1 Immigration To Brazil\n",
            "1 Imperatriz\n",
            "1 Income Inequality In Brazil\n",
            "1 Index Of Brazil-Related Articles\n",
            "1 Index Of Brazil–Related Articles\n",
            "1 India\n",
            "Could not load Indigenous Peoples In Brazil\n",
            "1 Indigenous Peoples In Brazil\n",
            "1 Industry In Brazil\n",
            "1 Infrastructure Of Brazil\n",
            "1 Ingá Stone\n",
            "1 International Tourism\n",
            "1 Ipojuca\n",
            "1 Ireland\n",
            "1 Irrigation In Brazil\n",
            "1 Israel\n",
            "1 Itacaré\n",
            "1 Itajaí\n",
            "1 Italy\n",
            "Could not load Itamaracá\n",
            "1 Itamaracá\n",
            "1 Itapema\n",
            "1 Itatiaia National Park\n",
            "1 Itu, Brazil\n",
            "1 Jamaica\n",
            "1 Japan\n",
            "1 Jardim, Mato Grosso Do Sul\n",
            "1 Jaú\n",
            "1 Jijoca De Jericoacoara\n",
            "1 Joinville\n",
            "1 João Pessoa, Paraíba\n",
            "1 Juazeiro Do Norte\n",
            "1 Judiciary Of Brazil\n",
            "1 Kinship\n",
            "1 Languages Of Brazil\n",
            "1 Laranjeiras, Sergipe\n",
            "1 Late-2000S Recession\n",
            "1 Latin America\n",
            "1 Lavras\n",
            "1 Law Enforcement In Brazil\n",
            "1 Law Of Brazil\n",
            "1 Lecturer\n",
            "1 Leisure\n",
            "1 Lençóis\n",
            "1 Lençóis Maranhenses National Park\n",
            "1 Liechtenstein\n",
            "1 Lithuania\n",
            "1 Luxembourg\n",
            "1 Macapá\n",
            "1 Macau\n",
            "1 Macau (Rio Grande Do Norte)\n",
            "1 Maceió\n",
            "1 Malandragem\n",
            "1 Malaysia\n",
            "1 Malta\n",
            "1 Mamirauá\n",
            "1 Mamirauá Sustainable Development Reserve\n",
            "1 Manaus\n",
            "Could not load Maragogi\n",
            "1 Maragogi\n",
            "1 Marajó\n",
            "1 Maranhão\n",
            "1 Marataízes\n",
            "1 Mariana, Minas Gerais\n",
            "1 Martins, Rio Grande Do Norte\n",
            "1 Mato Grosso\n",
            "1 Mato Grosso Do Sul\n",
            "1 Maxaranguape\n",
            "1 Meeting\n",
            "1 Metropolitan Porto Alegre\n",
            "1 Mexico\n",
            "1 Military Dictatorship In Brazil\n",
            "1 Military Of Brazil\n",
            "1 Minas Gerais\n",
            "1 Mining In Brazil\n",
            "1 Ministry Of Tourism (Brazil)\n",
            "1 Monaco\n",
            "Could not load Monte Roraima\n",
            "1 Monte Roraima\n",
            "1 Morretes\n",
            "1 Morro De São Paulo\n",
            "1 Mossoró\n",
            "1 Music Of Brazil\n",
            "1 Namibia\n",
            "1 Natal, Brazil\n",
            "1 Natal, Rio Grande Do Norte\n",
            "1 National Congress Of Brazil\n",
            "1 National Symbols Of Brazil\n",
            "1 Natividade, Tocantins\n",
            "1 Netherlands\n",
            "1 New Jerusalem Theater\n",
            "1 New Zealand\n",
            "1 North America\n",
            "Could not load North Region, Brazil\n",
            "1 North Region, Brazil\n",
            "1 Northeast Region, Brazil\n",
            "1 Norway\n",
            "Could not load Nova Friburgo\n",
            "1 Nova Friburgo\n",
            "1 O Globo\n",
            "1 Oiapoque\n",
            "1 Olinda\n",
            "1 Orange Production In Brazil\n",
            "1 Ouro Preto\n",
            "1 Outline Of Brazil\n",
            "1 Palaeobotanical Garden In Mata\n",
            "1 Paleontological Sites Of Santa Maria\n",
            "1 Paleorrota Geopark\n",
            "1 Palmas, Tocantins\n",
            "1 Panama\n",
            "Could not load Pangaea\n",
            "1 Pangaea\n",
            "1 Pantanal\n",
            "1 Paraguay\n",
            "1 Paranaguá\n",
            "1 Paraná (State)\n",
            "1 Paraty\n",
            "Could not load Paraíba\n",
            "1 Paraíba\n",
            "1 Parintins\n",
            "1 Paripueira\n",
            "1 Parnamirim\n",
            "1 Parnaíba\n",
            "1 Pará\n",
            "1 Penedo\n",
            "1 Permian\n",
            "1 Pernambuco\n",
            "1 Peru\n",
            "Could not load Petrobras\n",
            "1 Petrobras\n",
            "1 Petrolina\n",
            "1 Petrópolis\n",
            "1 Philippines\n",
            "1 Piauí\n",
            "1 Pico Da Bandeira\n",
            "1 Pirenópolis\n",
            "1 Piúma, Espírito Santo\n",
            "1 Plano Collor\n",
            "1 Plano Trienal\n",
            "1 Poconé\n",
            "1 Poland\n",
            "1 Politics Of Brazil\n",
            "1 Ponta Porã\n",
            "1 Porto Alegre\n",
            "1 Porto Seguro\n",
            "1 Porto Velho\n",
            "1 Porto De Galinhas\n",
            "1 Porto De Pedras\n",
            "1 Portugal\n",
            "Could not load Poá\n",
            "1 Poá\n",
            "1 President Of Brazil\n",
            "1 Propriá\n",
            "1 Protected Areas Of Brazil\n",
            "1 Public Holidays In Brazil\n",
            "1 Qatar\n",
            "Could not load Recife\n",
            "1 Recife\n",
            "1 Recreation\n",
            "1 Regions Of Brazil\n",
            "1 Religion In Brazil\n",
            "1 Renewable Energy In Brazil\n",
            "1 Resende, Rio De Janeiro\n",
            "1 Ribeirão Preto\n",
            "1 Rio Branco, Acre\n",
            "1 Rio Grande Do Norte\n",
            "1 Rio Grande Do Sul\n",
            "1 Rio De Janeiro\n",
            "1 Rio De Janeiro (State)\n",
            "1 Romania\n",
            "Could not load Rondônia\n",
            "1 Rondônia\n",
            "1 Roraima\n",
            "1 Rubber Boom\n",
            "1 Russia\n",
            "1 Sabará\n",
            "1 Salvador, Bahia\n",
            "1 Samba Effect\n",
            "1 San Marino\n",
            "Could not load Santa Catarina (State)\n",
            "1 Santa Catarina (State)\n",
            "1 Santa Catarina Island\n",
            "Could not load Santa Teresa, Espírito Santo\n",
            "1 Santa Teresa, Espírito Santo\n",
            "1 Santarém, Brazil\n",
            "1 Santos (São Paulo)\n",
            "1 Saquarema\n",
            "1 Science And Technology In Brazil\n",
            "1 Scientific\n",
            "1 Seminar\n",
            "1 Sergipe\n",
            "1 Serra Gaúcha\n",
            "1 Serra Da Capivara National Park\n",
            "1 Serra Do Cipó National Park\n",
            "1 Serra Dos Órgãos\n",
            "1 Sete Cidades National Park\n",
            "1 Singapore\n",
            "1 Slovakia\n",
            "1 Slovenia\n",
            "1 Sobral, Ceará\n",
            "1 Social Issues In Brazil\n",
            "1 Solar Power In Brazil\n",
            "1 Sorocaba\n",
            "1 Sousa, Paraíba\n",
            "1 South Africa\n",
            "1 South America\n",
            "1 South Korea\n",
            "1 Southeast Region, Brazil\n",
            "1 Southern Region, Brazil\n",
            "1 Spain\n",
            "Could not load Sport In Brazil\n",
            "1 Sport In Brazil\n",
            "1 Square Kilometre\n",
            "1 States Of Brazil\n",
            "1 Superagui National Park\n",
            "1 Surinam\n",
            "1 Sweden\n",
            "1 Switzerland\n",
            "1 São Cristóvão\n",
            "1 São José Dos Campos\n",
            "1 São João Del Rei\n",
            "1 São Lourenço, Minas Gerais\n",
            "1 São Luís, Maranhão\n",
            "1 São Miguel Do Gostoso\n",
            "1 São Paulo\n",
            "1 São Paulo (State)\n",
            "1 São Sebastião, São Paulo\n",
            "1 São Thomé Das Letras\n",
            "1 São Vicente, São Paulo\n",
            "1 Tangará Da Serra\n",
            "1 Tatajuba\n",
            "1 Taxation In Brazil\n",
            "1 Tefé\n",
            "1 Telecommunications In Brazil\n",
            "1 Television In Brazil\n",
            "1 Teresina\n",
            "Could not load Teresópolis\n",
            "1 Teresópolis\n",
            "1 Thailand\n",
            "1 The Americas\n",
            "1 Tibau\n",
            "1 Tibau Do Sul\n",
            "1 Time In Brazil\n",
            "1 Timeline Of Brazilian Economic Stabilization Plans\n",
            "1 Timeline Of Brazilian History\n",
            "1 Tiradentes, Minas Gerais\n",
            "1 Tocantins (State)\n",
            "1 Torres, Rio Grande Do Sul\n",
            "1 Tourism In Argentina\n",
            "1 Tourism In Bolivia\n",
            "1 Tourism In Chile\n",
            "1 Tourism In Colombia\n",
            "1 Tourism In Ecuador\n",
            "1 Tourism In French Guiana\n",
            "1 Tourism In Guyana\n",
            "1 Tourism In Paraguay\n",
            "1 Tourism In Peru\n",
            "1 Tourism In South Georgia And The South Sandwich Islands\n",
            "1 Tourism In Suriname\n",
            "1 Tourism In Uruguay\n",
            "1 Tourism In Venezuela\n",
            "1 Tourism In The Falkland Islands\n",
            "1 Tourist\n",
            "1 Touros\n",
            "1 Trancoso, Bahia\n",
            "1 Transport In Brazil\n",
            "1 Travel And Tourism Competitiveness Report\n",
            "1 Triassic\n",
            "1 Trinidad And Tobago\n",
            "1 Tripadvisor\n",
            "1 Triunfo, Pernambuco\n",
            "1 Turkey\n",
            "1 Usd\n",
            "Could not load Ubajara National Park\n",
            "1 Ubajara National Park\n",
            "1 Ukraine\n",
            "1 Unemployment In Brazil\n",
            "1 United Kingdom\n",
            "1 United Kingdom Of Portugal, Brazil And The Algarves\n",
            "1 United States\n",
            "1 Universidade De Brasília\n",
            "1 Uruguay\n",
            "1 Vale (Mining Company)\n",
            "1 Vale Do Catimbau National Park\n",
            "1 Vargas Era\n",
            "1 Vassouras\n",
            "1 Vatican City\n",
            "1 Venezuela\n",
            "1 Vila Velha, Brazil\n",
            "1 Visa (Document)\n",
            "1 Visa Policy Of Brazil\n",
            "1 Visconde De Mauá\n",
            "1 Vitória, Brazil\n",
            "1 Water Resources Management In Brazil\n",
            "1 Water Supply And Sanitation In Brazil\n",
            "1 Wildlife Of Brazil\n",
            "1 World Economic Forum\n",
            "Could not load World Heritage Site\n",
            "1 World Heritage Site\n",
            "Could not load World Tourism Organization\n",
            "1 World Tourism Organization\n",
            "1 Xapuri\n",
            "1 Youth In Brazil\n",
            "1 Índice Bovespa\n",
            "CPU times: user 32.6 s, sys: 2.25 s, total: 34.8 s\n",
            "Wall time: 7min 12s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JrtewM_hPqJa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4ef9d58d-fcef-4e4a-a51e-d96551a6af2a"
      },
      "source": [
        "print(\"{} nodes, {} edges\".format(len(g), nx.number_of_edges(g)))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "86652 nodes, 221562 edges\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LMX9A1-Nh-8B"
      },
      "source": [
        "The network of interest is now in the variable g. But it is “dirty”: inaccurate, incomplete, and erroneous."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qS911QoZMzb7"
      },
      "source": [
        "## 1.3 Eliminate Duplicates"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8VkbRjFcBQaX"
      },
      "source": [
        "# make a copy of raw graph\n",
        "original = g.copy()"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UFrNIuluN1__",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "97e96431-805b-4da5-eb5c-049047dd8164"
      },
      "source": [
        "# remove self loops\n",
        "g.remove_edges_from(nx.selfloop_edges(g))\n",
        "\n",
        "# identify duplicates like that: 'network' and 'networks'\n",
        "duplicates = [(node, node + \"s\") \n",
        "              for node in g if node + \"s\" in g\n",
        "             ]\n",
        "\n",
        "for dup in duplicates:\n",
        "  # *dup is a technique named 'unpacking'\n",
        "  g = nx.contracted_nodes(g, *dup, self_loops=False)\n",
        "\n",
        "print(duplicates)\n",
        "\n",
        "duplicates = [(x, y) for x, y in \n",
        "              [(node, node.replace(\"-\", \" \")) for node in g]\n",
        "                if x != y and y in g]\n",
        "print(duplicates)\n",
        "\n",
        "for dup in duplicates:\n",
        "  g = nx.contracted_nodes(g, *dup, self_loops=False)\n",
        "\n",
        "# nx.contracted creates a new node/edge attribute called contraction\n",
        "# the value of the attribute is a dictionary, but GraphML\n",
        "# does not support dictionary attributes\n",
        "nx.set_node_attributes(g, 0,\"contraction\")\n",
        "nx.set_edge_attributes(g, 0,\"contraction\")"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[('China', 'Chinas'), ('Dune', 'Dunes'), ('Friend', 'Friends'), ('World Heritage Site', 'World Heritage Sites'), ('Raw Material', 'Raw Materials'), ('Elementary School', 'Elementary Schools'), ('Elephant', 'Elephants'), ('Guild', 'Guilds'), (\"Master'S Degree\", \"Master'S Degrees\"), ('Modena', 'Modenas'), ('Primary School', 'Primary Schools'), ('Saint', 'Saints'), ('Secondary School', 'Secondary Schools'), ('Afro-Brazilian', 'Afro-Brazilians'), ('Asian Brazilian', 'Asian Brazilians'), ('Biome', 'Biomes'), ('Inca', 'Incas'), ('Latitude', 'Latitudes'), ('Ordinal Direction', 'Ordinal Directions'), ('Plantation', 'Plantations'), ('Plateau', 'Plateaus'), ('Rainforest', 'Rainforests'), ('Rubber Tree', 'Rubber Trees'), ('Temperature', 'Temperatures'), ('Time Zone', 'Time Zones'), ('White Brazilian', 'White Brazilians'), ('Metropolitan Area', 'Metropolitan Areas'), ('Sovereign State', 'Sovereign States'), ('Hostel', 'Hostels'), ('Passport', 'Passports'), ('Ski Resort', 'Ski Resorts'), ('Souvenir', 'Souvenirs'), ('Banana', 'Bananas'), ('Bean', 'Beans'), ('Brazil Nut', 'Brazil Nuts'), ('Cashew', 'Cashews'), ('Coconut', 'Coconuts'), ('Drought', 'Droughts'), ('Fertilizer', 'Fertilizers'), ('Grape', 'Grapes'), ('Mango', 'Mangos'), ('Oat', 'Oats'), ('Onion', 'Onions'), ('Papaya', 'Papayas'), ('Pea', 'Peas'), ('Pine Nut', 'Pine Nuts'), ('Pineapple', 'Pineapples'), ('Soybean', 'Soybeans'), ('Tangerine', 'Tangerines'), ('Dance', 'Dances'), ('Lake', 'Lakes'), ('Pirate', 'Pirates'), ('Port', 'Ports'), ('Quilombo', 'Quilombos'), ('Matinha', 'Matinhas'), ('Black Brazilian', 'Black Brazilians'), ('National Park', 'National Parks'), ('Pardo Brazilian', 'Pardo Brazilians'), ('Atlantic Forest', 'Atlantic Forests'), ('Commonwealth Realm', 'Commonwealth Realms'), ('Dinosaur', 'Dinosaurs'), ('Fossil Fuel', 'Fossil Fuels'), ('Gaul', 'Gauls'), ('Invertebrate', 'Invertebrates'), ('Mammal', 'Mammals'), ('Plant', 'Plants'), ('Savanna', 'Savannas'), ('Tree', 'Trees'), ('Bird', 'Birds'), ('Parrot', 'Parrots'), ('Reptile', 'Reptiles'), ('Barreirinha', 'Barreirinhas'), ('Carmelite', 'Carmelites'), ('Catholic', 'Catholics'), ('Hunter-Gatherer', 'Hunter-Gatherers'), ('Jesuit', 'Jesuits'), ('Motorcycle', 'Motorcycles'), ('Central Bank', 'Central Banks'), ('Commercial Bank', 'Commercial Banks'), ('Demand Note', 'Demand Notes'), ('Dollar', 'Dollars'), ('Federal Reserve Bank', 'Federal Reserve Banks'), ('Federal Reserve Note', 'Federal Reserve Notes'), ('Fixed Exchange Rate', 'Fixed Exchange Rates'), ('Government Bond', 'Government Bonds'), ('Indian Reservation', 'Indian Reservations'), ('International Dollar', 'International Dollars'), ('Spanish Dollar', 'Spanish Dollars'), ('United States Note', 'United States Notes'), ('United States Commemorative Coin', 'United States Commemorative Coins'), ('Baptist', 'Baptists'), ('German Brazilian', 'German Brazilians'), ('Italian Brazilian', 'Italian Brazilians'), ('Methodist', 'Methodists'), ('Pentecostal', 'Pentecostals'), ('Presbyterian', 'Presbyterians'), ('Textile', 'Textiles'), ('Ethnic Group', 'Ethnic Groups'), ('Island', 'Islands'), ('Manor House', 'Manor Houses'), ('Quince', 'Quinces'), ('Romance Language', 'Romance Languages'), ('Trade Union', 'Trade Unions'), ('Unesco World Heritage Site', 'Unesco World Heritage Sites'), ('Winter Olympic', 'Winter Olympics'), ('Chicken', 'Chickens'), ('Bric', 'Brics'), ('Ceramic', 'Ceramics'), ('Guerrilla', 'Guerrillas'), ('Karajá Language', 'Karajá Languages'), ('Tupi-Guarani Language', 'Tupi-Guarani Languages'), ('Amerindian', 'Amerindians'), ('Braúna', 'Braúnas'), ('Arab Argentine', 'Arab Argentines'), ('Asian Argentine', 'Asian Argentines'), ('Atheist', 'Atheists'), ('Cardiovascular Disease', 'Cardiovascular Diseases'), ('Caudillo', 'Caudillos'), ('Conifer', 'Conifers'), ('Ecosystem', 'Ecosystems'), ('Empanada', 'Empanadas'), ('Fern', 'Ferns'), ('Gaucho', 'Gauchos'), ('Guarani Language', 'Guarani Languages'), ('Llama', 'Llamas'), ('Mestizo', 'Mestizos'), ('Mormon', 'Mormons'), ('Nuclear Reactor', 'Nuclear Reactors'), ('Nuclear Weapon', 'Nuclear Weapons'), ('Pancake', 'Pancakes'), ('Pato', 'Patos'), ('Quechua Language', 'Quechua Languages'), ('Satellite', 'Satellites'), ('Sea Port', 'Sea Ports'), ('Steppe', 'Steppes'), ('Syrian', 'Syrians'), ('Trade Deficit', 'Trade Deficits'), ('Waterway', 'Waterways'), ('Argentine Brazilian', 'Argentine Brazilians'), ('Hydroelectric Plant', 'Hydroelectric Plants'), ('Brazilian State', 'Brazilian States'), ('Carioca', 'Cariocas'), ('District', 'Districts'), ('Art', 'Arts'), ('Art Movement', 'Art Movements'), ('Art School', 'Art Schools'), ('Cultural Artifact', 'Cultural Artifacts'), ('Fine Art', 'Fine Arts'), ('Muse', 'Muses'), ('Social Science', 'Social Sciences'), ('Visual Art', 'Visual Arts'), ('Buddhist', 'Buddhists'), ('Continent', 'Continents'), ('Guinness World Record', 'Guinness World Records'), ('Himalaya', 'Himalayas'), ('Multinational Corporation', 'Multinational Corporations'), ('National Symbol', 'National Symbols'), ('Bandeirante', 'Bandeirantes'), ('Aboriginal Australian', 'Aboriginal Australians'), ('Anglican', 'Anglicans'), ('Anglo-Celtic Australian', 'Anglo-Celtic Australians'), ('Chinese Australian', 'Chinese Australians'), ('Cyclone', 'Cyclones'), ('English Australian', 'English Australians'), ('Filipino Australian', 'Filipino Australians'), ('German Australian', 'German Australians'), ('Greek Australian', 'Greek Australians'), ('Indian Australian', 'Indian Australians'), ('Indigenous Australian', 'Indigenous Australians'), ('Irish Australian', 'Irish Australians'), ('Italian Australian', 'Italian Australians'), ('Monsoon', 'Monsoons'), ('Refugee', 'Refugees'), ('Roman Catholic', 'Roman Catholics'), ('Scottish Australian', 'Scottish Australians'), ('Tectonic Plate', 'Tectonic Plates'), ('Urban Area', 'Urban Areas'), ('Vietnamese Australian', 'Vietnamese Australians'), ('Wetland', 'Wetlands'), ('Motor Vehicle', 'Motor Vehicles'), ('Afro-Bahamian', 'Afro-Bahamians'), ('America', 'Americas'), ('Arawak', 'Arawaks'), ('Bahama', 'Bahamas'), ('Coral Reef', 'Coral Reefs'), ('Hindu', 'Hindus'), ('Muslim', 'Muslims'), ('Protestant', 'Protestants'), ('Puritan', 'Puritans'), ('Sand Dune', 'Sand Dunes'), ('White Bahamian', 'White Bahamians'), ('Capoeira', 'Capoeiras'), ('Diamond', 'Diamonds'), ('Goat', 'Goats'), ('Hydroelectric Power Plant', 'Hydroelectric Power Plants'), ('Mineral', 'Minerals'), ('Pride Fighting Championship', 'Pride Fighting Championships'), ('Reef', 'Reefs'), ('Whale', 'Whales'), ('Banana Island', 'Banana Islands'), ('School', 'Schools'), ('Arab', 'Arabs'), ('Economic Partnership Agreement', 'Economic Partnership Agreements'), ('Slave', 'Slaves'), ('Maravilha', 'Maravilhas'), ('Bolshevik', 'Bolsheviks'), ('Habsburg', 'Habsburgs'), ('Immigrant', 'Immigrants'), ('Merovingian', 'Merovingians'), ('Arab Brazilian', 'Arab Brazilians'), ('Araponga', 'Arapongas'), ('Automobile', 'Automobiles'), ('Favela', 'Favelas'), ('Gemstone', 'Gemstones'), ('Portuguese Brazilian', 'Portuguese Brazilians'), ('Spanish Brazilian', 'Spanish Brazilians'), ('Araucaria', 'Araucarias'), ('Birth Rate', 'Birth Rates'), ('Chiefdom', 'Chiefdoms'), ('Chinese Brazilian', 'Chinese Brazilians'), ('Commodity Market', 'Commodity Markets'), ('Creole Language', 'Creole Languages'), ('Emerald', 'Emeralds'), ('Emerging Power', 'Emerging Powers'), ('Interest Rate', 'Interest Rates'), ('Japanese Brazilian', 'Japanese Brazilians'), ('Lusophone', 'Lusophones'), ('Mulatto', 'Mulattos'), ('Petrochemical', 'Petrochemicals'), ('Physicist', 'Physicists'), ('Polish Brazilian', 'Polish Brazilians'), ('Public Service', 'Public Services'), ('Semiconductor', 'Semiconductors'), ('Soap Opera', 'Soap Operas'), ('Tapir', 'Tapirs'), ('Ukrainian Brazilian', 'Ukrainian Brazilians'), ('Wind Farm', 'Wind Farms'), ('Afro-Bolivian', 'Afro-Bolivians'), ('Artisan', 'Artisans'), ('Conquistador', 'Conquistadors'), ('Glacier', 'Glaciers'), ('Hectare', 'Hectares'), ('Medicinal Plant', 'Medicinal Plants'), ('Econometric', 'Econometrics'), ('Charqueada', 'Charqueadas'), ('Headquarter', 'Headquarters'), ('Helicopter', 'Helicopters'), ('Cave Painting', 'Cave Paintings'), ('Cereal', 'Cereals'), ('Dutch Brazilian', 'Dutch Brazilians'), ('Hot Dog', 'Hot Dogs'), ('Wine', 'Wines'), ('Communist', 'Communists'), ('Afro-American Religion', 'Afro-American Religions'), ('Aesthetic', 'Aesthetics'), ('Indigenous People', 'Indigenous Peoples'), ('Numismatic', 'Numismatics'), ('Afrikaner', 'Afrikaners'), ('Armenian Brazilian', 'Armenian Brazilians'), ('Austrian Brazilian', 'Austrian Brazilians'), ('Greek Brazilian', 'Greek Brazilians'), ('Iranian People', 'Iranian Peoples'), ('Irish Brazilian', 'Irish Brazilians'), ('Mameluco', 'Mamelucos'), ('Native Brazilian', 'Native Brazilians'), ('New Christian', 'New Christians'), ('Death Rate', 'Death Rates'), ('Grammy Award', 'Grammy Awards'), ('Neanderthal', 'Neanderthals'), ('Waterfall', 'Waterfalls'), ('Aimoré', 'Aimorés'), ('Film Format', 'Film Formats'), ('Image Sensor', 'Image Sensors'), ('Primary Color', 'Primary Colors'), ('Business Incubator', 'Business Incubators'), ('Caiuá', 'Caiuás'), ('Flower', 'Flowers'), ('Guariba', 'Guaribas'), ('Integrated Circuit', 'Integrated Circuits'), ('Locomotive', 'Locomotives'), ('Railway', 'Railways'), ('Paraguayan Brazilian', 'Paraguayan Brazilians'), ('Festival', 'Festivals'), ('Black Hole', 'Black Holes'), ('Christian Denomination', 'Christian Denominations'), ('French Canadian', 'French Canadians'), ('French And Indian War', 'French And Indian Wars'), ('Glacial Period', 'Glacial Periods'), ('Ice Field', 'Ice Fields'), ('Leek', 'Leeks'), ('Minority Language', 'Minority Languages'), ('Paleo-Indian', 'Paleo-Indians'), ('Prince Edward Island', 'Prince Edward Islands'), ('Indo-Caribbean', 'Indo-Caribbeans'), ('Indo-Trinidadian', 'Indo-Trinidadians'), ('Tropical Cyclone', 'Tropical Cyclones'), ('Alagoinha', 'Alagoinhas'), ('Gameleira', 'Gameleiras'), ('Handicraft', 'Handicrafts'), ('Paulista', 'Paulistas'), ('Crustacean', 'Crustaceans'), ('Mangrove', 'Mangroves'), ('Tropical Fruit', 'Tropical Fruits'), ('Emerging Market', 'Emerging Markets'), ('Gigawatt', 'Gigawatts'), ('Indo-European Language', 'Indo-European Languages'), ('Ngo', 'Ngos'), ('Potential Superpower', 'Potential Superpowers'), ('Sino-Tibetan Language', 'Sino-Tibetan Languages'), ('Water Shortage', 'Water Shortages'), ('Zhuang Language', 'Zhuang Languages'), ('Academy Award', 'Academy Awards'), ('Business Magnate', 'Business Magnates'), ('City-State', 'City-States'), ('Commoner', 'Commoners'), ('Gcse', 'Gcses'), ('Russian Oligarch', 'Russian Oligarchs'), ('Caiapo', 'Caiapos'), ('Jesuit Reduction', 'Jesuit Reductions'), ('Slave Ship', 'Slave Ships'), ('Sport', 'Sports'), ('Airport', 'Airports'), ('Tonne', 'Tonnes'), ('Constitutional Amendment', 'Constitutional Amendments'), ('Encomienda', 'Encomiendas'), ('English-Based Creole Language', 'English-Based Creole Languages'), ('Export', 'Exports'), ('Hacienda', 'Haciendas'), ('Homicide', 'Homicides'), ('Croat', 'Croats'), ('Yugoslav Partisan', 'Yugoslav Partisans'), ('Ancestral Puebloan', 'Ancestral Puebloans'), ('Asian American', 'Asian Americans'), ('Cash Crop', 'Cash Crops'), ('European American', 'European Americans'), ('Great Power', 'Great Powers'), ('Multiracial American', 'Multiracial Americans'), ('Human Race', 'Human Races'), ('Korean Brazilian', 'Korean Brazilians'), ('Swiss Brazilian', 'Swiss Brazilians'), ('Theater', 'Theaters'), ('Wage', 'Wages'), ('European Brazilian', 'European Brazilians'), ('Portuguese-Brazilian', 'Portuguese-Brazilians'), ('Slavic People', 'Slavic Peoples'), ('South American', 'South Americans'), ('Crusade', 'Crusades'), ('Hussite', 'Hussites'), ('Motorsport', 'Motorsports'), ('Nuclear Power Plant', 'Nuclear Power Plants'), ('Thunderstorm', 'Thunderstorms'), ('Indigenous Brazilian', 'Indigenous Brazilians'), ('Lutheran', 'Lutherans'), ('Mizrahi Jew', 'Mizrahi Jews'), ('Sephardic Jew', 'Sephardic Jews'), ('Civil Union', 'Civil Unions'), ('Environmental Issue', 'Environmental Issues'), ('Mental Disorder', 'Mental Disorders'), ('Barangay', 'Barangays'), ('Commune', 'Communes'), ('Region', 'Regions'), ('Vassal State', 'Vassal States'), ('Willis Island', 'Willis Islands'), ('Evangelical Protestant', 'Evangelical Protestants'), ('Seventh-Day Adventist', 'Seventh-Day Adventists'), ('Spiritist', 'Spiritists'), ('Socioeconomic', 'Socioeconomics'), ('Belgian Brazilian', 'Belgian Brazilians'), ('Bulgarian Brazilian', 'Bulgarian Brazilians'), ('Consumer Durable', 'Consumer Durables'), ('Czech Brazilian', 'Czech Brazilians'), ('English Brazilian', 'English Brazilians'), ('Finnish Brazilian', 'Finnish Brazilians'), ('French Brazilian', 'French Brazilians'), ('Hungarian Brazilian', 'Hungarian Brazilians'), ('Latvian Brazilian', 'Latvian Brazilians'), ('Lebanese Brazilian', 'Lebanese Brazilians'), ('Phosphate', 'Phosphates'), ('Road', 'Roads'), ('Scandinavian Brazilian', 'Scandinavian Brazilians'), ('Scottish Brazilian', 'Scottish Brazilians'), ('Syrian Brazilian', 'Syrian Brazilians'), ('Chemical', 'Chemicals'), ('Consumer Good', 'Consumer Goods'), ('Natural Resource', 'Natural Resources'), ('Theme Park', 'Theme Parks'), ('Afro-Ecuadorian', 'Afro-Ecuadorians'), ('Applied Science', 'Applied Sciences'), ('Evangelical', 'Evangelicals'), ('National Academy Of Science', 'National Academy Of Sciences'), ('White Ecuadorian', 'White Ecuadorians'), ('White Latin American', 'White Latin Americans'), ('Coalition Government', 'Coalition Governments'), ('Megawatt', 'Megawatts'), ('Indentured Servant', 'Indentured Servants'), ('Émigré', 'Émigrés'), ('Mountain Range', 'Mountain Ranges'), ('Greenhouse Gas Emission', 'Greenhouse Gas Emissions'), ('Liter', 'Liters'), ('Tariff', 'Tariffs'), ('Balkan', 'Balkans'), ('Dolphin', 'Dolphins'), ('Homeric Hymn', 'Homeric Hymns'), ('Language', 'Languages'), ('Temperate Broadleaf And Mixed Forest', 'Temperate Broadleaf And Mixed Forests'), ('Viking', 'Vikings'), ('Western Religion', 'Western Religions'), ('Coastal Plain', 'Coastal Plains'), ('Sea Cave', 'Sea Caves'), ('Tectonic', 'Tectonics'), ('Tide Pool', 'Tide Pools'), ('Petroglyph', 'Petroglyphs'), ('Politic', 'Politics'), ('Surfer', 'Surfers'), ('Financial Instrument', 'Financial Instruments'), ('Scholarship', 'Scholarships'), ('Restinga', 'Restingas'), ('Skyscraper', 'Skyscrapers'), ('French Wine', 'French Wines'), ('Huguenot', 'Huguenots'), ('International Organization', 'International Organizations'), ('Middle Age', 'Middle Ages'), ('Anger', 'Angers'), ('Cross-Sex Friendship', 'Cross-Sex Friendships'), ('Interpersonal Relationship', 'Interpersonal Relationships'), ('Social Cue', 'Social Cues'), ('Troubadour', 'Troubadours'), ('Vaccine', 'Vaccines'), ('Paraguayan', 'Paraguayans'), ('Pampa', 'Pampas'), ('Rapid', 'Rapids'), ('Shale', 'Shales'), ('Extermination Camp', 'Extermination Camps'), ('Kurdish Language', 'Kurdish Languages'), ('Buriti', 'Buritis'), ('Palace', 'Palaces'), ('Hydrangea', 'Hydrangeas'), ('Additive Color', 'Additive Colors'), ('Blue', 'Blues'), ('Chinese Character', 'Chinese Characters'), ('Complementary Color', 'Complementary Colors'), ('Firework', 'Fireworks'), ('Maroon', 'Maroons'), ('Prostitute', 'Prostitutes'), ('White', 'Whites'), ('X-Ray', 'X-Rays'), ('Igneous Rock', 'Igneous Rocks'), ('Palmito', 'Palmitos'), ('German Guatemalan', 'German Guatemalans'), ('Highway', 'Highways'), ('Manila Galleon', 'Manila Galleons'), ('Maya People', 'Maya Peoples'), ('Superstition', 'Superstitions'), ('Papal Bull', 'Papal Bulls'), ('Shoe', 'Shoes'), ('Weapon', 'Weapons'), ('Court', 'Courts'), ('Germanic People', 'Germanic Peoples'), ('Hungarian Defence Force', 'Hungarian Defence Forces'), ('Iihf World Championship', 'Iihf World Championships'), ('Non-Governmental Organization', 'Non-Governmental Organizations'), ('Small And Medium Enterprise', 'Small And Medium Enterprises'), ('Electron', 'Electrons'), ('Illuminated Manuscript', 'Illuminated Manuscripts'), ('Linen', 'Linens'), ('Mussel', 'Mussels'), ('Oyster', 'Oysters'), ('Proto-Indo-European', 'Proto-Indo-Europeans'), ('Ulster Scots Dialect', 'Ulster Scots Dialects'), ('Canoe', 'Canoes'), ('Sedimentary Basin', 'Sedimentary Basins'), ('Southern Right Whale', 'Southern Right Whales'), ('Croatian Brazilian', 'Croatian Brazilians'), ('Turkish Brazilian', 'Turkish Brazilians'), ('White American', 'White Americans'), ('American Brazilian', 'American Brazilians'), ('Caribbean Brazilian', 'Caribbean Brazilians'), ('Chilean Brazilian', 'Chilean Brazilians'), ('Peruvian', 'Peruvians'), ('Romanian Brazilian', 'Romanian Brazilians'), ('Russian Brazilian', 'Russian Brazilians'), ('United Kingdom Of Portugal, Brazil, And The Algarve', 'United Kingdom Of Portugal, Brazil, And The Algarves'), ('Uruguayan Brazilian', 'Uruguayan Brazilians'), ('Apiacá', 'Apiacás'), ('Jê Language', 'Jê Languages'), ('Hamburger', 'Hamburgers'), ('Hominid', 'Hominids'), ('Israeli Settlement', 'Israeli Settlements'), ('War Crime', 'War Crimes'), ('Afro-Jamaican', 'Afro-Jamaicans'), ('Iguana', 'Iguanas'), ('Indo-Jamaican', 'Indo-Jamaicans'), ('White Jamaican', 'White Jamaicans'), ('Animist', 'Animists'), ('Cave', 'Caves'), ('People', 'Peoples'), ('Social Group', 'Social Groups'), ('Carib Language', 'Carib Languages'), ('Germanic Language', 'Germanic Languages'), ('Krenak Language', 'Krenak Languages'), ('Xukuruan Language', 'Xukuruan Languages'), ('Aztec', 'Aztecs'), ('Casta', 'Castas'), ('Christian', 'Christians'), ('Compas', 'Compass'), ('Latin American', 'Latin Americans'), ('Freguesia', 'Freguesias'), ('Steamboat', 'Steamboats'), ('Computer', 'Computers'), ('Teacher', 'Teachers'), ('Concentration Camp', 'Concentration Camps'), ('Roman Road', 'Roman Roads'), ('Baltic Language', 'Baltic Languages'), ('Fair', 'Fairs'), ('Larger Urban Zone', 'Larger Urban Zones'), ('Pharmaceutical', 'Pharmaceuticals'), ('War In Donbas', 'War In Donbass'), ('Winter Sport', 'Winter Sports'), ('Respiratory Disease', 'Respiratory Diseases'), ('Taoist', 'Taoists'), ('Palm Tree', 'Palm Trees'), ('Austronesian People', 'Austronesian Peoples'), ('Malaysian Indian', 'Malaysian Indians'), ('Murder', 'Murders'), ('Telecommunication', 'Telecommunications'), ('Chinese Dragon', 'Chinese Dragons'), ('Dolmen', 'Dolmens'), ('Fortification', 'Fortifications'), ('Ice Age', 'Ice Ages'), ('Curralinho', 'Curralinhos'), ('Quaker', 'Quakers'), ('Arab Mexican', 'Arab Mexicans'), ('Converso', 'Conversos'), ('Mesoamerican Pyramid', 'Mesoamerican Pyramids'), ('Vegetable', 'Vegetables'), ('Tepui', 'Tepuis'), ('Shopping Center', 'Shopping Centers'), ('Ballad', 'Ballads'), ('Boteco', 'Botecos'), ('Bantu People', 'Bantu Peoples'), ('Bantustan', 'Bantustans'), ('Baster', 'Basters'), ('Investment', 'Investments'), ('Penguin', 'Penguins'), ('Angico', 'Angicos'), ('Cultural Christian', 'Cultural Christians'), ('Fertility Rate', 'Fertility Rates'), ('Party System', 'Party Systems'), ('Sea Level', 'Sea Levels'), ('Trading Post', 'Trading Posts'), ('Chatham Island', 'Chatham Islands'), ('Macroeconomic', 'Macroeconomics'), ('Pacific Island', 'Pacific Islands'), ('Plate Tectonic', 'Plate Tectonics'), ('Franciscan', 'Franciscans'), ('Economic', 'Economics'), ('Canavieira', 'Canavieiras'), ('Passenger', 'Passengers'), ('Piçarra', 'Piçarras'), ('Donkey', 'Donkeys'), ('Pig', 'Pigs'), ('Austroasiatic Language', 'Austroasiatic Languages'), ('Bikol Language', 'Bikol Languages'), ('Negrito', 'Negritos'), ('Philippine', 'Philippines'), ('Public Broadcasting Service', 'Public Broadcasting Services'), ('Think Tank', 'Think Tanks'), ('Crop', 'Crops'), ('Herbivore', 'Herbivores'), ('Hornwort', 'Hornworts'), ('Horsetail', 'Horsetails'), ('Park', 'Parks'), ('Spore', 'Spores'), ('Effluent', 'Effluents'), ('Environmentalist', 'Environmentalists'), ('Neighborhood', 'Neighborhoods'), ('Human Right', 'Human Rights'), ('Brahma Kumari', 'Brahma Kumaris'), ('Slum', 'Slums'), ('Urban Agglomeration', 'Urban Agglomerations'), ('Argentine', 'Argentines'), ('Italian-Brazilian', 'Italian-Brazilians'), ('Martial Art', 'Martial Arts'), ('Pharmaceutic', 'Pharmaceutics'), ('Tijuca', 'Tijucas'), ('Arikem Language', 'Arikem Languages'), ('Jabutí Language', 'Jabutí Languages'), ('Tuparí Language', 'Tuparí Languages'), ('Snake', 'Snakes'), ('Cosmonaut', 'Cosmonauts'), ('Cro-Magnon', 'Cro-Magnons'), ('Islamist', 'Islamists'), ('Mongol', 'Mongols'), ('Vehicle', 'Vehicles'), ('Hurricane', 'Hurricanes'), ('Scallop', 'Scallops'), ('Confederado', 'Confederados'), ('Small Appliance', 'Small Appliances'), ('Fiber Optic', 'Fiber Optics'), ('Natural Science', 'Natural Sciences'), ('Scientific Paper', 'Scientific Papers'), ('Celt', 'Celts'), ('Gospel', 'Gospels'), ('Season', 'Seasons'), ('Benedictine', 'Benedictines'), ('Marrano', 'Marranos'), ('Uncontacted People', 'Uncontacted Peoples'), ('Plains Indian', 'Plains Indians'), ('Antibiotic', 'Antibiotics'), ('Chopstick', 'Chopsticks'), ('Ashkenazi Jew', 'Ashkenazi Jews'), ('Volga German', 'Volga Germans'), ('Creole People', 'Creole Peoples'), ('Cossack', 'Cossacks'), ('Crime Rate', 'Crime Rates'), ('Dairy Product', 'Dairy Products'), ('Enclave', 'Enclaves'), ('Mollusk', 'Mollusks'), ('Tony Award', 'Tony Awards'), ('Howitzer', 'Howitzers'), ('Tithe', 'Tithes'), ('Asian Elephant', 'Asian Elephants'), ('Sunni Muslim', 'Sunni Muslims'), ('Aleut People', 'Aleut Peoples'), ('Anglo-Saxon', 'Anglo-Saxons'), ('Californio', 'Californios'), ('Cloud Forest', 'Cloud Forests'), ('Chimichurri', 'Chimichurris'), ('Indian Reduction', 'Indian Reductions'), ('Sea Lion', 'Sea Lions'), ('Atacameño', 'Atacameños'), ('Meter', 'Meters'), ('Chestnut', 'Chestnuts'), ('Candlemas Island', 'Candlemas Islands'), ('King Penguin', 'King Penguins'), ('Airline', 'Airlines'), ('Alloy', 'Alloys'), ('Canal', 'Canals'), ('Final Good', 'Final Goods'), ('Home Appliance', 'Home Appliances'), ('Intermediate Good', 'Intermediate Goods'), ('Pedestrian', 'Pedestrians'), ('Perfume', 'Perfumes'), ('Aetosaur', 'Aetosaurs'), ('Archosaur', 'Archosaurs'), ('Phytosaur', 'Phytosaurs'), ('Trade Wind', 'Trade Winds'), ('Castle', 'Castles'), ('Mongol Invasion', 'Mongol Invasions'), ('Varangian', 'Varangians'), ('Channel Island', 'Channel Islands'), ('Algarve', 'Algarves'), ('Kingdom Of The Algarve', 'Kingdom Of The Algarves'), ('Linguist', 'Linguists'), ('Afro-Uruguayan', 'Afro-Uruguayans'), ('Asian Latin American', 'Asian Latin Americans'), ('Tribunal', 'Tribunals'), ('Foodstuff', 'Foodstuffs'), ('Italo-Venezuelan', 'Italo-Venezuelans'), ('Manatee', 'Manatees'), ('Mariche', 'Mariches'), ('Squirrel Monkey', 'Squirrel Monkeys')]\n",
            "[('Late-2000S Recession', 'Late 2000S Recession'), ('European Sovereign-Debt Crisis', 'European Sovereign Debt Crisis'), ('Afro-Brazilian', 'Afro Brazilian'), ('Carne-De-Sol', 'Carne De Sol'), ('Zip-Line', 'Zip Line'), ('Slash-And-Burn', 'Slash And Burn'), ('Sugar-Apple', 'Sugar Apple'), ('Anglo-America', 'Anglo America'), ('Guinea-Bissau', 'Guinea Bissau'), ('Commander-In-Chief', 'Commander In Chief'), ('Artist-In-Residence', 'Artist In Residence'), ('Alberto Santos-Dumont', 'Alberto Santos Dumont'), ('Asia-Pacific Economic Cooperation', 'Asia Pacific Economic Cooperation'), ('Semi-Arid', 'Semi Arid'), ('Mercedes-Benz', 'Mercedes Benz'), ('Working-Class', 'Working Class'), ('Flexible-Fuel Vehicle', 'Flexible Fuel Vehicle'), ('Brussels-Capital Region', 'Brussels Capital Region'), ('Sapucaí-Mirim', 'Sapucaí Mirim'), ('Brazilian Jiu-Jitsu', 'Brazilian Jiu Jitsu'), ('Free-Float', 'Free Float'), ('Jean-Baptiste Debret', 'Jean Baptiste Debret'), ('Per-Capita Income', 'Per Capita Income'), ('Great-Britain', 'Great Britain'), ('Ground-Glass', 'Ground Glass'), ('Medium-Format', 'Medium Format'), ('Eco-Tourism', 'Eco Tourism'), ('Saint-Barthélemy', 'Saint Barthélemy'), ('Governor-General', 'Governor General'), ('Japanese-Brazilian', 'Japanese Brazilian'), ('Portuguese-Brazilian', 'Portuguese Brazilian'), ('Asian-Brazilian', 'Asian Brazilian'), ('European-American', 'European American'), ('Middle-East', 'Middle East'), ('Sugar-Cane', 'Sugar Cane'), ('Case-Law', 'Case Law'), ('Value-Added Tax', 'Value Added Tax'), ('African-Brazilian', 'African Brazilian'), ('São João Del-Rei', 'São João Del Rei'), ('Free-Trade Area', 'Free Trade Area'), ('Morona-Santiago Province', 'Morona Santiago Province'), ('Manuel De Araújo Porto-Alegre, Baron Of Santo Ângelo', 'Manuel De Araújo Porto Alegre, Baron Of Santo Ângelo'), ('Free-Market', 'Free Market'), ('Vice-President Of Brazil', 'Vice President Of Brazil'), ('Foreign-Exchange Reserves', 'Foreign Exchange Reserves'), ('Baden-Württemberg', 'Baden Württemberg'), ('Italo-Brazilian', 'Italo Brazilian'), ('Al-Khidr', 'Al Khidr'), ('High-Income Economy', 'High Income Economy'), ('Upper-Hungary', 'Upper Hungary'), ('Tupi-Guarani', 'Tupi Guarani'), ('Public-Private Partnership', 'Public Private Partnership'), ('Horse-Racing', 'Horse Racing'), ('Savanna-La-Mar', 'Savanna La Mar'), ('German-Brazilian', 'German Brazilian'), ('German-Americans', 'German Americans'), ('Mexican-American War', 'Mexican American War'), ('Grand-Duke Of Luxembourg', 'Grand Duke Of Luxembourg'), ('Light-Rail', 'Light Rail'), ('Itapecuru-Mirim', 'Itapecuru Mirim'), ('Stainless-Steel', 'Stainless Steel'), ('Free-Trade Agreement', 'Free Trade Agreement'), ('Rock-And-Roll', 'Rock And Roll'), ('Hip-Hop', 'Hip Hop'), ('German South-West Africa', 'German South West Africa'), ('Free-Trade', 'Free Trade'), ('Mixed-Race', 'Mixed Race'), ('Ice-Cream', 'Ice Cream'), ('Art-Deco', 'Art Deco'), ('Non-Food Crop', 'Non Food Crop'), ('Italian-Brazilian', 'Italian Brazilian'), ('Bobó-De-Camarão', 'Bobó De Camarão'), ('Samba-Reggae', 'Samba Reggae'), ('Mother-Tongue', 'Mother Tongue'), ('Dokdo-Class Amphibious Assault Ship', 'Dokdo Class Amphibious Assault Ship'), ('Nieuw-Nickerie', 'Nieuw Nickerie'), ('Pan-American Games', 'Pan American Games'), ('Scorched-Earth', 'Scorched Earth'), ('Vaza-Barris River', 'Vaza Barris River'), ('South-West Africa', 'South West Africa'), ('Greenhouse-Gas', 'Greenhouse Gas'), ('Food-Web', 'Food Web'), ('Chutney-Soca', 'Chutney Soca'), ('Pan-American Health Organization', 'Pan American Health Organization')]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bqe2ruDmPcNH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3d2bc1c9-c2b5-434d-dd82-4823f7bd1fee"
      },
      "source": [
        "print(\"{} nodes, {} edges\".format(len(g), nx.number_of_edges(g)))"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "85906 nodes, 221021 edges\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TulPj3ZQQAS8"
      },
      "source": [
        "## 1.4 Truncate the network"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AaHxaOwMLW9B",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "outputId": "ef8ccac8-766c-44da-e173-492d528389e9"
      },
      "source": [
        "plt.style.use(\"default\")\n",
        "# degree sequence\n",
        "degree_sequence = sorted([d for n, d in g.degree()], reverse=True)  \n",
        "\n",
        "fig, ax = plt.subplots(1,2,figsize=(8,4))\n",
        "\n",
        "# all_data has information about degree_sequence and the width of each bin\n",
        "ax[0].hist(degree_sequence)\n",
        "ax[1].hist(degree_sequence,bins=[1,2,3,4,5,6,7,8,9,10])\n",
        "\n",
        "ax[0].set_title(\"Degree Histogram\")\n",
        "ax[0].set_ylabel(\"Count\")\n",
        "ax[0].set_xlabel(\"Degree\")\n",
        "ax[0].set_ylim(0,60000)\n",
        "\n",
        "ax[1].set_title(\"Degree Histogram - Zoom\")\n",
        "ax[1].set_ylabel(\"Count\")\n",
        "ax[1].set_xlabel(\"Degree\")\n",
        "ax[1].set_xlim(0,10)\n",
        "ax[1].set_ylim(0,60000)\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.show()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAGGCAYAAADmRxfNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdfVgU9f7/8Rcgd4K7eAdoolCWd3mTNyFpqYVuip1j6UnLk2hmamgpp1ROedPNN0tPpaWmnr6lv3OyzG485g1mmFpJaniLhWlZ3oIowaopKszvD7/McQNTGHRZeT6ua6/amffOfGa0effa2ZnxMgzDEAAAAABY4O3uAQAAAADwfAQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUEC8CDeHl5adKkSe4eBgDg/3BcBv6LYIGrbt68efLy8jJfAQEBqlu3rhwOh15//XUdP37c3UO8KiZNmiQvLy8dPXq0xPmRkZHq2bOn5fUsWLBA06ZNs7wcANcujsvncVyumIr+XC71gvtVcfcAUHk999xzioqK0tmzZ5WZmak1a9Zo1KhRevXVV7VkyRK1aNHC3UOscE6dOqUqVUr3n+2CBQuUnp6uUaNGXaFRAbhWcFwuPY7LV959992nhg0bljhv+/btmjp1qqKjo6/yqFASggXcpnv37mrbtq35PikpSatXr1bPnj31pz/9Sd9//70CAwOv2ngMw9Dp06ev6jpLKyAgwN1DKLWTJ08qKCjI3cMAcBk4Lpcex+Urr0WLFiWG2pMnT+qFF16Q3W7Xe++954aR4ff4KRQqlDvvvFPjx4/XL7/8on//+98u8zIyMtSnTx/VqFFDAQEBatu2rZYsWVJsGdu3b1enTp0UGBioevXq6YUXXtA777wjLy8v/fzzz2Zd0SntlStXqm3btgoMDNScOXMkSbm5uRo1apQiIiLk7++vhg0b6uWXX1ZhYaHLugoLCzVt2jQ1a9ZMAQEBCgsL09ChQ/Xrr7+W/85R8d/yHj9+XKNGjVJkZKT8/f0VGhqqrl27avPmzZKkzp07a9myZfrll1/MU8WRkZHm548cOaLBgwcrLCxMAQEBatmypebPn19svceOHdNDDz0km82mkJAQxcfHa9u2bfLy8tK8efPMuoEDByo4OFg//vijevTooWrVqql///6SpC+//FJ/+ctfVL9+ffn7+ysiIkKjR4/WqVOnXNZVtIx9+/apZ8+eCg4O1nXXXaeZM2dKknbs2KE777xTQUFBatCggRYsWFBOexdASTgu/zGOy+47Lj/22GPatWuX5s6dq6ioKJd5q1ev1u23366goCCFhIToz3/+s77//vtiy9iyZYu6d+8um82m4OBg3XXXXfrmm29caop+KvjVV1/p8ccfV+3atRUSEqKhQ4fqzJkzys3N1YABA1S9enVVr15dY8aMkWEYV3TbKyrOWKDCeeihh/T3v/9dn332mYYMGSJJ2rlzpzp06KDrrrtO48aNU1BQkD744AP16tVLH330ke69915J0sGDB9WlSxd5eXkpKSlJQUFBeuutt+Tv71/iunbt2qUHHnhAQ4cO1ZAhQ9SoUSP99ttv6tSpkw4ePKihQ4eqfv36Wr9+vZKSknT48GGX38UOHTpU8+bN06BBg/T4449r7969mjFjhrZs2aKvv/5avr6+l9zenJycEqf/vlmWZNiwYfrwww81YsQINW3aVMeOHdNXX32l77//Xq1bt9bTTz+tvLw8HThwQK+99pokKTg4WNL50/edO3fWnj17NGLECEVFRWnRokUaOHCgcnNz9cQTT5jjuOeee7Rx40YNHz5cjRs31n/+8x/Fx8eXOKZz587J4XCoY8eO+sc//qGqVatKkhYtWqTffvtNw4cPV82aNbVx40a98cYbOnDggBYtWuSyjIKCAnXv3l133HGHpkyZonfffVcjRoxQUFCQnn76afXv31/33XefZs+erQEDBigmJqZYUwFQfjgun8dxueIcl+fPn6//9//+n4YMGaL777/fZd7nn3+u7t276/rrr9ekSZN06tQpvfHGG+rQoYM2b95sBrmdO3fq9ttvl81m05gxY+Tr66s5c+aoc+fOWrt2bbGfV40cOVLh4eF69tln9c0332ju3LkKCQnR+vXrVb9+fb344otavny5pk6dqptvvlkDBgy4YttfYRnAVfbOO+8YkoxNmzZdtMZutxu33HKL+f6uu+4ymjdvbpw+fdqcVlhYaNx2223GjTfeaE4bOXKk4eXlZWzZssWcduzYMaNGjRqGJGPv3r3m9AYNGhiSjOTkZJd1P//880ZQUJDxww8/uEwfN26c4ePjY+zbt88wDMP48ssvDUnGu+++61KXnJxc4vTfmzhxoiHpD19xcXEun5FkTJw40WU/JSQk/OF64uLijAYNGhSbPm3aNEOS8e9//9ucdubMGSMmJsYIDg42nE6nYRiG8dFHHxmSjGnTppl1BQUFxp133mlIMt555x1zenx8vCHJGDduXLH1/fbbb8WmTZ482fDy8jJ++eWXYst48cUXzWm//vqrERgYaHh5eRnvv/++OT0jI6PYPgFQehyXz+O47BnH5e+//94ICgoymjVrVuI2tGrVyggNDTWOHTtmTtu2bZvh7e1tDBgwwJzWq1cvw8/Pz/jxxx/NaYcOHTKqVatm3HHHHea0ov8+HA6HUVhYaE6PiYkxvLy8jGHDhpnTzp07Z9SrV8/o1KlTeW2uR+GnUKiQgoODzbuQ5OTkaPXq1br//vt1/PhxHT16VEePHtWxY8fkcDi0e/duHTx4UJKUnJysmJgYtWrVylxWjRo1zNO+vxcVFSWHw+EybdGiRbr99ttVvXp1c11Hjx5VbGysCgoKtG7dOrPObrera9euLnVt2rRRcHCwvvjii8va1o8++kirVq0q9goLC7vkZ0NCQrRhwwYdOnTostZ1oeXLlys8PFwPPPCAOc3X11ePP/64Tpw4obVr10o6v099fX3NbyklydvbWwkJCRdd9vDhw4tNu/A30idPntTRo0d12223yTAMbdmypVj9I488Yv57SEiIGjVqpKCgIJdvpho1aqSQkBD99NNPl7nVAMqK4zLH5YpwXD59+rT69u2rwsJCLVy4sNj1N4cPH9bWrVs1cOBA1ahRw5zeokULde3aVcuXL5d0/gzMZ599pl69eun666836+rUqaMHH3xQX331lZxOp8uyBw8e7HL3qejoaBmGocGDB5vTfHx81LZt20rbl/gpFCqkEydOKDQ0VJK0Z88eGYah8ePHa/z48SXWHzlyRNddd51++eUXxcTEFJt/sbtJlHSadvfu3dq+fbtq16590XUV1eXl5ZnjvFjdpdxxxx2qVatWsemXc0HglClTFB8fr4iICLVp00Y9evTQgAEDXA6SF/PLL7/oxhtvlLe36/cLTZo0MecX/bNOnTrmqfMiF9unVapUUb169YpN37dvnyZMmKAlS5YU+61zXl6ey/uAgIBi+99ut6tevXrFbilot9uv2G+nAfwXx2WOy+V5XM7OzlZBQYH5Pjg42PxJ2B8ZNWqUtm/frjlz5qhZs2bF5hfto0aNGhWb16RJE61cuVInT57U8ePH9dtvv120rrCwUPv373dZR/369V3q7Ha7JCkiIqLY9MralwgWqHAOHDigvLw88wBZ9JvWJ598sti3WEUudjC9lJLuNFJYWKiuXbtqzJgxJX7mpptuMutCQ0P17rvvllh3sQZYnu6//37dfvvt+uSTT/TZZ59p6tSpevnll/Xxxx+re/fuV3z9JfH39y/WFAsKCtS1a1fl5ORo7Nixaty4sYKCgnTw4EENHDiw2O+WfXx8Slz2xaYblfQiOeBq4bh8+Tgun3ep43K7du3MECBJEydOvOSDBhctWqQ5c+bo/vvv16OPPvqHtVdCafZBZe1LBAtUOP/6178kyWxWRd/y+Pr6KjY29g8/26BBA+3Zs6fY9JKmXcwNN9ygEydOXHJdN9xwgz7//HN16NDBrbdCrFOnjh577DE99thjOnLkiFq3bq3/+Z//MRvYxR4a1KBBA23fvl2FhYUuDScjI8OcX/TPL774Qr/99pvLt2Ol2ac7duzQDz/8oPnz57tczLZq1arL31AAbsNxuXQ4Ll/au+++63L3qUud0fnpp580ZMgQRUVFae7cuRetK9pHu3btKjYvIyNDtWrVUlBQkAICAlS1atWL1nl7exc7E4FL4xoLVCirV6/W888/r6ioKPP3t6GhoercubPmzJmjw4cPF/tMdna2+e8Oh0OpqanaunWrOS0nJ+ei316V5P7771dqaqpWrlxZbF5ubq7OnTtn1hUUFOj5558vVnfu3Dnl5uZe9jrLoqCgoNip6tDQUNWtW1f5+fnmtKCgoGJ1ktSjRw9lZmZq4cKF5rRz587pjTfeUHBwsDp16iTp/D49e/as/vnPf5p1hYWF5m0GL0fRtzkXfoNjGIamT59+2csA4B4cly8fx+XL16FDB8XGxpqvPwoWZ8+eVb9+/fTbb7/pvffeM3+CVJI6deqoVatWmj9/vsufd3p6uj777DP16NFD0vnt79atm/7zn/+43PI4KytLCxYsUMeOHWWz2axvaCXDGQu4zYoVK5SRkaFz584pKytLq1ev1qpVq9SgQQMtWbLE5besM2fOVMeOHdW8eXMNGTJE119/vbKyspSamqoDBw5o27ZtkqQxY8bo3//+t7p27aqRI0eatzWsX7++cnJyLvot0YWeeuopLVmyRD179tTAgQPVpk0bnTx5Ujt27NCHH36on3/+WbVq1VKnTp00dOhQTZ48WVu3blW3bt3k6+ur3bt3a9GiRZo+fbr69Olzxfbf8ePHVa9ePfXp00ctW7ZUcHCwPv/8c23atEmvvPKKWdemTRstXLhQiYmJateunYKDg3XPPffo0Ucf1Zw5czRw4EClpaUpMjJSH374ob7++mtNmzZN1apVkyT16tVLt956q/72t79pz549aty4sZYsWWLejvFy9mnjxo11ww036Mknn9TBgwdls9n00UcfVdrfoAIVFcdlazguXxnjx4/Xpk2bdOedd2r37t3avXt3iXX33nuvgoKCNHXqVHXv3l0xMTEaPHiwebtZu93u8nOrF154QatWrVLHjh312GOPqUqVKpozZ47y8/M1ZcqUq7R11xj33IwKlVnRbduKXn5+fkZ4eLjRtWtXY/r06ebt9H7vxx9/NAYMGGCEh4cbvr6+xnXXXWf07NnT+PDDD13qtmzZYtx+++2Gv7+/Ua9ePWPy5MnG66+/bkgyMjMzzboGDRoUu21gkePHjxtJSUlGw4YNDT8/P6NWrVrGbbfdZvzjH/8wzpw541I7d+5co02bNkZgYKBRrVo1o3nz5saYMWOMQ4cO/eF+KLqtYXZ2donzSxqfLriFX35+vvHUU08ZLVu2NKpVq2YEBQUZLVu2NGbNmuXymRMnThgPPvigERISYkhyucVhVlaWMWjQIKNWrVqGn5+f0bx5c5fbFBbJzs42HnzwQaNatWqG3W43Bg4caHz99deGJJfbDMbHxxtBQUElbs93331nxMbGGsHBwUatWrWMIUOGGNu2bSvx1oglLaNTp05Gs2bNLms/ASgdjsvncVyumMflTp06XfI2wPrdrYs///xzo0OHDkZgYKBhs9mMe+65x/juu++KLXvz5s2Gw+EwgoODjapVqxpdunQx1q9f71JzsdsxX+zvyx/t82udl2FU0qtLUKmMGjVKc+bM0YkTJy568RVKZ/Hixbr33nv11VdfqUOHDu4eDgAPw3G5/HFchrtxjQWuORdeDCZJx44d07/+9S917NiR5lVGv9+nBQUFeuONN2Sz2dS6dWs3jQqAp+C4XP44LqMi4hoLXHNiYmLUuXNnNWnSRFlZWfrf//1fOZ3Oi95rHZc2cuRInTp1SjExMcrPz9fHH3+s9evX68UXX3TrnVcAeAaOy+WP4zIqIn4KhWvO3//+d3344Yc6cOCAvLy81Lp1a02cOPGStynExS1YsECvvPKK9uzZo9OnT6thw4YaPny4RowY4e6hAfAAHJfLH8dlVEjuvcTDMA4cOGD079/fqFGjhhEQEGDcfPPNLhfHFBYWGuPHjzfCw8ONgIAA46677jJ++OEHl2UcO3bM5QKmhx9+2Dh+/LhLzbZt24yOHTuaF469/PLLxcbywQcfGI0aNTL8/f2Nm2++2Vi2bNmV2WgAwEXRFwDAM7n1Gotff/1VHTp0kK+vr1asWKHvvvtOr7zyiqpXr27WTJkyRa+//rpmz56tDRs2KCgoSA6HQ6dPnzZr+vfvr507d2rVqlVaunSp1q1b5/JERqfTqW7duqlBgwZKS0vT1KlTNWnSJJcHrKxfv14PPPCABg8erC1btqhXr17q1auX0tPTr87OAADQFwDAk7kz1YwdO9bo2LHjRecXFhYa4eHhxtSpU81pubm5hr+/v/Hee+8ZhnH+Vmn63S3AVqxYYXh5eRkHDx40DMMwZs2aZVSvXt3Iz893WXejRo3M9/fff3+xW6NFR0cbQ4cOtbaRAIDLRl8AAM/l1ou3lyxZIofDob/85S9au3atrrvuOj322GMaMmSIJGnv3r3KzMx0+Q2m3W5XdHS0UlNT1a9fP6WmpiokJERt27Y1a2JjY+Xt7a0NGzbo3nvvVWpqqu644w75+fmZNQ6HQy+//LJ+/fVXVa9eXampqUpMTHQZn8Ph0OLFi0sce35+vstTNAsLC5WTk6OaNWte1oNpAOBqMwxDx48fV926deXtXTFvCujJfUGiNwDwLOXdF9waLH766Se9+eabSkxM1N///ndt2rRJjz/+uPz8/BQfH6/MzExJUlhYmMvnwsLCzHmZmZkKDQ11mV+lShXVqFHDpSYqKqrYMormVa9eXZmZmX+4nt+bPHmynn322TJuOQC4z/79+1WvXj13D6NEntwXJHoDAM9UXn3BrcGisLBQbdu21YsvvihJuuWWW5Senq7Zs2crPj7enUO7pKSkJJdvsvLy8lS/fn3t379fNputVMu6eeLK8h7eRaU/67hq6wJQsTidTkVERKhatWruHspFeXJfkMq3NwDAlVbefcGtwaJOnTpq2rSpy7QmTZroo48+kiSFh4dLkrKyslSnTh2zJisrS61atTJrjhw54rKMc+fOKScnx/x8eHi4srKyXGqK3l+qpmj+7/n7+8vf37/YdJvNVurm4e1ftVT1VtDYAFTkn+R4cl+Qyrc3AMDVUl59wa0/su3QoYN27drlMu2HH35QgwYNJElRUVEKDw9XSkqKOd/pdGrDhg2KiYmRdP6hO7m5uUpLSzNrVq9ercLCQkVHR5s169at09mzZ82aVatWqVGjRuadRmJiYlzWU1RTtB4AwJVHXwAAz+XWYDF69Gh98803evHFF7Vnzx4tWLBAc+fOVUJCgqTz6WnUqFF64YUXtGTJEu3YsUMDBgxQ3bp11atXL0nnv8m6++67NWTIEG3cuFFff/21RowYoX79+qlu3bqSpAcffFB+fn4aPHiwdu7cqYULF2r69Okup6ufeOIJJScn65VXXlFGRoYmTZqkb7/9lgfNAMBVRF8AAM/l9idvL126VElJSdq9e7eioqKUmJho3v1DOn+1+sSJEzV37lzl5uaqY8eOmjVrlm666SazJicnRyNGjNCnn34qb29v9e7dW6+//rqCg4PNmu3btyshIUGbNm1SrVq1NHLkSI0dO9ZlLIsWLdIzzzyjn3/+WTfeeKOmTJmiHj16XNZ2OJ1O2e125eXllfp0d+S4ZaWqt+Lnl+Ku2roAVCxWjlNX07XSFyTP2ecAKqfyPka5PVhcKwgWACo6/if36mOfA6jIyvsYVTFvZA4AAADAo7j1rlAAAKD8Xc0z4RfirDhQuXHGAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlrk1WEyaNEleXl4ur8aNG5vzT58+rYSEBNWsWVPBwcHq3bu3srKyXJaxb98+xcXFqWrVqgoNDdVTTz2lc+fOudSsWbNGrVu3lr+/vxo2bKh58+YVG8vMmTMVGRmpgIAARUdHa+PGjVdkmwEAF0dfAADP5fYzFs2aNdPhw4fN11dffWXOGz16tD799FMtWrRIa9eu1aFDh3TfffeZ8wsKChQXF6czZ85o/fr1mj9/vubNm6cJEyaYNXv37lVcXJy6dOmirVu3atSoUXrkkUe0cuVKs2bhwoVKTEzUxIkTtXnzZrVs2VIOh0NHjhy5OjsBAGCiLwCAZ/IyDMNw18onTZqkxYsXa+vWrcXm5eXlqXbt2lqwYIH69OkjScrIyFCTJk2Umpqq9u3ba8WKFerZs6cOHTqksLAwSdLs2bM1duxYZWdny8/PT2PHjtWyZcuUnp5uLrtfv37Kzc1VcnKyJCk6Olrt2rXTjBkzJEmFhYWKiIjQyJEjNW7cuMvaFqfTKbvdrry8PNlstlLth8hxy0pVb8XPL8VdtXUBqFisHKeulmupL0ju2+dXs69ciB4DeJbyPka5/YzF7t27VbduXV1//fXq37+/9u3bJ0lKS0vT2bNnFRsba9Y2btxY9evXV2pqqiQpNTVVzZs3N5uHJDkcDjmdTu3cudOsuXAZRTVFyzhz5ozS0tJcary9vRUbG2vWlCQ/P19Op9PlBQCwzlP7gkRvAFC5uTVYREdHa968eUpOTtabb76pvXv36vbbb9fx48eVmZkpPz8/hYSEuHwmLCxMmZmZkqTMzEyX5lE0v2jeH9U4nU6dOnVKR48eVUFBQYk1RcsoyeTJk2W3281XRERE2XYCAMDkyX1BojcAqNyquHPl3bt3N/+9RYsWio6OVoMGDfTBBx8oMDDQjSO7tKSkJCUmJprvnU4nDQQALPLkviDRGwBUbm7/KdSFQkJCdNNNN2nPnj0KDw/XmTNnlJub61KTlZWl8PBwSVJ4eHixu4EUvb9Ujc1mU2BgoGrVqiUfH58Sa4qWURJ/f3/ZbDaXFwCgfHlSX5DoDQAqtwoVLE6cOKEff/xRderUUZs2beTr66uUlBRz/q5du7Rv3z7FxMRIkmJiYrRjxw6Xu3SsWrVKNptNTZs2NWsuXEZRTdEy/Pz81KZNG5eawsJCpaSkmDUAAPegLwCA53BrsHjyySe1du1a/fzzz1q/fr3uvfde+fj46IEHHpDdbtfgwYOVmJioL774QmlpaRo0aJBiYmLUvn17SVK3bt3UtGlTPfTQQ9q2bZtWrlypZ555RgkJCfL395ckDRs2TD/99JPGjBmjjIwMzZo1Sx988IFGjx5tjiMxMVH//Oc/NX/+fH3//fcaPny4Tp48qUGDBrllvwBAZUVfAADP5dZrLA4cOKAHHnhAx44dU+3atdWxY0d98803ql27tiTptddek7e3t3r37q38/Hw5HA7NmjXL/LyPj4+WLl2q4cOHKyYmRkFBQYqPj9dzzz1n1kRFRWnZsmUaPXq0pk+frnr16umtt96Sw+Ewa/r27avs7GxNmDBBmZmZatWqlZKTk4tduAcAuLLoCwDgudz6HItrCc+xAFDRecJzLK41PMcCQEV2zT3HAgAAAIDnI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMCyChMsXnrpJXl5eWnUqFHmtNOnTyshIUE1a9ZUcHCwevfuraysLJfP7du3T3FxcapatapCQ0P11FNP6dy5cy41a9asUevWreXv76+GDRtq3rx5xdY/c+ZMRUZGKiAgQNHR0dq4ceMV2U4AwOWjNwCA56gQwWLTpk2aM2eOWrRo4TJ99OjR+vTTT7Vo0SKtXbtWhw4d0n333WfOLygoUFxcnM6cOaP169dr/vz5mjdvniZMmGDW7N27V3FxcerSpYu2bt2qUaNG6ZFHHtHKlSvNmoULFyoxMVETJ07U5s2b1bJlSzkcDh05cuTKbzwAoET0BgDwLF6GYRjuHMCJEyfUunVrzZo1Sy+88IJatWqladOmKS8vT7Vr19aCBQvUp08fSVJGRoaaNGmi1NRUtW/fXitWrFDPnj116NAhhYWFSZJmz56tsWPHKjs7W35+fho7dqyWLVum9PR0c539+vVTbm6ukpOTJUnR0dFq166dZsyYIUkqLCxURESERo4cqXHjxl3WdjidTtntduXl5clms5VqH0SOW1aqeit+finuqq0LQMVi5Th1tdEbrLmafeVC9BjAs5T3McrtZywSEhIUFxen2NhYl+lpaWk6e/asy/TGjRurfv36Sk1NlSSlpqaqefPmZuOQJIfDIafTqZ07d5o1v1+2w+Ewl3HmzBmlpaW51Hh7eys2NtasAQBcXfQGAPA8Vdy58vfff1+bN2/Wpk2bis3LzMyUn5+fQkJCXKaHhYUpMzPTrLmwcRTNL5r3RzVOp1OnTp3Sr7/+qoKCghJrMjIyLjr2/ODJhqcAACAASURBVPx85efnm++dTuelNhcAcBnoDQDgmdx2xmL//v164okn9O677yogIMBdwyizyZMny263m6+IiAh3DwkAPB69AQA8l9uCRVpamo4cOaLWrVurSpUqqlKlitauXavXX39dVapUUVhYmM6cOaPc3FyXz2VlZSk8PFySFB4eXuxOIEXvL1Vjs9kUGBioWrVqycfHp8SaomWUJCkpSXl5eeZr//79ZdsRAAATvQEAPJfbgsVdd92lHTt2aOvWrearbdu26t+/v/nvvr6+SklJMT+za9cu7du3TzExMZKkmJgY7dixw+UOHatWrZLNZlPTpk3NmguXUVRTtAw/Pz+1adPGpaawsFApKSlmTUn8/f1ls9lcXgAAa+gNAOC53HaNRbVq1XTzzTe7TAsKClLNmjXN6YMHD1ZiYqJq1Kghm82mkSNHKiYmRu3bt5ckdevWTU2bNtVDDz2kKVOmKDMzU88884wSEhLk7+8vSRo2bJhmzJihMWPG6OGHH9bq1av1wQcfaNmy/94xIzExUfHx8Wrbtq1uvfVWTZs2TSdPntSgQYOu0t4AAEj0BgDwZG69ePtSXnvtNXl7e6t3797Kz8+Xw+HQrFmzzPk+Pj5aunSphg8frpiYGAUFBSk+Pl7PPfecWRMVFaVly5Zp9OjRmj59uurVq6e33npLDofDrOnbt6+ys7M1YcIEZWZmqlWrVkpOTi520R4AwP3oDQBQMbn9ORbXCp5jAaCi86TnWFwreI4FgIrsmnuOBQAAAADPR7AAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYFmZgsX111+vY8eOFZuem5ur66+/3vKgAACehb4AAChTsPj5559VUFBQbHp+fr4OHjxoeVAAAM9CXwAAVClN8ZIlS8x/X7lypex2u/m+oKBAKSkpioyMLLfBAQAqNvoCAKBIqYJFr169JEleXl6Kj493mefr66vIyEi98sor5Tc6AECFRl8AABQpVbAoLCyUJEVFRWnTpk2qVavWFRkUAMAz0BcAAEVKFSyK7N27t7zHAQDwYPQFAECZgoUkpaSkKCUlRUeOHDG/sSry9ttvWx4YAMCz0BcAoHIrU7B49tln9dxzz6lt27aqU6eOvLy8yntcAAAPQl8AAJQpWMyePVvz5s3TQw89VN7jAQB4IPoCAKBMz7E4c+aMbrvttvIeCwDAQ9EXAABlChaPPPKIFixYUN5jAQB4KPoCAKBMP4U6ffq05s6dq88//1wtWrSQr6+vy/xXX321XAYHAPAM9AUAQJmCxfbt29WqVStJUnp6uss8LtgDgMqHvgAAKFOw+OKLL8p7HAAAD0ZfAACU6RqL8vLmm2+qRYsWstlsstlsiomJ0YoVK8z5p0+fVkJCgmrWrKng4GD17t1bWVlZLsvYt2+f4uLiVLVqVYWGhuqpp57SuXPnXGrWrFmj1q1by9/fXw0bNtS8efOKjWXmzJmKjIxUQECAoqOjtXHjxiuyzQCAi6MvAIDnKtMZiy5duvzhqe3Vq1df1nLq1aunl156STfeeKMMw9D8+fP15z//WVu2bFGzZs00evRoLVu2TIsWLZLdbteIESN033336euvv5YkFRQUKC4uTuHh4Vq/fr0OHz6sAQMGyNfXVy+++KKk80+DjYuL07Bhw/Tuu+8qJSVFjzzyiOrUqSOHwyFJWrhwoRITEzV79mxFR0dr2rRpcjgc2rVrl0JDQ8uyiwCgUqEvAAC8DMMwSvuh0aNHu7w/e/astm7dqvT0dMXHx2v69OllHlCNGjU0depU9enTR7Vr19aCBQvUp08fSVJGRoaaNGmi1NRUtW/fXitWrFDPnj116NAhhYWFSTp/L/WxY8cqOztbfn5+Gjt2rJYtW+bym99+/fopNzdXycnJkqTo6Gi1a9dOM2bMkCQVFhYqIiJCI0eO1Lhx4y5r3E6nU3a7XXl5ebLZbKXa5shxy0pVb8XPL8VdtXUBqFisHKcuhb5Qsiu5z//I1ewrF6LHAJ6lvI9RZTpj8dprr5U4fdKkSTpx4kSZBlJQUKBFixbp5MmTiomJUVpams6ePavY2FizpnHjxqpfv77ZQFJTU9W8eXOzeUiSw+HQ8OHDtXPnTt1yyy1KTU11WUZRzahRoySdv/d6WlqakpKSzPne3t6KjY1VampqmbYFACob+gIAoFyvsfjrX/+qt99+u1Sf2bFjh4KDg+Xv769hw4bpk08+UdOmTZWZmSk/Pz+FhIS41IeFhSkzM1OSlJmZ6dI8iuYXzfujGqfTqVOnTuno0aMqKCgosaZoGSXJz8+X0+l0eQEAXFWmviDRGwBUbuUaLFJTUxUQEFCqzzRq1Ehbt27Vhg0bNHz4cMXHx+u7774rz2FdEZMnT5bdbjdfERER7h4SAFQ4lakvSPQGAJVbmX4Kdd9997m8NwxDhw8f1rfffqvx48eXall+fn5q2LChJKlNmzbatGmTpk+frr59++rMmTPKzc11+XYqKytL4eHhkqTw8PBid+koujvIhTW/v2NIVlaWbDabAgMD5ePjIx8fnxJripZRkqSkJCUmJprvnU4nDQRApUVfOI/eAKAyK9MZiwu/jbHb7apRo4Y6d+6s5cuXa+LEiZYGVFhYqPz8fLVp00a+vr5KSUkx5+3atUv79u1TTEyMJCkmJkY7duzQkSNHzJpVq1bJZrOpadOmZs2FyyiqKVqGn5+f2rRp41JTWFiolJQUs6Yk/v7+5u0Qi14AUFnRF86jNwCozMp0xuKdd94pl5UnJSWpe/fuql+/vo4fP64FCxZozZo1Wrlypex2uwYPHqzExETVqFFDNptNI0eOVExMjNq3by9J6tatm5o2baqHHnpIU6ZMUWZmpp555hklJCTI399fkjRs2DDNmDFDY8aM0cMPP6zVq1frgw8+0LJl/71jRmJiouLj49W2bVvdeuutmjZtmk6ePKlBgwaVy3YCwLWOvgAAKFOwKJKWlqbvv/9ektSsWTPdcsstpfr8kSNHNGDAAB0+fFh2u10tWrTQypUr1bVrV0nn7zLi7e2t3r17Kz8/Xw6HQ7NmzTI/7+Pjo6VLl2r48OGKiYlRUFCQ4uPj9dxzz5k1UVFRWrZsmUaPHq3p06erXr16euutt8x7lUtS3759lZ2drQkTJigzM1OtWrVScnJysQv3AAB/jL4AAJVXmZ5jceTIEfXr109r1qwxf+eam5urLl266P3331ft2rXLfaAVHc+xAFDRXclnKtAXSsZzLABUZOV9jCrTNRYjR47U8ePHtXPnTuXk5CgnJ0fp6elyOp16/PHHLQ8KAOBZ6AsAgDL9FCo5OVmff/65mjRpYk5r2rSpZs6cqW7dupXb4AAAnoG+AAAo0xmLwsJC+fr6Fpvu6+urwsJCy4MCAHgW+gIAoEzB4s4779QTTzyhQ4cOmdMOHjyo0aNH66677iq3wQEAPAN9AQBQpmAxY8YMOZ1ORUZG6oYbbtANN9ygqKgoOZ1OvfHGG+U9RgBABUdfAACU6RqLiIgIbd68WZ9//rkyMjIkSU2aNFFsbGy5Dg4A4BnoCwCAUp2xWL16tZo2bSqn0ykvLy917dpVI0eO1MiRI9WuXTs1a9ZMX3755ZUaKwCggqEvAACKlCpYTJs2TUOGDCnxPrd2u11Dhw7Vq6++Wm6DAwBUbPQFAECRUgWLbdu26e67777o/G7duiktLc3yoAAAnoG+AAAoUqpgkZWVVeLtBItUqVJF2dnZlgcFAPAM9AUAQJFSBYvrrrtO6enpF52/fft21alTx/KgAACegb4AAChSqmDRo0cPjR8/XqdPny4279SpU5o4caJ69uxZboMDAFRs9AUAQJFS3W72mWee0ccff6ybbrpJI0aMUKNGjSRJGRkZmjlzpgoKCvT0009fkYECACoe+gIAoEipgkVYWJjWr1+v4cOHKykpSYZhSJK8vLzkcDg0c+ZMhYWFXZGBAgAqHvoCAKBIqR+Q16BBAy1fvly//vqr9uzZI8MwdOONN6p69epXYnwAgAqOvgAAkMr45G1Jql69utq1a1eeYwEAeDD6AgBUbqW6eBsAAAAASkKwAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWuTVYTJ48We3atVO1atUUGhqqXr16adeuXS41p0+fVkJCgmrWrKng4GD17t1bWVlZLjX79u1TXFycqlatqtDQUD311FM6d+6cS82aNWvUunVr+fv7q2HDhpo3b16x8cycOVORkZEKCAhQdHS0Nm7cWO7bDAC4OPoCAHgutwaLtWvXKiEhQd98841WrVqls2fPqlu3bjp58qRZM3r0aH366adatGiR1q5dq0OHDum+++4z5xcUFCguLk5nzpzR+vXrNX/+fM2bN08TJkwwa/bu3au4uDh16dJFW7du1ahRo/TII49o5cqVZs3ChQuVmJioiRMnavPmzWrZsqUcDoeOHDlydXYGAIC+AAAezMswDMPdgyiSnZ2t0NBQrV27VnfccYfy8vJUu3ZtLViwQH369JEkZWRkqEmTJkpNTVX79u21YsUK9ezZU4cOHVJYWJgkafbs2Ro7dqyys7Pl5+ensWPHatmyZUpPTzfX1a9fP+Xm5io5OVmSFB0drXbt2mnGjBmSpMLCQkVERGjkyJEaN27cJcfudDplt9uVl5cnm81Wqu2OHLesVPVW/PxS3FVbF4CKxcpxyl08uS9I7tvnV7OvXIgeA3iW8j5GVahrLPLy8iRJNWrUkCSlpaXp7Nmzio2NNWsaN26s+vXrKzU1VZKUmpqq5s2bm81DkhwOh5xOp3bu3GnWXLiMopqiZZw5c0ZpaWkuNd7e3oqNjTVrfi8/P19Op9PlBQAoX57UFyR6A4DKrcIEi8LCQo0aNUodOnTQzTffLEnKzMyUn5+fQkJCXGrDwsKUmZlp1lzYPIrmF837oxqn06lTp07p6NGjKigoKLGmaBm/N3nyZNntdvMVERFRxi0HAJTE0/qCRG8AULlVmGCRkJCg9PR0vf/+++4eymVJSkpSXl6e+dq/f7+7hwQA1xRP6wsSvQFA5VbF3QOQpBEjRmjp0qVat26d6tWrZ04PDw/XmTNnlJub6/LtVFZWlsLDw82a39+lo+juIBfW/P6OIVlZWbLZbAoMDJSPj498fHxKrClaxu/5+/vL39+/jFsMAPgjntgXJHoDgMrNrWcsDMPQiBEj9Mknn2j16tWKiopymd+mTRv5+voqJSXFnLZr1y7t27dPMTExkqSYmBjt2LHD5S4dq1atks1mU9OmTc2aC5dRVFO0DD8/P7Vp08alprCwUCkpKWYNAODKoy8AgOdy6xmLhIQELViwQP/5z39UrVo183erdrtdgYGBstvtGjx4sBITE1WjRg3ZbDaNHDlSMTExat++vSSpW7duatq0qR566CFNmTJFmZmZeuaZZ5SQkGB+azRs2DDNmDFDY8aM0cMPP6zVq1frgw8+0LJl/71rRmJiouLj49W2bVvdeuutmjZtmk6ePKlBgwZd/R0DAJUUfQEAPJdbg8Wbb74pSercubPL9HfeeUcDBw6UJL322mvy9vZW7969lZ+fL4fDoVmzZpm1Pj4+Wrp0qYYPH66YmBgFBQUpPj5ezz33nFkTFRWlZcuWafTo0Zo+fbrq1aunt956Sw6Hw6zp27evsrOzNWHCBGVmZqpVq1ZKTk4uduEeAODKoS8AgOeqUM+x8GQ8xwJAReeJz7HwdDzHAkBFdk0/xwIAAACAZyJYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwLIq7h4AAAC4NkSOW+aW9f78Upxb1gvAFWcsAAAAAFhGsAAAAABgGcECAAAAgGVuDRbr1q3TPffco7p168rLy0uLFy92mW8YhiZMmKA6deooMDBQsbGx2r17t0tNTk6O+vfvL5vNppCQEA0ePFgnTpxwqdm+fbtuv/12BQQEKCIiQlOmTCk2lkWLFqlx48YKCAhQ8+bNtXz58vLfYADAJdEbAMAzuTVYnDx5Ui1bttTMmTNLnD9lyhS9/vrrmj17tjZs2KCgoCA5HA6dPn3arOnfv7927typVatWaenSpVq3bp0effRRc77T6VS3bt3UoEEDpaWlaerUqZo0aZLmzp1r1qxfv14PPPCABg8erC1btqhXr17q1auX0tPTr9zGAwBKRG8AAM/kZRiG4e5BSJKXl5c++eQT9erVS9L5b6Tq1q2rv/3tb3ryySclSXl5eQoLC9O8efPUr18/ff/992ratKk2bdqktm3bSpKSk5PVo0cPHThwQHXr1tWbb76pp59+WpmZmfLz85MkjRs3TosXL1ZGRoYkqW/fvjp58qSWLl1qjqd9+/Zq1aqVZs+efVnjdzqdstvtysvLk81mK9W2X827aHDnDKDysnKccpfK3BuscNfdmdyF3gaUTXkfoyrsNRZ79+5VZmamYmNjzWl2u13R0dFKTU2VJKWmpiokJMRsHJIUGxsrb29vbdiwway54447zMYhSQ6HQ7t27dKvv/5q1ly4nqKaovUAACoGegMAVFwV9jkWmZmZkqSwsDCX6WFhYea8zMxMhYaGusyvUqWKatSo4VITFRVVbBlF86pXr67MzMw/XE9J8vPzlZ+fb753Op2l2TwAQBnQGwCg4qqwZywqusmTJ8tut5uviIgIdw8JAOBm9AYAlVmFDRbh4eGSpKysLJfpWVlZ5rzw8HAdOXLEZf65c+eUk5PjUlPSMi5cx8VqiuaXJCkpSXl5eeZr//79pd1EAEAp0RsAoOKqsMEiKipK4eHhSklJMac5nU5t2LBBMTExkqSYmBjl5uYqLS3NrFm9erUKCwsVHR1t1qxbt05nz541a1atWqVGjRqpevXqZs2F6ymqKVpPSfz9/WWz2VxeAIAri94AABWXW4PFiRMntHXrVm3dulXS+Yvytm7dqn379snLy0ujRo3SCy+8oCVLlmjHjh0aMGCA6tata94dpEmTJrr77rs1ZMgQbdy4UV9//bVGjBihfv36qW7dupKkBx98UH5+fho8eLB27typhQsXavr06UpMTDTH8cQTTyg5OVmvvPKKMjIyNGnSJH377bcaMWLE1d8pAFDJ0RsAwDO59eLtb7/9Vl26dDHfFx3Q4+PjNW/ePI0ZM0YnT57Uo48+qtzcXHXs2FHJyckKCAgwP/Puu+9qxIgRuuuuu+Tt7a3evXvr9ddfN+fb7XZ99tlnSkhIUJs2bVSrVi1NmDDB5X7mt912mxYsWKBnnnlGf//733XjjTdq8eLFuvnmm6/CXgAAXIjeAACeqcI8x8LT8RwLABWdJz7HwtPxHIurg94GlE2leY4FAAAAAM9BsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhWxd0DAAAAsCJy3DK3rPfnl+Lcsl6gouKMBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyHpAHAABQBu54MF9leygfDz/0LJyxAAAAAGAZwQIAAACAZQQLAAAAAJZxjQUAAICH4JqDa5c7/mwL838r1+URLAAAAPCH3BVo4Fn4KdTvzJw5U5GRkQoICFB0dLQ2btzo7iEBANyIvgAAl4czFhdYuHChEhMTNXv2bEVHR2vatGlyOBzatWuXQkND3T08AMBVRl8AKifO0JQNZywu8Oqrr2rIkCEaNGiQmjZtqtmzZ6tq1ap6++233T00AIAb0BcA4PIRLP7PmTNnlJaWptjYWHOat7e3YmNjlZqa6saRAQDcgb4AAKXDT6H+z9GjR1VQUKCwsDCX6WFhYcrIyChWn5+fr/z8fPN9Xl6eJMnpdJZ63eV9Rf4fqT960VVb19WU/qzD3UMAKryi45NhGG4eiWcobV+Qyrc3WHE1+woAz1V0rCivvkCwKKPJkyfr2WefLTY9IiLCDaOBfZq7RwB4juPHj8tut7t7GNckegMAT3Ts2LFy6QsEi/9Tq1Yt+fj4KCsry2V6VlaWwsPDi9UnJSUpMTHRfF9YWKicnBzVrFlTXl5el71ep9OpiIgI7d+/XzabrewbUMGwXZ6F7fIsZd0uwzB0/Phx1a1b9wqO7tpR2r4gFe8Nubm5atCggfbt23fNh7lr9b+3i6lM21uZtlWqXNubl5en+vXrq0aNGuWyPILF//Hz81ObNm2UkpKiXr16STofFlJSUjRixIhi9f7+/vL393eZFhISUub122y2a/IvL9vlWdguz1KW7brW/+e2PJW2L0gl9wbp/H6/Fv8OluRa/e/tYirT9lambZUq1/Z6e5fPZdcEiwskJiYqPj5ebdu21a233qpp06bp5MmTGjRokLuHBgBwA/oCAFw+gsUF+vbtq+zsbE2YMEGZmZlq1aqVkpOTi124BwCoHOgLAHD5fCZNmjTJ3YOoSG699VaNHj1a48eP15AhQ1SvXr0rvk4fHx917txZVapcWzmP7fIsbJdnuVa3qyKy2hcq059VZdpWqXJtb2XaVqlybW95bquXwX0HAQAAAFjEA/IAAAAAWEawAAAAAGAZwQIAAACAZQQLN5o5c6YiIyMVEBCg6Ohobdy40d1D+kOTJk2Sl5eXy6tx48bm/NOnTyshIUE1a9ZUcHCwevfuXezBUvv27VNcXJyqVq2q0NBQPfXUUzp37txV3Y5169bpnnvuUd26deXl5aXFixe7zDcMQxMmTFCdOnUUGBio2NhY7d6926UmJydH/fv3l81mU0hIiAYPHqwTJ0641Gzfvl233367AgICFBERoSlTprh1uwYOHFjsz+/uu++u8Ns1efJktWvXTtWqVVNoaKh69eqlXbt2udSU19+9NWvWqHXr1vL391fDhg01b948t25X586di/2ZDRs2rEJvF/7L047xZXU5f5evVS+99JK8vLw0atQodw/lijl48KD++te/qmbNmgoMDFTz5s317bffuntY5a6goEDjx49XVFSUAgMDdcMNN+j555/XtXIpcnn8v89lMeAW77//vuHn52e8/fbbxs6dO40hQ4YYISEhRlZWlruHdlETJ040mjVrZhw+fNh8ZWdnm/OHDRtmREREGCkpKca3335rtG/f3rjtttvM+efOnTNuvvlmIzY21tiyZYuxfPlyo1atWkZSUtJV3Y7ly5cbTz/9tPHxxx8bkoxPPvnEZf5LL71k2O12Y/Hixca2bduMP/3pT0ZUVJRx6tQps+buu+82WrZsaXzzzTfGl19+aTRs2NB44IEHzPl5eXlGWFiY0b9/fyM9Pd147733jMDAQGPOnDlu2674+Hjj7rvvdvnzy8nJcampiNvlcDiMd955x0hPTze2bt1q9OjRw6hfv75x4sQJs6Y8/u799NNPRtWqVY3ExETju+++M9544w3Dx8fHSE5Odtt2derUyRgyZIjLn1leXl6F3i6c54nH+LK6nL/L16KNGzcakZGRRosWLYwnnnjC3cO5InJycowGDRoYAwcONDZs2GD89NNPxsqVK409e/a4e2jl7n/+f3t3HxRV9f8B/A27LIKyPMjDAg4EZAsposjIbBozCYqONRQzisUYWuqkOKkpYpqVFVrTjDPZJJZNmoaazmiWiQ3y4NOQIgFKEgpiWkmUhkCAIvv5/eGwv/YLKrq7Xh7erxlm4J7DvZ/P7r33nA+7nM3IkMGDB8v+/fulpqZGdu/eLYMGDZKPPvpI6dCswhpzn+5gYaGQMWPGSGpqqunn9vZ28fPzk7Vr1yoY1d299dZbEhER0WVbfX29ODg4yO7du03bKioqBIAUFhaKyO2T2t7eXmpra019MjMzRavVyo0bN2wb/B3878VlNBpFp9PJhx9+aNpWX18vjo6OsmPHDhEROXv2rACQoqIiU5/s7Gyxs7OT33//XURENmzYIO7u7mZ5paeni16vt3VKItI5L5HbhUVCQsIdf6c35CUiUldXJwDk8OHDImK9c2/ZsmUybNgws2MlJSVJfHy8rVMSkc55idwuLO42YekNefVXvfEeby1dnct9TWNjowwdOlRycnLueZ32Zunp6TJu3Dilw3gopkyZIi+99JLZtsTERElOTlYoItt5KXfneQAADVlJREFUkLlPd/GtUAq4efMmiouLERcXZ9pmb2+PuLg4FBYWKhjZvZ0/fx5+fn4IDg5GcnIyLl26BAAoLi5GW1ubWU6hoaEICAgw5VRYWIjw8HCzD5aKj49HQ0MDfv7554ebyB3U1NSgtrbWLA9XV1dER0eb5eHm5oaoqChTn7i4ONjb2+PEiROmPjExMdBoNKY+8fHxqKysxD///POQsumsoKAA3t7e0Ov1mDdvHq5evWpq6y15Xb9+HQDg4eEBwHrnXmFhodk+Ovo8rGvyf/PqkJWVBU9PTwwfPhyvv/46mpubTW29Ia/+qDff463hTudyX5KamoopU6Z0urb6mm+//RZRUVGYOnUqvL29MWrUKGzatEnpsGziiSeeQG5uLs6dOwcAKCsrw7FjxzB58mSFI7O97sx9uqvvf+pHD/T333+jvb290ye3+vj44JdfflEoqnuLjo7Gli1boNfrceXKFaxevRpPPvkkysvLUVtbC41GAzc3N7Pf8fHxQW1tLQCgtra2y5w72nqCjji6ivO/eXh7e5u1q9VqeHh4mPUJCgrqtI+ONnd3d5vEfzeTJk1CYmIigoKCUF1djRUrVmDy5MkoLCyESqXqFXkZjUYsWrQIY8eOxfDhw03Htca5d6c+DQ0NaGlpgZOTk01yArrOCwBeeOEFBAYGws/PD6dPn0Z6ejoqKyuxZ8+eXpFXf9Vb7/HWcKdzuS/ZuXMnfvrpJxQVFSkdis1duHABmZmZeO2117BixQoUFRXh1VdfhUajQUpKitLhWdXy5cvR0NCA0NBQqFQqtLe3IyMjA8nJyUqHZnPdmft0FwsL6rb/Vu0jRoxAdHQ0AgMDsWvXLk5OeoHp06ebvg8PD8eIESMQEhKCgoICxMbGKhhZ96WmpqK8vBzHjh1TOhSrulNec+fONX0fHh4OX19fxMbGorq6GiEhIQ87TKJ76qvXaIfLly9j4cKFyMnJwYABA5QOx+aMRiOioqKwZs0aAMCoUaNQXl6OjRs39rnCYteuXcjKysL27dsxbNgwlJaWYtGiRfDz8+tzudoS3wqlAE9PT6hUqk6r1vz555/Q6XQKRXX/3Nzc8Nhjj6Gqqgo6nQ43b95EfX29WZ//5qTT6brMuaOtJ+iI427PjU6nQ11dnVn7rVu3cO3atV6Va3BwMDw9PVFVVQWg5+e1YMEC7N+/H/n5+RgyZIhpu7XOvTv10Wq1Ni2c75RXV6KjowHA7DnrqXn1Z33lHn+/7udc7q2Ki4tRV1eHyMhIqNVqqNVqHD58GOvXr4darUZ7e7vSIVqVr68vHn/8cbNtYWFhprdB9yVpaWlYvnw5pk+fjvDwcMyYMQOLFy/G2rVrlQ7N5roz9+kuFhYK0Gg0GD16NHJzc03bjEYjcnNzYTAYFIzs/jQ1NaG6uhq+vr4YPXo0HBwczHKqrKzEpUuXTDkZDAacOXPGbPKak5MDrVbb6callKCgIOh0OrM8GhoacOLECbM86uvrUVxcbOqTl5cHo9FomvgZDAYcOXIEbW1tpj45OTnQ6/WKvA2qK7/99huuXr0KX19fAD03LxHBggULsHfvXuTl5XV6K5a1zj2DwWC2j44+trom75VXV0pLSwHA7DnraXlR37nHd9eDnMu9VWxsLM6cOYPS0lLTV1RUFJKTk1FaWgqVSqV0iFY1duzYTksHnzt3DoGBgQpFZDvNzc2wtzefFqtUKhiNRoUieni6M/fpNuv8fzndr507d4qjo6Ns2bJFzp49K3PnzhU3Nzez1V16miVLlkhBQYHU1NTI8ePHJS4uTjw9PaWurk5Ebi/5GRAQIHl5eXLq1CkxGAxiMBhMv9+xNObEiROltLRUDh48KF5eXg99udnGxkYpKSmRkpISASDr1q2TkpIS+fXXX0Xk9pJrbm5usm/fPjl9+rQkJCR0udzsqFGj5MSJE3Ls2DEZOnSo2bKs9fX14uPjIzNmzJDy8nLZuXOnODs723RZ1rvl1djYKEuXLpXCwkKpqamRQ4cOSWRkpAwdOlRaW1t7dF7z5s0TV1dXKSgoMFt2tbm52dTHGudex7KsaWlpUlFRIZ988olNl2W9V15VVVXyzjvvyKlTp6Smpkb27dsnwcHBEhMT06Pzott64z3+QXXnGu3L+vKqUCdPnhS1Wi0ZGRly/vx5ycrKEmdnZ/nqq6+UDs3qUlJSxN/f37Tc7J49e8TT01OWLVumdGhWYY25T3ewsFDQxx9/LAEBAaLRaGTMmDHy448/Kh3SXSUlJYmvr69oNBrx9/eXpKQks7WsW1paZP78+eLu7i7Ozs7y3HPPyZUrV8z2cfHiRZk8ebI4OTmJp6enLFmyRNra2h5qHvn5+QKg01dKSoqI3F52bdWqVeLj4yOOjo4SGxsrlZWVZvu4evWqPP/88zJo0CDRarUya9YsaWxsNOtTVlYm48aNE0dHR/H395f3339fsbyam5tl4sSJ4uXlJQ4ODhIYGChz5szpNMnpiXl1lRMA2bx5s6mPtc69/Px8GTlypGg0GgkODjY7xsPO69KlSxITEyMeHh7i6Ogojz76qKSlpZl9jkVPzIv+X2+7xz+o7lyjfVlfLixERL777jsZPny4ODo6SmhoqHz22WdKh2QTDQ0NsnDhQgkICJABAwZIcHCwrFy5UrHl8K3NGnOf7rAT6SMfKUhERERERIrh/1gQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQEREREZHFWFgQWWDmzJmws7ODnZ0dHBwc4OPjgwkTJuCLL76A0WhUOjwiInrIOC5Qf8bCgshCkyZNwpUrV3Dx4kVkZ2fjqaeewsKFC/H000/j1q1bNjvuzZs3bbZvIiJ6cBwXqL9iYUFkIUdHR+h0Ovj7+yMyMhIrVqzAvn37kJ2djS1btgAA6uvrMXv2bHh5eUGr1WL8+PEoKysz2897770Hb29vuLi4YPbs2Vi+fDlGjhxpap85cyaeffZZZGRkwM/PD3q9HgBw+fJlTJs2DW5ubvDw8EBCQgIuXrxotu/PP/8cYWFhGDBgAEJDQ7FhwwabPiZERP0ZxwXqr1hYENnA+PHjERERgT179gAApk6dirq6OmRnZ6O4uBiRkZGIjY3FtWvXAABZWVnIyMjABx98gOLiYgQEBCAzM7PTfnNzc1FZWYmcnBzs378fbW1tiI+Ph4uLC44ePYrjx49j0KBBmDRpkukvV1lZWXjzzTeRkZGBiooKrFmzBqtWrcKXX3758B4QIqJ+juMC9QtCRA8sJSVFEhISumxLSkqSsLAwOXr0qGi1WmltbTVrDwkJkU8//VRERKKjoyU1NdWsfezYsRIREWF2LB8fH7lx44Zp27Zt20Sv14vRaDRtu3Hjhjg5OckPP/xgOs727dvN9v3uu++KwWB4gIyJiOhuOC5Qf6ZWurAh6qtEBHZ2digrK0NTUxMGDx5s1t7S0oLq6moAQGVlJebPn2/WPmbMGOTl5ZltCw8Ph0ajMf1cVlaGqqoquLi4mPVrbW1FdXU1/v33X1RXV+Pll1/GnDlzTO23bt2Cq6urVfIkIqLu4bhAfR0LCyIbqaioQFBQEJqamuDr64uCgoJOfdzc3O5rnwMHDjT7uampCaNHj0ZWVlanvl5eXmhqagIAbNq0CdHR0WbtKpXqvo5NRESW4bhAfR0LCyIbyMvLw5kzZ7B48WIMGTIEtbW1UKvVeOSRR7rsr9frUVRUhBdffNG0raio6J7HiYyMxNdffw1vb29otdpO7a6urvDz88OFCxeQnJz8wPkQEZFlOC5Qf6B6++2331Y6CKLe6ptvvsH169cRFxeH69evo6qqCps3b0ZqaiomTpyItWvXIiQkBIcOHcLWrVsRHBwMlUqFs2fPYuPGjaYbvJOTE1avXo0hQ4Zg4MCByMzMxN69e+Hh4YFXXnnFdKzW1lZMnz7ddPywsDBs3boV33//PUJCQmA0GlFWVoZ169YhPDwcWq0Wrq6uWLlyJVxcXODu7o4//vgDBw4cQH5+PgwGg1IPHRFRn8RxgfozvmJBZKGDBw/C19cXarUa7u7uiIiIwPr165GSkgJ7+9sLrx04cAArV67ErFmz8Ndff0Gn0yEmJgY+Pj4AgOTkZFy4cAFLly5Fa2srpk2bhpkzZ+LkyZN3PbazszOOHDmC9PR0JCYmorGxEf7+/oiNjTX9pWr27NlwdnbGhx9+iLS0NAwcOBDh4eFYtGiRbR8YIqJ+iuMC9Vd2IiJKB0FEnU2YMAE6nQ7btm1TOhQiIuoBOC5QT8dXLIh6gObmZmzcuBHx8fFQqVTYsWMHDh06hJycHKVDIyIiBXBcoN6Ir1gQ9QAtLS145plnUFJSgtbWVuj1erzxxhtITExUOjQiIlIAxwXqjVhYEBERERGRxeyVDoCIiIiIiHo/FhZERERERGQxFhZERERERGQxFhZERERERGQxFhZERERERGQxFhZERERERGQxFhZERERERGQxFhZERERERGQxFhZERERERGSx/wMdYxl5KoVtNgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 800x400 with 2 Axes>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Br8-5IGIUbVT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d2edd5be-8673-4e25-bd3b-149a28128877"
      },
      "source": [
        "# filter nodes with degree greater than or equal to 2\n",
        "core = [node for node, deg in dict(g.degree()).items() if deg >= 3]\n",
        "\n",
        "# select a subgraph with 'core' nodes\n",
        "gsub = nx.subgraph(g, core)\n",
        "\n",
        "print(\"{} nodes, {} edges\".format(len(gsub), nx.number_of_edges(gsub)))\n",
        "\n",
        "nx.write_graphml(gsub, \"brazil_tourism2.graphml\")"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "15610 nodes, 138883 edges\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GD2AghG1eXMB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dc88838f-bd0c-4d82-dac9-b2331aedf2f0"
      },
      "source": [
        "print(\"Nodes removed: {:.2f}%\".format(100*(1 - 15610/85906)))\n",
        "print(\"Edges removed: {:.2f}%\".format(100*(1 - 138883/221021)))\n",
        "print(\"Edges per nodes: {:.2f}\".format(138883/15610))"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Nodes removed: 81.83%\n",
            "Edges removed: 37.16%\n",
            "Edges per nodes: 8.90\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5zRjfGLtZ2qB"
      },
      "source": [
        "## 1.5 Explore the network"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gwlRxZ2kmDL-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7599c2e8-8756-4ab4-93bb-b941cc9ce467"
      },
      "source": [
        "top_indegree = sorted(dict(gsub.in_degree()).items(),\n",
        "                      reverse=True, key=itemgetter(1))[:20]\n",
        "print(\"\\n\".join(map(lambda t: \"{} {}\".format(*reversed(t)), top_indegree)))"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "382 Brazil\n",
            "298 Geographic Coordinate System\n",
            "268 Regions Of Brazil\n",
            "252 Viaf (Identifier)\n",
            "219 States Of Brazil\n",
            "207 Time In Brazil\n",
            "197 Wayback Machine\n",
            "193 Economy Of Brazil\n",
            "192 Brazilian Real\n",
            "190 Time Zone\n",
            "181 Tourism In Brazil\n",
            "174 Pantanal\n",
            "173 Economic History Of Brazil\n",
            "172 Central Bank Of Brazil\n",
            "172 Energy Policy Of Brazil\n",
            "172 Mining In Brazil\n",
            "171 Industry In Brazil\n",
            "171 Agriculture In Brazil\n",
            "171 Corruption In Brazil\n",
            "171 Transport In Brazil\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}