Segmenting and Clustering Neighborhoods in Toronto copy 1.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# The answer is in section \"Second question\""
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Web scraping from Wikipedia"
        },
        {
            "cell_type": "code",
            "execution_count": 100,
            "metadata": {},
            "outputs": [],
            "source": "# import the library we use to open URLs\nimport urllib.request"
        },
        {
            "cell_type": "code",
            "execution_count": 101,
            "metadata": {},
            "outputs": [],
            "source": "url = \"https://en.wikipedia.org/wiki/List_of_postal_codes_of_Canada:_M\""
        },
        {
            "cell_type": "code",
            "execution_count": 102,
            "metadata": {},
            "outputs": [],
            "source": "# open the url using urllib.request and put the HTML into the page variable\npage = urllib.request.urlopen(url)"
        },
        {
            "cell_type": "code",
            "execution_count": 103,
            "metadata": {},
            "outputs": [],
            "source": "# import the BeautifulSoup library so we can parse HTML and XML documents\nfrom bs4 import BeautifulSoup"
        },
        {
            "cell_type": "code",
            "execution_count": 104,
            "metadata": {},
            "outputs": [],
            "source": "# parse the HTML from our URL into the BeautifulSoup parse tree format\nsoup = BeautifulSoup(page, \"lxml\")"
        },
        {
            "cell_type": "code",
            "execution_count": 105,
            "metadata": {},
            "outputs": [],
            "source": "# use the 'find_all' function to bring back all instances of the 'table' tag in the HTML and store in 'all_tables' variable\nall_tables=soup.find_all(\"table\")"
        },
        {
            "cell_type": "code",
            "execution_count": 106,
            "metadata": {
                "scrolled": false
            },
            "outputs": [],
            "source": "# the table we are searching for is the 'wikitable sortable'. Let's find it\nright_table=soup.find('table', class_='wikitable sortable')\n#right_table"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "#### Create the Dataframe\n\nLet's start looping through the rows to get the data for every borough in the table.\n\nThere are three columns in our table that we want to scrape the data from to store our data in.\n\nLet's use the Beautiful Soup \u2018find_all\u2019 function again and set it to look for the string \u2018tr\u2019. Then let's set up a FOR loop for each row within that array and set Python to loop through the rows, one by one.\n\nWithin the loop I'm going to use use find_all again to search each row for <td> tags with the \u2018td\u2019 string. I'll add all of these to a variable called \u2018cells\u2019 and then check to make sure that there are 3 items in our \u2018cells\u2019 array.\n\nIf there are then we use the find(text=True)) option to extract the content string from within each <td> element in that row and add them to the A-C lists we created at the start of this step. Let\u2019s have a look at the code:"
        },
        {
            "cell_type": "code",
            "execution_count": 107,
            "metadata": {},
            "outputs": [],
            "source": "A=[]\nB=[]\nC=[]\n\nfor row in right_table.findAll('tr'):\n    cells=row.findAll('td')\n    if len(cells)==3:\n        A.append(cells[0].find(text=True))\n        B.append(cells[1].find(text=True))\n        C.append(cells[2].find(text=True))"
        },
        {
            "cell_type": "code",
            "execution_count": 108,
            "metadata": {},
            "outputs": [],
            "source": "import pandas as pd\ndf=pd.DataFrame(A,columns=['Postal Code'])\ndf['Borough']=B\ndf['Neighborhood']=C"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Cleaning the Dataframe"
        },
        {
            "cell_type": "code",
            "execution_count": 109,
            "metadata": {},
            "outputs": [],
            "source": "# Ignore cells with a borough that is Not assigned.\nnew_df = df[~df.Borough.str.contains(\"Not assigned\") == True]"
        },
        {
            "cell_type": "code",
            "execution_count": 110,
            "metadata": {},
            "outputs": [],
            "source": "new_df = new_df.reset_index(drop=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Second question"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Solving environment: done\n\n## Package Plan ##\n\n  environment location: /opt/conda/envs/Python36\n\n  added / updated specs: \n    - geopy\n\n\nThe following packages will be downloaded:\n\n    package                    |            build\n    ---------------------------|-----------------\n    geographiclib-1.50         |             py_0          34 KB  conda-forge\n    geopy-1.22.0               |     pyh9f0ad1d_0          63 KB  conda-forge\n    ------------------------------------------------------------\n                                           Total:          97 KB\n\nThe following NEW packages will be INSTALLED:\n\n    geographiclib: 1.50-py_0           conda-forge\n    geopy:         1.22.0-pyh9f0ad1d_0 conda-forge\n\n\nDownloading and Extracting Packages\ngeographiclib-1.50   | 34 KB     | ##################################### | 100% \ngeopy-1.22.0         | 63 KB     | ##################################### | 100% \nPreparing transaction: done\nVerifying transaction: done\nExecuting transaction: done\nLibraries imported.\n"
                }
            ],
            "source": "import numpy as np # library to handle data in a vectorized manner\n\nimport pandas as pd # library for data analsysis\npd.set_option('display.max_columns', None)\npd.set_option('display.max_rows', None)\n\nimport json # library to handle JSON files\n\n!conda install -c conda-forge geopy --yes # uncomment this line if you haven't completed the Foursquare API lab\nfrom geopy.geocoders import Nominatim # convert an address into latitude and longitude values\n\nimport requests # library to handle requests\nfrom pandas.io.json import json_normalize # tranform JSON file into a pandas dataframe\n\n# Matplotlib and associated plotting modules\nimport matplotlib.cm as cm\nimport matplotlib.colors as colors\n\n# import k-means from clustering stage\nfrom sklearn.cluster import KMeans\n\nprint('Libraries imported.')"
        },
        {
            "cell_type": "code",
            "execution_count": 111,
            "metadata": {},
            "outputs": [],
            "source": "#!conda install -c conda-forge folium=0.5.0 --yes\n#import folium # map rendering library"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Let's download the csv file with our lat and long data"
        },
        {
            "cell_type": "code",
            "execution_count": 70,
            "metadata": {},
            "outputs": [],
            "source": "csv_path = 'http://cocl.us/Geospatial_data'\n\ntor_df = pd.read_csv(csv_path)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Let's merge new_df (with Borough and Neighborhood) with tor_df (with Lat and Long)"
        },
        {
            "cell_type": "code",
            "execution_count": 99,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Postal Code</th>\n      <th>Borough</th>\n      <th>Neighborhood</th>\n      <th>Latitude</th>\n      <th>Longitude</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>M3A</td>\n      <td>North York</td>\n      <td>Parkwoods</td>\n      <td>43.806686</td>\n      <td>-79.194353</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>M4A</td>\n      <td>North York</td>\n      <td>Victoria Village</td>\n      <td>43.784535</td>\n      <td>-79.160497</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>M5A</td>\n      <td>Downtown Toronto</td>\n      <td>Regent Park, Harbourfront</td>\n      <td>43.763573</td>\n      <td>-79.188711</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>M6A</td>\n      <td>North York</td>\n      <td>Lawrence Manor, Lawrence Heights</td>\n      <td>43.770992</td>\n      <td>-79.216917</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>M7A</td>\n      <td>Downtown Toronto</td>\n      <td>Queen's Park, Ontario Provincial Government</td>\n      <td>43.773136</td>\n      <td>-79.239476</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>M9A</td>\n      <td>Etobicoke</td>\n      <td>Islington Avenue, Humber Valley Village</td>\n      <td>43.744734</td>\n      <td>-79.239476</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>M1B</td>\n      <td>Scarborough</td>\n      <td>Malvern, Rouge</td>\n      <td>43.727929</td>\n      <td>-79.262029</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>M3B</td>\n      <td>North York</td>\n      <td>Don Mills</td>\n      <td>43.711112</td>\n      <td>-79.284577</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>M4B</td>\n      <td>East York</td>\n      <td>Parkview Hill, Woodbine Gardens</td>\n      <td>43.716316</td>\n      <td>-79.239476</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>M5B</td>\n      <td>Downtown Toronto</td>\n      <td>Garden District, Ryerson</td>\n      <td>43.692657</td>\n      <td>-79.264848</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>M6B</td>\n      <td>North York</td>\n      <td>Glencairn</td>\n      <td>43.757410</td>\n      <td>-79.273304</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>M9B</td>\n      <td>Etobicoke</td>\n      <td>West Deane Park, Princess Gardens, Martin Grov...</td>\n      <td>43.750072</td>\n      <td>-79.295849</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>M1C</td>\n      <td>Scarborough</td>\n      <td>Rouge Hill, Port Union, Highland Creek</td>\n      <td>43.794200</td>\n      <td>-79.262029</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>M3C</td>\n      <td>North York</td>\n      <td>Don Mills</td>\n      <td>43.781638</td>\n      <td>-79.304302</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>M4C</td>\n      <td>East York</td>\n      <td>Woodbine Heights</td>\n      <td>43.815252</td>\n      <td>-79.284577</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>M5C</td>\n      <td>Downtown Toronto</td>\n      <td>St. James Town</td>\n      <td>43.799525</td>\n      <td>-79.318389</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>M6C</td>\n      <td>York</td>\n      <td>Humewood-Cedarvale</td>\n      <td>43.836125</td>\n      <td>-79.205636</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>M9C</td>\n      <td>Etobicoke</td>\n      <td>Eringate, Bloordale Gardens, Old Burnhamthorpe...</td>\n      <td>43.803762</td>\n      <td>-79.363452</td>\n    </tr>\n    <tr>\n      <th>18</th>\n      <td>M1E</td>\n      <td>Scarborough</td>\n      <td>Guildwood, Morningside, West Hill</td>\n      <td>43.778517</td>\n      <td>-79.346556</td>\n    </tr>\n    <tr>\n      <th>19</th>\n      <td>M4E</td>\n      <td>East Toronto</td>\n      <td>The Beaches</td>\n      <td>43.786947</td>\n      <td>-79.385975</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>M5E</td>\n      <td>Downtown Toronto</td>\n      <td>Berczy Park</td>\n      <td>43.757490</td>\n      <td>-79.374714</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>M6E</td>\n      <td>York</td>\n      <td>Caledonia-Fairbanks</td>\n      <td>43.789053</td>\n      <td>-79.408493</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>M1G</td>\n      <td>Scarborough</td>\n      <td>Woburn</td>\n      <td>43.770120</td>\n      <td>-79.408493</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>M4G</td>\n      <td>East York</td>\n      <td>Leaside</td>\n      <td>43.752758</td>\n      <td>-79.400049</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>M5G</td>\n      <td>Downtown Toronto</td>\n      <td>Central Bay Street</td>\n      <td>43.782736</td>\n      <td>-79.442259</td>\n    </tr>\n    <tr>\n      <th>25</th>\n      <td>M6G</td>\n      <td>Downtown Toronto</td>\n      <td>Christie</td>\n      <td>43.753259</td>\n      <td>-79.329656</td>\n    </tr>\n    <tr>\n      <th>26</th>\n      <td>M1H</td>\n      <td>Scarborough</td>\n      <td>Cedarbrae</td>\n      <td>43.745906</td>\n      <td>-79.352188</td>\n    </tr>\n    <tr>\n      <th>27</th>\n      <td>M2H</td>\n      <td>North York</td>\n      <td>Hillcrest Village</td>\n      <td>43.725900</td>\n      <td>-79.340923</td>\n    </tr>\n    <tr>\n      <th>28</th>\n      <td>M3H</td>\n      <td>North York</td>\n      <td>Bathurst Manor, Wilson Heights, Downsview North</td>\n      <td>43.754328</td>\n      <td>-79.442259</td>\n    </tr>\n    <tr>\n      <th>29</th>\n      <td>M4H</td>\n      <td>East York</td>\n      <td>Thorncliffe Park</td>\n      <td>43.767980</td>\n      <td>-79.487262</td>\n    </tr>\n    <tr>\n      <th>30</th>\n      <td>M5H</td>\n      <td>Downtown Toronto</td>\n      <td>Richmond, Adelaide, King</td>\n      <td>43.737473</td>\n      <td>-79.464763</td>\n    </tr>\n    <tr>\n      <th>31</th>\n      <td>M6H</td>\n      <td>West Toronto</td>\n      <td>Dufferin, Dovercourt Village</td>\n      <td>43.739015</td>\n      <td>-79.506944</td>\n    </tr>\n    <tr>\n      <th>32</th>\n      <td>M1J</td>\n      <td>Scarborough</td>\n      <td>Scarborough Village</td>\n      <td>43.728496</td>\n      <td>-79.495697</td>\n    </tr>\n    <tr>\n      <th>33</th>\n      <td>M2J</td>\n      <td>North York</td>\n      <td>Fairview, Henry Farm, Oriole</td>\n      <td>43.761631</td>\n      <td>-79.520999</td>\n    </tr>\n    <tr>\n      <th>34</th>\n      <td>M3J</td>\n      <td>North York</td>\n      <td>Northwood Park, York University</td>\n      <td>43.725882</td>\n      <td>-79.315572</td>\n    </tr>\n    <tr>\n      <th>35</th>\n      <td>M4J</td>\n      <td>East York</td>\n      <td>East Toronto, Broadview North (Old East York)</td>\n      <td>43.706397</td>\n      <td>-79.309937</td>\n    </tr>\n    <tr>\n      <th>36</th>\n      <td>M5J</td>\n      <td>Downtown Toronto</td>\n      <td>Harbourfront East, Union Station, Toronto Isla...</td>\n      <td>43.695344</td>\n      <td>-79.318389</td>\n    </tr>\n    <tr>\n      <th>37</th>\n      <td>M6J</td>\n      <td>West Toronto</td>\n      <td>Little Portugal, Trinity</td>\n      <td>43.676357</td>\n      <td>-79.293031</td>\n    </tr>\n    <tr>\n      <th>38</th>\n      <td>M1K</td>\n      <td>Scarborough</td>\n      <td>Kennedy Park, Ionview, East Birchmount Park</td>\n      <td>43.709060</td>\n      <td>-79.363452</td>\n    </tr>\n    <tr>\n      <th>39</th>\n      <td>M2K</td>\n      <td>North York</td>\n      <td>Bayview Village</td>\n      <td>43.705369</td>\n      <td>-79.349372</td>\n    </tr>\n    <tr>\n      <th>40</th>\n      <td>M3K</td>\n      <td>North York</td>\n      <td>Downsview</td>\n      <td>43.685347</td>\n      <td>-79.338106</td>\n    </tr>\n    <tr>\n      <th>41</th>\n      <td>M4K</td>\n      <td>East Toronto</td>\n      <td>The Danforth West, Riverdale</td>\n      <td>43.679557</td>\n      <td>-79.352188</td>\n    </tr>\n    <tr>\n      <th>42</th>\n      <td>M5K</td>\n      <td>Downtown Toronto</td>\n      <td>Toronto Dominion Centre, Design Exchange</td>\n      <td>43.668999</td>\n      <td>-79.315572</td>\n    </tr>\n    <tr>\n      <th>43</th>\n      <td>M6K</td>\n      <td>West Toronto</td>\n      <td>Brockton, Parkdale Village, Exhibition Place</td>\n      <td>43.659526</td>\n      <td>-79.340923</td>\n    </tr>\n    <tr>\n      <th>44</th>\n      <td>M1L</td>\n      <td>Scarborough</td>\n      <td>Golden Mile, Clairlea, Oakridge</td>\n      <td>43.728020</td>\n      <td>-79.388790</td>\n    </tr>\n    <tr>\n      <th>45</th>\n      <td>M2L</td>\n      <td>North York</td>\n      <td>York Mills, Silver Hills</td>\n      <td>43.712751</td>\n      <td>-79.390197</td>\n    </tr>\n    <tr>\n      <th>46</th>\n      <td>M3L</td>\n      <td>North York</td>\n      <td>Downsview</td>\n      <td>43.715383</td>\n      <td>-79.405678</td>\n    </tr>\n    <tr>\n      <th>47</th>\n      <td>M4L</td>\n      <td>East Toronto</td>\n      <td>India Bazaar, The Beaches West</td>\n      <td>43.704324</td>\n      <td>-79.388790</td>\n    </tr>\n    <tr>\n      <th>48</th>\n      <td>M5L</td>\n      <td>Downtown Toronto</td>\n      <td>Commerce Court, Victoria Hotel</td>\n      <td>43.689574</td>\n      <td>-79.383160</td>\n    </tr>\n    <tr>\n      <th>49</th>\n      <td>M6L</td>\n      <td>North York</td>\n      <td>North Park, Maple Leaf Park, Upwood Park</td>\n      <td>43.686412</td>\n      <td>-79.400049</td>\n    </tr>\n    <tr>\n      <th>50</th>\n      <td>M9L</td>\n      <td>North York</td>\n      <td>Humber Summit</td>\n      <td>43.679563</td>\n      <td>-79.377529</td>\n    </tr>\n    <tr>\n      <th>51</th>\n      <td>M1M</td>\n      <td>Scarborough</td>\n      <td>Cliffside, Cliffcrest, Scarborough Village West</td>\n      <td>43.667967</td>\n      <td>-79.367675</td>\n    </tr>\n    <tr>\n      <th>52</th>\n      <td>M2M</td>\n      <td>North York</td>\n      <td>Willowdale, Newtonbrook</td>\n      <td>43.665860</td>\n      <td>-79.383160</td>\n    </tr>\n    <tr>\n      <th>53</th>\n      <td>M3M</td>\n      <td>North York</td>\n      <td>Downsview</td>\n      <td>43.654260</td>\n      <td>-79.360636</td>\n    </tr>\n    <tr>\n      <th>54</th>\n      <td>M4M</td>\n      <td>East Toronto</td>\n      <td>Studio District</td>\n      <td>43.657162</td>\n      <td>-79.378937</td>\n    </tr>\n    <tr>\n      <th>55</th>\n      <td>M5M</td>\n      <td>North York</td>\n      <td>Bedford Park, Lawrence Manor East</td>\n      <td>43.651494</td>\n      <td>-79.375418</td>\n    </tr>\n    <tr>\n      <th>56</th>\n      <td>M6M</td>\n      <td>York</td>\n      <td>Del Ray, Mount Dennis, Keelsdale and Silverthorn</td>\n      <td>43.644771</td>\n      <td>-79.373306</td>\n    </tr>\n    <tr>\n      <th>57</th>\n      <td>M9M</td>\n      <td>North York</td>\n      <td>Humberlea, Emery</td>\n      <td>43.657952</td>\n      <td>-79.387383</td>\n    </tr>\n    <tr>\n      <th>58</th>\n      <td>M1N</td>\n      <td>Scarborough</td>\n      <td>Birch Cliff, Cliffside West</td>\n      <td>43.650571</td>\n      <td>-79.384568</td>\n    </tr>\n    <tr>\n      <th>59</th>\n      <td>M2N</td>\n      <td>North York</td>\n      <td>Willowdale, Willowdale East</td>\n      <td>43.640816</td>\n      <td>-79.381752</td>\n    </tr>\n    <tr>\n      <th>60</th>\n      <td>M3N</td>\n      <td>North York</td>\n      <td>Downsview</td>\n      <td>43.647177</td>\n      <td>-79.381576</td>\n    </tr>\n    <tr>\n      <th>61</th>\n      <td>M4N</td>\n      <td>Central Toronto</td>\n      <td>Lawrence Park</td>\n      <td>43.648198</td>\n      <td>-79.379817</td>\n    </tr>\n    <tr>\n      <th>62</th>\n      <td>M5N</td>\n      <td>Central Toronto</td>\n      <td>Roselawn</td>\n      <td>43.733283</td>\n      <td>-79.419750</td>\n    </tr>\n    <tr>\n      <th>63</th>\n      <td>M6N</td>\n      <td>York</td>\n      <td>Runnymede, The Junction North</td>\n      <td>43.711695</td>\n      <td>-79.416936</td>\n    </tr>\n    <tr>\n      <th>64</th>\n      <td>M9N</td>\n      <td>York</td>\n      <td>Weston</td>\n      <td>43.696948</td>\n      <td>-79.411307</td>\n    </tr>\n    <tr>\n      <th>65</th>\n      <td>M1P</td>\n      <td>Scarborough</td>\n      <td>Dorset Park, Wexford Heights, Scarborough Town...</td>\n      <td>43.672710</td>\n      <td>-79.405678</td>\n    </tr>\n    <tr>\n      <th>66</th>\n      <td>M2P</td>\n      <td>North York</td>\n      <td>York Mills West</td>\n      <td>43.662696</td>\n      <td>-79.400049</td>\n    </tr>\n    <tr>\n      <th>67</th>\n      <td>M4P</td>\n      <td>Central Toronto</td>\n      <td>Davisville North</td>\n      <td>43.653206</td>\n      <td>-79.400049</td>\n    </tr>\n    <tr>\n      <th>68</th>\n      <td>M5P</td>\n      <td>Central Toronto</td>\n      <td>Forest Hill North &amp; West, Forest Hill Road Park</td>\n      <td>43.628947</td>\n      <td>-79.394420</td>\n    </tr>\n    <tr>\n      <th>69</th>\n      <td>M6P</td>\n      <td>West Toronto</td>\n      <td>High Park, The Junction South</td>\n      <td>43.646435</td>\n      <td>-79.374846</td>\n    </tr>\n    <tr>\n      <th>70</th>\n      <td>M9P</td>\n      <td>Etobicoke</td>\n      <td>Westmount</td>\n      <td>43.648429</td>\n      <td>-79.382280</td>\n    </tr>\n    <tr>\n      <th>71</th>\n      <td>M1R</td>\n      <td>Scarborough</td>\n      <td>Wexford, Maryvale</td>\n      <td>43.718518</td>\n      <td>-79.464763</td>\n    </tr>\n    <tr>\n      <th>72</th>\n      <td>M2R</td>\n      <td>North York</td>\n      <td>Willowdale, Willowdale West</td>\n      <td>43.709577</td>\n      <td>-79.445073</td>\n    </tr>\n    <tr>\n      <th>73</th>\n      <td>M4R</td>\n      <td>Central Toronto</td>\n      <td>North Toronto West,  Lawrence Park</td>\n      <td>43.693781</td>\n      <td>-79.428191</td>\n    </tr>\n    <tr>\n      <th>74</th>\n      <td>M5R</td>\n      <td>Central Toronto</td>\n      <td>The Annex, North Midtown, Yorkville</td>\n      <td>43.689026</td>\n      <td>-79.453512</td>\n    </tr>\n    <tr>\n      <th>75</th>\n      <td>M6R</td>\n      <td>West Toronto</td>\n      <td>Parkdale, Roncesvalles</td>\n      <td>43.669542</td>\n      <td>-79.422564</td>\n    </tr>\n    <tr>\n      <th>76</th>\n      <td>M7R</td>\n      <td>Mississauga</td>\n      <td>Canada Post Gateway Processing Centre</td>\n      <td>43.669005</td>\n      <td>-79.442259</td>\n    </tr>\n    <tr>\n      <th>77</th>\n      <td>M9R</td>\n      <td>Etobicoke</td>\n      <td>Kingsview Village, St. Phillips, Martin Grove ...</td>\n      <td>43.647927</td>\n      <td>-79.419750</td>\n    </tr>\n    <tr>\n      <th>78</th>\n      <td>M1S</td>\n      <td>Scarborough</td>\n      <td>Agincourt</td>\n      <td>43.636847</td>\n      <td>-79.428191</td>\n    </tr>\n    <tr>\n      <th>79</th>\n      <td>M4S</td>\n      <td>Central Toronto</td>\n      <td>Davisville</td>\n      <td>43.713756</td>\n      <td>-79.490074</td>\n    </tr>\n    <tr>\n      <th>80</th>\n      <td>M5S</td>\n      <td>Downtown Toronto</td>\n      <td>University of Toronto, Harbord</td>\n      <td>43.691116</td>\n      <td>-79.476013</td>\n    </tr>\n    <tr>\n      <th>81</th>\n      <td>M6S</td>\n      <td>West Toronto</td>\n      <td>Runnymede, Swansea</td>\n      <td>43.673185</td>\n      <td>-79.487262</td>\n    </tr>\n    <tr>\n      <th>82</th>\n      <td>M1T</td>\n      <td>Scarborough</td>\n      <td>Clarks Corners, Tam O'Shanter, Sullivan</td>\n      <td>43.661608</td>\n      <td>-79.464763</td>\n    </tr>\n    <tr>\n      <th>83</th>\n      <td>M4T</td>\n      <td>Central Toronto</td>\n      <td>Moore Park, Summerhill East</td>\n      <td>43.648960</td>\n      <td>-79.456325</td>\n    </tr>\n    <tr>\n      <th>84</th>\n      <td>M5T</td>\n      <td>Downtown Toronto</td>\n      <td>Kensington Market, Chinatown, Grange Park</td>\n      <td>43.651571</td>\n      <td>-79.484450</td>\n    </tr>\n    <tr>\n      <th>85</th>\n      <td>M1V</td>\n      <td>Scarborough</td>\n      <td>Milliken, Agincourt North, Steeles East, L'Amo...</td>\n      <td>43.662301</td>\n      <td>-79.389494</td>\n    </tr>\n    <tr>\n      <th>86</th>\n      <td>M4V</td>\n      <td>Central Toronto</td>\n      <td>Summerhill West, Rathnelly, South Hill, Forest...</td>\n      <td>43.636966</td>\n      <td>-79.615819</td>\n    </tr>\n    <tr>\n      <th>87</th>\n      <td>M5V</td>\n      <td>Downtown Toronto</td>\n      <td>CN Tower, King and Spadina, Railway Lands, Har...</td>\n      <td>43.662744</td>\n      <td>-79.321558</td>\n    </tr>\n    <tr>\n      <th>88</th>\n      <td>M8V</td>\n      <td>Etobicoke</td>\n      <td>New Toronto, Mimico South, Humber Bay Shores</td>\n      <td>43.605647</td>\n      <td>-79.501321</td>\n    </tr>\n    <tr>\n      <th>89</th>\n      <td>M9V</td>\n      <td>Etobicoke</td>\n      <td>South Steeles, Silverstone, Humbergate, Jamest...</td>\n      <td>43.602414</td>\n      <td>-79.543484</td>\n    </tr>\n    <tr>\n      <th>90</th>\n      <td>M1W</td>\n      <td>Scarborough</td>\n      <td>Steeles West, L'Amoreaux West</td>\n      <td>43.653654</td>\n      <td>-79.506944</td>\n    </tr>\n    <tr>\n      <th>91</th>\n      <td>M4W</td>\n      <td>Downtown Toronto</td>\n      <td>Rosedale</td>\n      <td>43.636258</td>\n      <td>-79.498509</td>\n    </tr>\n    <tr>\n      <th>92</th>\n      <td>M5W</td>\n      <td>Downtown Toronto</td>\n      <td>Stn A PO Boxes</td>\n      <td>43.628841</td>\n      <td>-79.520999</td>\n    </tr>\n    <tr>\n      <th>93</th>\n      <td>M8W</td>\n      <td>Etobicoke</td>\n      <td>Alderwood, Long Branch</td>\n      <td>43.667856</td>\n      <td>-79.532242</td>\n    </tr>\n    <tr>\n      <th>94</th>\n      <td>M9W</td>\n      <td>Etobicoke</td>\n      <td>Northwest, West Humber - Clairville</td>\n      <td>43.650943</td>\n      <td>-79.554724</td>\n    </tr>\n    <tr>\n      <th>95</th>\n      <td>M1X</td>\n      <td>Scarborough</td>\n      <td>Upper Rouge</td>\n      <td>43.643515</td>\n      <td>-79.577201</td>\n    </tr>\n    <tr>\n      <th>96</th>\n      <td>M4X</td>\n      <td>Downtown Toronto</td>\n      <td>St. James Town, Cabbagetown</td>\n      <td>43.756303</td>\n      <td>-79.565963</td>\n    </tr>\n    <tr>\n      <th>97</th>\n      <td>M5X</td>\n      <td>Downtown Toronto</td>\n      <td>First Canadian Place, Underground city</td>\n      <td>43.724766</td>\n      <td>-79.532242</td>\n    </tr>\n    <tr>\n      <th>98</th>\n      <td>M8X</td>\n      <td>Etobicoke</td>\n      <td>The Kingsway, Montgomery Road, Old Mill North</td>\n      <td>43.706876</td>\n      <td>-79.518188</td>\n    </tr>\n    <tr>\n      <th>99</th>\n      <td>M4Y</td>\n      <td>Downtown Toronto</td>\n      <td>Church and Wellesley</td>\n      <td>43.696319</td>\n      <td>-79.532242</td>\n    </tr>\n    <tr>\n      <th>100</th>\n      <td>M7Y</td>\n      <td>East Toronto</td>\n      <td>Business reply mail Processing Centre, South C...</td>\n      <td>43.688905</td>\n      <td>-79.554724</td>\n    </tr>\n    <tr>\n      <th>101</th>\n      <td>M8Y</td>\n      <td>Etobicoke</td>\n      <td>Old Mill South, King's Mill Park, Sunnylea, Hu...</td>\n      <td>43.739416</td>\n      <td>-79.588437</td>\n    </tr>\n    <tr>\n      <th>102</th>\n      <td>M8Z</td>\n      <td>Etobicoke</td>\n      <td>Mimico NW, The Queensway West, South of Bloor,...</td>\n      <td>43.706748</td>\n      <td>-79.594054</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "    Postal Code            Borough  \\\n0          M3A\n        North York\n   \n1          M4A\n        North York\n   \n2          M5A\n  Downtown Toronto\n   \n3          M6A\n        North York\n   \n4          M7A\n  Downtown Toronto\n   \n5          M9A\n         Etobicoke\n   \n6          M1B\n       Scarborough\n   \n7          M3B\n        North York\n   \n8          M4B\n         East York\n   \n9          M5B\n  Downtown Toronto\n   \n10         M6B\n        North York\n   \n11         M9B\n         Etobicoke\n   \n12         M1C\n       Scarborough\n   \n13         M3C\n        North York\n   \n14         M4C\n         East York\n   \n15         M5C\n  Downtown Toronto\n   \n16         M6C\n              York\n   \n17         M9C\n         Etobicoke\n   \n18         M1E\n       Scarborough\n   \n19         M4E\n      East Toronto\n   \n20         M5E\n  Downtown Toronto\n   \n21         M6E\n              York\n   \n22         M1G\n       Scarborough\n   \n23         M4G\n         East York\n   \n24         M5G\n  Downtown Toronto\n   \n25         M6G\n  Downtown Toronto\n   \n26         M1H\n       Scarborough\n   \n27         M2H\n        North York\n   \n28         M3H\n        North York\n   \n29         M4H\n         East York\n   \n30         M5H\n  Downtown Toronto\n   \n31         M6H\n      West Toronto\n   \n32         M1J\n       Scarborough\n   \n33         M2J\n        North York\n   \n34         M3J\n        North York\n   \n35         M4J\n         East York\n   \n36         M5J\n  Downtown Toronto\n   \n37         M6J\n      West Toronto\n   \n38         M1K\n       Scarborough\n   \n39         M2K\n        North York\n   \n40         M3K\n        North York\n   \n41         M4K\n      East Toronto\n   \n42         M5K\n  Downtown Toronto\n   \n43         M6K\n      West Toronto\n   \n44         M1L\n       Scarborough\n   \n45         M2L\n        North York\n   \n46         M3L\n        North York\n   \n47         M4L\n      East Toronto\n   \n48         M5L\n  Downtown Toronto\n   \n49         M6L\n        North York\n   \n50         M9L\n        North York\n   \n51         M1M\n       Scarborough\n   \n52         M2M\n        North York\n   \n53         M3M\n        North York\n   \n54         M4M\n      East Toronto\n   \n55         M5M\n        North York\n   \n56         M6M\n              York\n   \n57         M9M\n        North York\n   \n58         M1N\n       Scarborough\n   \n59         M2N\n        North York\n   \n60         M3N\n        North York\n   \n61         M4N\n   Central Toronto\n   \n62         M5N\n   Central Toronto\n   \n63         M6N\n              York\n   \n64         M9N\n              York\n   \n65         M1P\n       Scarborough\n   \n66         M2P\n        North York\n   \n67         M4P\n   Central Toronto\n   \n68         M5P\n   Central Toronto\n   \n69         M6P\n      West Toronto\n   \n70         M9P\n         Etobicoke\n   \n71         M1R\n       Scarborough\n   \n72         M2R\n        North York\n   \n73         M4R\n   Central Toronto\n   \n74         M5R\n   Central Toronto\n   \n75         M6R\n      West Toronto\n   \n76         M7R\n       Mississauga\n   \n77         M9R\n         Etobicoke\n   \n78         M1S\n       Scarborough\n   \n79         M4S\n   Central Toronto\n   \n80         M5S\n  Downtown Toronto\n   \n81         M6S\n      West Toronto\n   \n82         M1T\n       Scarborough\n   \n83         M4T\n   Central Toronto\n   \n84         M5T\n  Downtown Toronto\n   \n85         M1V\n       Scarborough\n   \n86         M4V\n   Central Toronto\n   \n87         M5V\n  Downtown Toronto\n   \n88         M8V\n         Etobicoke\n   \n89         M9V\n         Etobicoke\n   \n90         M1W\n       Scarborough\n   \n91         M4W\n  Downtown Toronto\n   \n92         M5W\n  Downtown Toronto\n   \n93         M8W\n         Etobicoke\n   \n94         M9W\n         Etobicoke\n   \n95         M1X\n       Scarborough\n   \n96         M4X\n  Downtown Toronto\n   \n97         M5X\n  Downtown Toronto\n   \n98         M8X\n         Etobicoke\n   \n99         M4Y\n  Downtown Toronto\n   \n100        M7Y\n      East Toronto\n   \n101        M8Y\n         Etobicoke\n   \n102        M8Z\n         Etobicoke\n   \n\n                                          Neighborhood   Latitude  Longitude  \n0                                           Parkwoods\n  43.806686 -79.194353  \n1                                    Victoria Village\n  43.784535 -79.160497  \n2                           Regent Park, Harbourfront\n  43.763573 -79.188711  \n3                    Lawrence Manor, Lawrence Heights\n  43.770992 -79.216917  \n4         Queen's Park, Ontario Provincial Government\n  43.773136 -79.239476  \n5             Islington Avenue, Humber Valley Village\n  43.744734 -79.239476  \n6                                      Malvern, Rouge\n  43.727929 -79.262029  \n7                                           Don Mills\n  43.711112 -79.284577  \n8                     Parkview Hill, Woodbine Gardens\n  43.716316 -79.239476  \n9                            Garden District, Ryerson\n  43.692657 -79.264848  \n10                                          Glencairn\n  43.757410 -79.273304  \n11   West Deane Park, Princess Gardens, Martin Grov...  43.750072 -79.295849  \n12             Rouge Hill, Port Union, Highland Creek\n  43.794200 -79.262029  \n13                                          Don Mills\n  43.781638 -79.304302  \n14                                   Woodbine Heights\n  43.815252 -79.284577  \n15                                     St. James Town\n  43.799525 -79.318389  \n16                                 Humewood-Cedarvale\n  43.836125 -79.205636  \n17   Eringate, Bloordale Gardens, Old Burnhamthorpe...  43.803762 -79.363452  \n18                  Guildwood, Morningside, West Hill\n  43.778517 -79.346556  \n19                                        The Beaches\n  43.786947 -79.385975  \n20                                        Berczy Park\n  43.757490 -79.374714  \n21                                Caledonia-Fairbanks\n  43.789053 -79.408493  \n22                                             Woburn\n  43.770120 -79.408493  \n23                                            Leaside\n  43.752758 -79.400049  \n24                                 Central Bay Street\n  43.782736 -79.442259  \n25                                           Christie\n  43.753259 -79.329656  \n26                                          Cedarbrae\n  43.745906 -79.352188  \n27                                  Hillcrest Village\n  43.725900 -79.340923  \n28    Bathurst Manor, Wilson Heights, Downsview North\n  43.754328 -79.442259  \n29                                   Thorncliffe Park\n  43.767980 -79.487262  \n30                           Richmond, Adelaide, King\n  43.737473 -79.464763  \n31                       Dufferin, Dovercourt Village\n  43.739015 -79.506944  \n32                                Scarborough Village\n  43.728496 -79.495697  \n33                       Fairview, Henry Farm, Oriole\n  43.761631 -79.520999  \n34                    Northwood Park, York University\n  43.725882 -79.315572  \n35      East Toronto, Broadview North (Old East York)\n  43.706397 -79.309937  \n36   Harbourfront East, Union Station, Toronto Isla...  43.695344 -79.318389  \n37                           Little Portugal, Trinity\n  43.676357 -79.293031  \n38        Kennedy Park, Ionview, East Birchmount Park\n  43.709060 -79.363452  \n39                                    Bayview Village\n  43.705369 -79.349372  \n40                                          Downsview\n  43.685347 -79.338106  \n41                       The Danforth West, Riverdale\n  43.679557 -79.352188  \n42           Toronto Dominion Centre, Design Exchange\n  43.668999 -79.315572  \n43       Brockton, Parkdale Village, Exhibition Place\n  43.659526 -79.340923  \n44                    Golden Mile, Clairlea, Oakridge\n  43.728020 -79.388790  \n45                           York Mills, Silver Hills\n  43.712751 -79.390197  \n46                                          Downsview\n  43.715383 -79.405678  \n47                     India Bazaar, The Beaches West\n  43.704324 -79.388790  \n48                     Commerce Court, Victoria Hotel\n  43.689574 -79.383160  \n49           North Park, Maple Leaf Park, Upwood Park\n  43.686412 -79.400049  \n50                                      Humber Summit\n  43.679563 -79.377529  \n51    Cliffside, Cliffcrest, Scarborough Village West\n  43.667967 -79.367675  \n52                            Willowdale, Newtonbrook\n  43.665860 -79.383160  \n53                                          Downsview\n  43.654260 -79.360636  \n54                                    Studio District\n  43.657162 -79.378937  \n55                  Bedford Park, Lawrence Manor East\n  43.651494 -79.375418  \n56   Del Ray, Mount Dennis, Keelsdale and Silverthorn\n  43.644771 -79.373306  \n57                                   Humberlea, Emery\n  43.657952 -79.387383  \n58                        Birch Cliff, Cliffside West\n  43.650571 -79.384568  \n59                        Willowdale, Willowdale East\n  43.640816 -79.381752  \n60                                          Downsview\n  43.647177 -79.381576  \n61                                      Lawrence Park\n  43.648198 -79.379817  \n62                                           Roselawn\n  43.733283 -79.419750  \n63                      Runnymede, The Junction North\n  43.711695 -79.416936  \n64                                             Weston\n  43.696948 -79.411307  \n65   Dorset Park, Wexford Heights, Scarborough Town...  43.672710 -79.405678  \n66                                    York Mills West\n  43.662696 -79.400049  \n67                                   Davisville North\n  43.653206 -79.400049  \n68    Forest Hill North & West, Forest Hill Road Park\n  43.628947 -79.394420  \n69                      High Park, The Junction South\n  43.646435 -79.374846  \n70                                          Westmount\n  43.648429 -79.382280  \n71                                  Wexford, Maryvale\n  43.718518 -79.464763  \n72                        Willowdale, Willowdale West\n  43.709577 -79.445073  \n73                 North Toronto West,  Lawrence Park\n  43.693781 -79.428191  \n74                The Annex, North Midtown, Yorkville\n  43.689026 -79.453512  \n75                             Parkdale, Roncesvalles\n  43.669542 -79.422564  \n76              Canada Post Gateway Processing Centre\n  43.669005 -79.442259  \n77   Kingsview Village, St. Phillips, Martin Grove ...  43.647927 -79.419750  \n78                                          Agincourt\n  43.636847 -79.428191  \n79                                         Davisville\n  43.713756 -79.490074  \n80                     University of Toronto, Harbord\n  43.691116 -79.476013  \n81                                 Runnymede, Swansea\n  43.673185 -79.487262  \n82            Clarks Corners, Tam O'Shanter, Sullivan\n  43.661608 -79.464763  \n83                        Moore Park, Summerhill East\n  43.648960 -79.456325  \n84          Kensington Market, Chinatown, Grange Park\n  43.651571 -79.484450  \n85   Milliken, Agincourt North, Steeles East, L'Amo...  43.662301 -79.389494  \n86   Summerhill West, Rathnelly, South Hill, Forest...  43.636966 -79.615819  \n87   CN Tower, King and Spadina, Railway Lands, Har...  43.662744 -79.321558  \n88       New Toronto, Mimico South, Humber Bay Shores\n  43.605647 -79.501321  \n89   South Steeles, Silverstone, Humbergate, Jamest...  43.602414 -79.543484  \n90                      Steeles West, L'Amoreaux West\n  43.653654 -79.506944  \n91                                           Rosedale\n  43.636258 -79.498509  \n92                                     Stn A PO Boxes\n  43.628841 -79.520999  \n93                             Alderwood, Long Branch\n  43.667856 -79.532242  \n94                Northwest, West Humber - Clairville\n  43.650943 -79.554724  \n95                                        Upper Rouge\n  43.643515 -79.577201  \n96                        St. James Town, Cabbagetown\n  43.756303 -79.565963  \n97             First Canadian Place, Underground city\n  43.724766 -79.532242  \n98      The Kingsway, Montgomery Road, Old Mill North\n  43.706876 -79.518188  \n99                               Church and Wellesley\n  43.696319 -79.532242  \n100  Business reply mail Processing Centre, South C...  43.688905 -79.554724  \n101  Old Mill South, King's Mill Park, Sunnylea, Hu...  43.739416 -79.588437  \n102  Mimico NW, The Queensway West, South of Bloor,...  43.706748 -79.594054  "
                    },
                    "execution_count": 99,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "result = pd.concat([new_df, tor_df], axis = 1, sort=False)\n\n#this line of code will drop duplicate column -to avoid the NaN problem-\nresult = result.loc[:,~result.columns.duplicated()]\n\nresult"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 1
}