examples/notebooks/us_census_data.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "<a href=\"https://githubtocolab.com/gee-community/geemap/blob/master/examples/notebooks/us_census_data.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/></a>\n",
                "\n",
                "Uncomment the following line to install [geemap](https://geemap.org) if needed."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# U.S. Census Data \n",
                "\n",
                "The United States Census Bureau Topologically Integrated Geographic Encoding and Referencing (TIGER) dataset contains the 2018 boundaries for the primary governmental divisions of the United States. In addition to the fifty states, the Census Bureau treats the District of Columbia, Puerto Rico, and each of the island areas (American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands) as the statistical equivalents of States for the purpose of data presentation. Each feature represents a state or state equivalent.\n",
                "\n",
                "For full technical details on all TIGER 2018 products, see the [TIGER technical documentation](https://www2.census.gov/geo/pdfs/maps-data/data/tiger/tgrshp2018/TGRSHP2018_TechDoc.pdf).\n",
                "\n",
                "* [TIGER: US Census States](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2018_States): `ee.FeatureCollection(\"TIGER/2018/States\")`\n",
                "* [TIGER: US Census Counties](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2018_Counties): `ee.FeatureCollection(\"TIGER/2018/Counties\")`\n",
                "* [TIGER: US Census Tracts](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2010_Tracts_DP1): `ee.FeatureCollection(\"TIGER/2010/Tracts_DP1\")`\n",
                "* [TIGER: US Census Blocks](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2010_Blocks): `ee.FeatureCollection(\"TIGER/2010/Blocks\")`\n",
                "* [TIGER: US Census Roads](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2016_Roads): `ee.FeatureCollection(\"TIGER/2016/Roads\")`\n",
                "* [TIGER: US Census 5-digit ZIP Code](https://developers.google.com/earth-engine/datasets/catalog/TIGER_2010_ZCTA5): `ee.FeatureCollection(\"TIGER/2010/ZCTA5\")`"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Install Earth Engine API and geemap\n",
                "Install the [Earth Engine Python API](https://developers.google.com/earth-engine/python_install) and [geemap](https://github.com/gee-community/geemap). The **geemap** Python package is built upon the [ipyleaflet](https://github.com/jupyter-widgets/ipyleaflet) and [folium](https://github.com/python-visualization/folium) packages and implements several methods for interacting with Earth Engine data layers, such as `Map.addLayer()`, `Map.setCenter()`, and `Map.centerObject()`.\n",
                "The following script checks if the geemap package has been installed. If not, it will install geemap, which automatically installs its [dependencies](https://github.com/gee-community/geemap#dependencies), including earthengine-api, folium, and ipyleaflet."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Installs geemap package\n",
                "import subprocess\n",
                "\n",
                "try:\n",
                "    import geemap\n",
                "except ImportError:\n",
                "    print('geemap package not installed. Installing ...')\n",
                "    subprocess.check_call([\"python\", '-m', 'pip', 'install', 'geemap'])"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import ee\n",
                "import geemap"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## TIGER: US Census States\n",
                "\n",
                "https://developers.google.com/earth-engine/datasets/catalog/TIGER_2018_States\n",
                "\n",
                "![](https://i.imgur.com/DAESBK4.png)\n",
                "\n",
                "### Displaying data"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "\n",
                "Map.centerObject(states, 4)\n",
                "Map.addLayer(states, {}, 'US States')\n",
                "\n",
                "Map.addLayerControl()  # This line is not needed for ipyleaflet-based Map\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Displaying vector as raster"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "image = ee.Image().paint(states, 0, 2)\n",
                "\n",
                "Map.centerObject(states, 4)\n",
                "Map.addLayer(image, {}, 'US States')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Select by attribute\n",
                "\n",
                "#### Select one single state"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "tn = ee.FeatureCollection('TIGER/2018/States').filter(ee.Filter.eq(\"NAME\", 'Tennessee'))\n",
                "\n",
                "Map.centerObject(tn, 6)\n",
                "Map.addLayer(tn, {}, 'Tennessee')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "tn = (\n",
                "    ee.FeatureCollection('TIGER/2018/States')\n",
                "    .filter(ee.Filter.eq(\"NAME\", 'Tennessee'))\n",
                "    .first()\n",
                ")\n",
                "\n",
                "props = tn.toDictionary().getInfo()\n",
                "print(props)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Select multiple states"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "selected = ee.FeatureCollection('TIGER/2018/States').filter(\n",
                "    ee.Filter.inList(\"NAME\", ['Tennessee', 'Alabama', 'Georgia'])\n",
                ")\n",
                "\n",
                "Map.centerObject(selected, 6)\n",
                "Map.addLayer(selected, {}, 'Selected states')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Printing all values of a column"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "states = ee.FeatureCollection('TIGER/2018/States').sort('ALAND', False)\n",
                "names = states.aggregate_array(\"STUSPS\").getInfo()\n",
                "print(names)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "areas = states.aggregate_array(\"ALAND\").getInfo()\n",
                "print(areas)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import matplotlib.pyplot as plt\n",
                "\n",
                "%matplotlib notebook"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "plt.bar(names, areas)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Descriptive statistics of a column\n",
                "\n",
                "For example, we can calculate the total land area of all states:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "area_m2 = states.aggregate_sum(\"ALAND\").getInfo()\n",
                "area_km2 = area_m2 / 1000000\n",
                "print(\"Total land area: \", area_km2, \" km2\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "stats = states.aggregate_stats(\"ALAND\").getInfo()\n",
                "print(stats)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Add a new column to the attribute table"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "states = ee.FeatureCollection('TIGER/2018/States').sort('ALAND', False)\n",
                "states = states.map(lambda x: x.set('AreaKm2', x.area().divide(1000000).toLong()))\n",
                "first = states.first().toDictionary().getInfo()\n",
                "print(first)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Set symbology based on column  values"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "\n",
                "visParams = {\n",
                "    'palette': ['purple', 'blue', 'green', 'yellow', 'orange', 'red'],\n",
                "    'min': 500000000.0,\n",
                "    'max': 5e11,\n",
                "    'opacity': 0.8,\n",
                "}\n",
                "\n",
                "image = ee.Image().float().paint(states, 'ALAND')\n",
                "Map.addLayer(image, visParams, 'TIGER/2018/States')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Download attribute table as a CSV"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "url = states.getDownloadURL(\n",
                "    filetype=\"csv\",\n",
                "    selectors=['NAME', 'ALAND', 'REGION', 'STATEFP', 'STUSPS'],\n",
                "    filename=\"states\",\n",
                ")\n",
                "print(url)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Formatting the output"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "first = states.first()\n",
                "props = first.propertyNames().getInfo()\n",
                "print(props)\n",
                "props = states.first().toDictionary(props).getInfo()\n",
                "print(props)\n",
                "\n",
                "for key, value in props.items():\n",
                "    print(\"{}: {}\".format(key, value))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Download data as shapefile to Google Drive"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# function for converting GeometryCollection to Polygon/MultiPolygon\n",
                "def filter_polygons(ftr):\n",
                "    geometries = ftr.geometry().geometries()\n",
                "    geometries = geometries.map(\n",
                "        lambda geo: ee.Feature(ee.Geometry(geo)).set('geoType', ee.Geometry(geo).type())\n",
                "    )\n",
                "\n",
                "    polygons = (\n",
                "        ee.FeatureCollection(geometries)\n",
                "        .filter(ee.Filter.eq('geoType', 'Polygon'))\n",
                "        .geometry()\n",
                "    )\n",
                "    return ee.Feature(polygons).copyProperties(ftr)\n",
                "\n",
                "\n",
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "new_states = states.map(filter_polygons)\n",
                "\n",
                "col_names = states.first().propertyNames().getInfo()\n",
                "print(\"Column names: \", col_names)\n",
                "\n",
                "url = new_states.getDownloadURL(\"shp\", col_names, 'states')\n",
                "print(url)\n",
                "\n",
                "desc = 'states'\n",
                "\n",
                "# Set configuration parameters for output vector\n",
                "task_config = {\n",
                "    'folder': 'gee-data',  # output Google Drive folder\n",
                "    'fileFormat': 'SHP',\n",
                "    'selectors': col_names,  # a list of properties/attributes to be exported\n",
                "}\n",
                "\n",
                "print('Exporting {}'.format(desc))\n",
                "task = ee.batch.Export.table.toDrive(new_states, desc, **task_config)\n",
                "task.start()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## TIGER: US Census Blocks\n",
                "\n",
                "https://developers.google.com/earth-engine/datasets/catalog/TIGER_2010_Blocks\n",
                "\n",
                "![](https://i.imgur.com/KTQqb8v.png)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "dataset = ee.FeatureCollection('TIGER/2010/Blocks').filter(\n",
                "    ee.Filter.eq('statefp10', '47')\n",
                ")\n",
                "\n",
                "pop = dataset.aggregate_sum('pop10')\n",
                "\n",
                "print(\"The number of census blocks: \", dataset.size().getInfo())\n",
                "print(\"Total population: \", pop.getInfo())\n",
                "\n",
                "Map.setCenter(-86.79, 35.87, 6)\n",
                "Map.addLayer(dataset, {}, \"Census Block\", False)\n",
                "\n",
                "visParams = {\n",
                "    'min': 0.0,\n",
                "    'max': 700.0,\n",
                "    'palette': ['black', 'brown', 'yellow', 'orange', 'red'],\n",
                "}\n",
                "\n",
                "image = ee.Image().float().paint(dataset, 'pop10')\n",
                "\n",
                "Map.setCenter(-73.99172, 40.74101, 13)\n",
                "Map.addLayer(image, visParams, 'TIGER/2010/Blocks')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## TIGER: US Census Counties 2018\n",
                "\n",
                "https://developers.google.com/earth-engine/datasets/catalog/TIGER_2018_Counties\n",
                "\n",
                "![](https://i.imgur.com/K3j99eA.png)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "Map.setCenter(-110, 40, 5)\n",
                "states = ee.FeatureCollection('TIGER/2018/States')\n",
                "#     .filter(ee.Filter.eq('STUSPS', 'TN'))\n",
                "# // Turn the strings into numbers\n",
                "states = states.map(lambda f: f.set('STATEFP', ee.Number.parse(f.get('STATEFP'))))\n",
                "\n",
                "state_image = ee.Image().float().paint(states, 'STATEFP')\n",
                "\n",
                "visParams = {\n",
                "    'palette': ['purple', 'blue', 'green', 'yellow', 'orange', 'red'],\n",
                "    'min': 0,\n",
                "    'max': 50,\n",
                "    'opacity': 0.8,\n",
                "}\n",
                "\n",
                "counties = ee.FeatureCollection('TIGER/2016/Counties')\n",
                "# print(counties.first().propertyNames().getInfo())\n",
                "\n",
                "image = ee.Image().paint(states, 0, 2)\n",
                "# Map.setCenter(-99.844, 37.649, 4)\n",
                "# Map.addLayer(image, {'palette': 'FF0000'}, 'TIGER/2018/States')\n",
                "Map.addLayer(state_image, visParams, 'TIGER/2016/States')\n",
                "Map.addLayer(ee.Image().paint(counties, 0, 1), {}, 'TIGER/2016/Counties')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## TIGER: US Census Tracts\n",
                "\n",
                "https://developers.google.com/earth-engine/datasets/catalog/TIGER_2010_Tracts_DP1\n",
                "\n",
                "http://magic.lib.uconn.edu/magic_2/vector/37800/demogprofilehousect_37800_0000_2010_s100_census_1_t.htm\n",
                "\n",
                "![](https://i.imgur.com/WV2XA9b.png)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "dataset = ee.FeatureCollection('TIGER/2010/Tracts_DP1')\n",
                "visParams = {\n",
                "    'min': 0,\n",
                "    'max': 4000,\n",
                "    'opacity': 0.8,\n",
                "    'palette': [\n",
                "        '#ece7f2',\n",
                "        '#d0d1e6',\n",
                "        '#a6bddb',\n",
                "        '#74a9cf',\n",
                "        '#3690c0',\n",
                "        '#0570b0',\n",
                "        '#045a8d',\n",
                "        '#023858',\n",
                "    ],\n",
                "}\n",
                "\n",
                "# print(dataset.first().propertyNames().getInfo())\n",
                "\n",
                "# Turn the strings into numbers\n",
                "dataset = dataset.map(\n",
                "    lambda f: f.set('shape_area', ee.Number.parse(f.get('dp0010001')))\n",
                ")\n",
                "\n",
                "# Map.setCenter(-103.882, 43.036, 8)\n",
                "image = ee.Image().float().paint(dataset, 'dp0010001')\n",
                "\n",
                "Map.addLayer(image, visParams, 'TIGER/2010/Tracts_DP1')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## TIGER: US Census Roads\n",
                "\n",
                "https://developers.google.com/earth-engine/datasets/catalog/TIGER_2016_Roads\n",
                "\n",
                "![](https://i.imgur.com/mguD0IQ.png)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "Map = geemap.Map(center=[40, -100], zoom=4)\n",
                "\n",
                "fc = ee.FeatureCollection('TIGER/2016/Roads')\n",
                "Map.setCenter(-73.9596, 40.7688, 12)\n",
                "Map.addLayer(fc, {}, 'Census roads')\n",
                "\n",
                "Map.addLayerControl()\n",
                "Map"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": []
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3",
            "language": "python",
            "name": "python3"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}