sodapy-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sodapy Basics Tutorial Using NYC Open Data \n",
    "Author: Mark Bauer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Table of Contents\n",
    "=================\n",
    "\n",
    "   1. Introduction\n",
    "   2. Sodapy\n",
    "       - 2.1 Using Sodapy\n",
    "       - 2.2 Sodapy Methods\n",
    "   3. Importing Libraries\n",
    "   4. Socrata Class\n",
    "   5. Sodapy Methods\n",
    "       - 5.1 .datasets()\n",
    "       - 5.2 .get()\n",
    "       - 5.3 .get_metadata()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction  \n",
    "This notebook demonstrates how to use sodapy, the python client for the Socrata Open Data API (SODA), and reviews various methods to retrieve data from Socrata Open Data. The data in this tutorial is from NYC Open Data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Sodapy\n",
    "\n",
    "## 2.1 Using Sodapy\n",
    "\n",
    "In order use sodapy, a source domain (i.e. the Socrata Open Data source you are trying to connect to) needs to be passed to the Socrata class. Additionally, if a user wants to query a specific dataset on Socrata Open Data, then the dataset identifier (i.e. the dataset id on the given source domain) needs to be passed as well. Below, we identify NYC Open Data's source domain: `data.cityofnewyork.us` and the dataset identifier for the NYC 311 data set: `erm2-nwe9`. The screenshot below displays where we retrieve this information.\n",
    "\n",
    "![nyc-311-api-docs](images/nyc-311-api-docs.png)  \n",
    "\n",
    "**Source**: https://dev.socrata.com/foundry/data.cityofnewyork.us/erm2-nwe9\n",
    "\n",
    "## 2.2 Sodapy Methods\n",
    "\n",
    "![socrata-methods](images/socrata-methods.png)\n",
    "\n",
    "**Source**: https://github.com/xmunoz/sodapy#datasetslimit0-offset0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be focusing on three sodapy methods:\n",
    "-  `.datasets()`\n",
    "Returns the list of datasets associated with a particular domain.\n",
    "\n",
    "- `.get()`\n",
    "Read data from the requested resource. Options for content_type are json, csv, and xml.\n",
    "\n",
    "- `.get_metadata()`\n",
    "Retrieve the metadata for a particular dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Importing Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries\n",
    "import pandas as pd\n",
    "from sodapy import Socrata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last updated: 2024-04-23 15:53:19\n",
      "\n",
      "Python implementation: CPython\n",
      "Python version       : 3.8.13\n",
      "IPython version      : 8.4.0\n",
      "\n",
      "pandas: 1.4.3\n",
      "sodapy: 2.1.1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# documention for installing watermark: https://github.com/rasbt/watermark\n",
    "%reload_ext watermark\n",
    "%watermark -u -t -d -v -p pandas,sodapy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Socrata Class "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note:  \n",
    "`WARNING:root:Requests made without an app_token will be subject to strict throttling limits.`\n",
    "\n",
    "Read more from the SODA documentation here: https://dev.socrata.com/docs/app-tokens.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sodapy.socrata.Socrata at 0x15efb4b50>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# source domain for NYC Open Data on Socrata\n",
    "socrata_domain = 'data.cityofnewyork.us'\n",
    "\n",
    "# Socrata(): The main class that interacts with the SODA API.\n",
    "\n",
    "# The required arguments are:\n",
    "#     domain: the domain you wish you to access\n",
    "#     app_token: your Socrata application token\n",
    "# Simple requests are possible without an app_token, though these\n",
    "# requests will be rate-limited.\n",
    "\n",
    "client = Socrata(\n",
    "    socrata_domain,\n",
    "    app_token=None,\n",
    "    timeout=100\n",
    ")\n",
    "\n",
    "client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<sodapy.socrata.Socrata object at 0x15efb4b50>\n",
      "type: <class 'sodapy.socrata.Socrata'>\n"
     ]
    }
   ],
   "source": [
    "# printing information about the Socrata object\n",
    "print(client)\n",
    "print('type: {}'.format(type(client)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "domain: data.cityofnewyork.us\n",
      "session: <requests.sessions.Session object at 0x15efb4ca0>\n",
      "uri_prefix: https://\n",
      "timeout: 100\n"
     ]
    }
   ],
   "source": [
    "# printing attributes of object\n",
    "for key, value in client.__dict__.items():\n",
    "    print('{}: {}'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# close the session when finished\n",
    "client.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Sodapy Methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1  .datasets()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`datasets` method: Returns the list of datasets associated with a particular domain.\n",
    "WARNING: Large limits (>1000) will return megabytes of data,\n",
    "which can be slow on low-bandwidth networks, and is also a lot of\n",
    "data to hold in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "object type: <class 'list'>\n",
      "Number of datasets on NYC Open Data: 3,342.\n"
     ]
    }
   ],
   "source": [
    "# soure domain for NYC Open Data on Socrata\n",
    "socrata_domain = 'data.cityofnewyork.us'\n",
    "\n",
    "client = Socrata(\n",
    "    socrata_domain,\n",
    "    app_token=None,\n",
    "    timeout=100\n",
    ")\n",
    "\n",
    "print('object type: {}'.format(type(client.datasets())))\n",
    "\n",
    "datasets = len(client.datasets())\n",
    "print('Number of datasets on NYC Open Data: {:,}.'.format(datasets))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'dict'>\n"
     ]
    }
   ],
   "source": [
    "# reviewing type about first dataset\n",
    "print(type(client.datasets()[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['resource', 'classification', 'metadata', 'permalink', 'link', 'owner', 'creator'])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reviewing information about keys first dataset\n",
    "client.datasets()[0].keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name: For Hire Vehicles (FHV) - Active\n",
      "\n",
      "id: 8wbx-tsch\n",
      "\n",
      "parent_fxf: []\n",
      "\n",
      "description: <b>PLEASE NOTE:</b> This dataset, which includes all TLC licensed for-hire vehicles which are in good standing and able to drive, is updated every day in the evening between 4-7pm. Please check the 'Last Update Date' field to make sure the list has updated successfully. 'Last Update Date'  should show either today or yesterday's date, depending on the time of day. If the list is outdated, please download the most recent list from the link below. \r\n",
      "http://www1.nyc.gov/assets/tlc/downloads/datasets/tlc_for_hire_vehicle_active_and_inactive.csv\r\n",
      "\r\n",
      "TLC authorized For-Hire vehicles that are active. This list is accurate to the date and time represented in the Last Date Updated and Last Time Updated fields. For inquiries about the contents of this dataset, please email licensinginquiries@tlc.nyc.gov.\n",
      "\n",
      "attribution: Taxi and Limousine Commission (TLC)\n",
      "\n",
      "attribution_link: None\n",
      "\n",
      "contact_email: None\n",
      "\n",
      "type: dataset\n",
      "\n",
      "updatedAt: 2024-04-23T18:55:30.000Z\n",
      "\n",
      "createdAt: 2015-07-16T17:33:32.000Z\n",
      "\n",
      "metadata_updated_at: 2022-09-06T21:05:32.000Z\n",
      "\n",
      "data_updated_at: 2024-04-23T18:55:30.000Z\n",
      "\n",
      "page_views: {'page_views_last_week': 8582, 'page_views_last_month': 36950, 'page_views_total': 2657120, 'page_views_last_week_log': 13.06726628304434, 'page_views_last_month_log': 15.17332578791327, 'page_views_total_log': 21.341432496108254}\n",
      "\n",
      "columns_name: ['Base Address', 'Last Time Updated', 'Active', 'License Type', 'VEH', 'Hack Up Date', 'Vehicle License Number', 'Name', 'Permit License Number', 'Vehicle VIN Number', 'Wheelchair Accessible', 'Base Number', 'Vehicle Year', 'Order Date', 'Base Type', 'Website', 'Expiration Date', 'Certification Date', 'Last Date Updated', 'Base Name', 'DMV License Plate Number', 'Base Telephone Number', 'Reason']\n",
      "\n",
      "columns_field_name: ['base_address', 'last_time_updated', 'active', 'license_type', 'veh', 'hack_up_date', 'vehicle_license_number', 'name', 'permit_license_number', 'vehicle_vin_number', 'wheelchair_accessible', 'base_number', 'vehicle_year', 'order_date', 'base_type', 'website', 'expiration_date', 'certification_date', 'last_date_updated', 'base_name', 'dmv_license_plate_number', 'base_telephone_number', 'reason']\n",
      "\n",
      "columns_datatype: ['Text', 'Text', 'Text', 'Text', 'Text', 'Calendar date', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Number', 'Calendar date', 'Text', 'Text', 'Calendar date', 'Calendar date', 'Calendar date', 'Text', 'Text', 'Text', 'Text']\n",
      "\n",
      "columns_description: ['Base Address', 'Last Time Updated', 'Permit active or not\\n', 'TLC License Type\\n', 'Hybrid vehicle indicator\\n', 'Hack Up Date', 'FHV Vehicle License Number', 'Owner Name\\n', 'Permit License Number', 'Vehicle VIN Number', 'Wheelchair Accessible Indicator\\n', 'Base Number', 'Vehicle Year', 'Date Suspension ordered\\n', 'Base Type', 'Base Website', 'Expiration Date', 'Certification Date', 'Last Date Updated', 'Base Name', 'DMV License Plate Number', 'Base Telephone NumberBase Telephone Number', 'Reason Code A,B,C or G\\n']\n",
      "\n",
      "columns_format: [{'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'view': 'date', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'precisionStyle': 'standard', 'noCommas': 'true', 'align': 'right'}, {'view': 'date_time', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'view': 'date', 'align': 'left'}, {'view': 'date', 'align': 'left'}, {'view': 'date', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}]\n",
      "\n",
      "download_count: 459469\n",
      "\n",
      "provenance: official\n",
      "\n",
      "lens_view_type: tabular\n",
      "\n",
      "lens_display_type: table\n",
      "\n",
      "blob_mime_type: None\n",
      "\n",
      "hide_from_data_json: False\n",
      "\n",
      "publication_date: 2021-04-05T13:20:47.000Z\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# reviewing information about resource key\n",
    "resources = client.datasets()[0]['resource'].items()\n",
    "\n",
    "for key, value in resources:\n",
    "    print('{}: {}\\n'.format(key, value))      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "categories: []\n",
      "tags: []\n",
      "domain_category: Transportation\n",
      "domain_tags: ['active', 'for-hire', 'for hire', 'fhv', 'drivers', 'inactive', 'taxi', 'for-hire-vehicles']\n",
      "domain_metadata: [{'key': 'Update_Automation', 'value': 'Yes'}, {'key': 'Update_Date-Made-Public', 'value': '7/20/2015'}, {'key': 'Update_Update-Frequency', 'value': 'Daily'}, {'key': 'Dataset-Information_Agency', 'value': 'Taxi and Limousine Commission (TLC)'}]\n"
     ]
    }
   ],
   "source": [
    "# reviewing information about classification key\n",
    "classification = client.datasets()[0]['classification'].items()\n",
    "\n",
    "for key, value in classification:\n",
    "    print('{}: {}'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "domain: data.cityofnewyork.us\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# reviewing information about metadata key\n",
    "metadata = client.datasets()[0]['metadata'].items()\n",
    "\n",
    "for key, value in metadata:\n",
    "    print('{}: {}\\n'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id: 5fuc-pqz2\n",
      "user_type: interactive\n",
      "display_name: NYC OpenData\n"
     ]
    }
   ],
   "source": [
    "owner = client.datasets()[0]['owner'].items()\n",
    "\n",
    "for key, value in owner:\n",
    "    print('{}: {}'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id: 5fuc-pqz2\n",
      "user_type: interactive\n",
      "display_name: NYC OpenData\n"
     ]
    }
   ],
   "source": [
    "creator = list(client.datasets()[0]['creator'].items())\n",
    "     \n",
    "for key, value in creator:\n",
    "    print('{}: {}'.format(key, value))     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we've identified the structure of the dictionary, we try to find the 311 dataset and identify its position in the datasets list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We found the NYC 311 dataset!\n",
      "Index is: 5\n"
     ]
    }
   ],
   "source": [
    "# NYC 311 dataset identifier\n",
    "socrata_dataset_identifier = 'erm2-nwe9'\n",
    "\n",
    "for idx in range(len(client.datasets())):\n",
    "    \n",
    "    if client.datasets()[idx]['resource']['id'] == socrata_dataset_identifier:\n",
    "        print('We found the NYC 311 dataset!')\n",
    "        print('Index is: {}'.format(idx))\n",
    "        dataset_index = idx\n",
    "        \n",
    "        break "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the datasets method is long, let's see if we can identify specific keys we want to preview in the resource dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name: 311 Service Requests from 2010 to Present\n",
      "\n",
      "id: erm2-nwe9\n",
      "\n",
      "parent_fxf: []\n",
      "\n",
      "description: <b>Please note: Due to pandemic call handling modifications, the ‘Open Data Channel Type’ values may not accurately indicate the channel the Service Request was submitted in for the period starting March 2020.</b>\r\n",
      "<p>\r\n",
      "All 311 Service Requests from 2010 to present. This information is automatically updated daily.\r\n",
      "</p>\n",
      "\n",
      "attribution: 311\n",
      "\n",
      "attribution_link: None\n",
      "\n",
      "contact_email: None\n",
      "\n",
      "type: dataset\n",
      "\n",
      "updatedAt: 2024-04-23T01:34:04.000Z\n",
      "\n",
      "createdAt: 2011-10-10T05:52:17.000Z\n",
      "\n",
      "metadata_updated_at: 2024-01-24T21:32:48.000Z\n",
      "\n",
      "data_updated_at: 2024-04-23T01:34:04.000Z\n",
      "\n",
      "page_views: {'page_views_last_week': 2173, 'page_views_last_month': 9758, 'page_views_total': 829836, 'page_views_last_week_log': 11.08613622502731, 'page_views_last_month_log': 13.252517607762288, 'page_views_total_log': 19.66246845862453}\n",
      "\n",
      "columns_name: ['Borough Boundaries', 'Community Districts', 'Zip Codes', 'Bridge Highway Segment', 'Road Ramp', 'Bridge Highway Direction', 'Resolution Description', 'Due Date', 'City', 'Descriptor', 'Closed Date', 'Created Date', 'Unique Key', 'Location Type', 'Community Board', 'Intersection Street 2', 'Incident Address', 'Longitude', 'X Coordinate (State Plane)', 'Resolution Action Updated Date', 'Landmark', 'Location (zip)', 'Address Type', 'Location (city)', 'Park Facility Name', 'Y Coordinate (State Plane)', 'Complaint Type', 'Agency', 'Street Name', 'Cross Street 2', 'Bridge Highway Name', 'Vehicle Type', 'BBL', 'Location', 'Latitude', 'Taxi Pick Up Location', 'Park Borough', 'Borough', 'Status', 'Intersection Street 1', 'Cross Street 1', 'Incident Zip', 'Agency Name', 'Taxi Company Borough', 'Facility Type', 'Open Data Channel Type', 'Location (state)', 'Location (address)', 'Police Precinct', 'Police Precincts', 'City Council Districts']\n",
      "\n",
      "columns_field_name: [':@computed_region_yeji_bk3q', ':@computed_region_f5dn_yrer', ':@computed_region_efsh_h5xi', 'bridge_highway_segment', 'road_ramp', 'bridge_highway_direction', 'resolution_description', 'due_date', 'city', 'descriptor', 'closed_date', 'created_date', 'unique_key', 'location_type', 'community_board', 'intersection_street_2', 'incident_address', 'longitude', 'x_coordinate_state_plane', 'resolution_action_updated_date', 'landmark', 'location_zip', 'address_type', 'location_city', 'park_facility_name', 'y_coordinate_state_plane', 'complaint_type', 'agency', 'street_name', 'cross_street_2', 'bridge_highway_name', 'vehicle_type', 'bbl', 'location', 'latitude', 'taxi_pick_up_location', 'park_borough', 'borough', 'status', 'intersection_street_1', 'cross_street_1', 'incident_zip', 'agency_name', 'taxi_company_borough', 'facility_type', 'open_data_channel_type', 'location_state', 'location_address', ':@computed_region_7mpf_4k6g', ':@computed_region_sbqj_enih', ':@computed_region_92fq_4b7q']\n",
      "\n",
      "columns_datatype: ['Number', 'Number', 'Number', 'Text', 'Text', 'Text', 'Text', 'Calendar date', 'Text', 'Text', 'Calendar date', 'Calendar date', 'Text', 'Text', 'Text', 'Text', 'Text', 'Number', 'Number', 'Calendar date', 'Text', 'Text', 'Text', 'Text', 'Text', 'Number', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Point', 'Number', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Text', 'Number', 'Number', 'Number']\n",
      "\n",
      "columns_description: ['', '', '', 'Additional information on the section of the Bridge/Highway were the incident took place.\\n', 'If the incident location was Bridge/Highway this column differentiates if the issue was on the Road or the Ramp.\\n', 'If the incident is identified as a Bridge/Highway, the direction where the issue took place would be displayed here.\\n', 'Describes the last action taken on the SR by the responding agency.  May describe next or future steps. \\n', 'Date when responding agency is expected to update the SR.  This is based on the Complaint Type and internal Service Level Agreements (SLAs).\\n', 'City of the incident location provided by geovalidation.\\n', 'This is  associated to the Complaint Type, and provides further detail on the incident or condition. Descriptor values are dependent on the Complaint Type, and are not always required in SR. \\n', 'Date SR was closed by responding agency\\n', 'Date SR  was created\\n', 'Unique identifier of a Service Request (SR) in the open data set\\n', 'Describes the type of location used in the address information \\n', 'Provided by geovalidation.\\n', 'Second intersecting street based on geo validated incident location\\n', 'House number of incident address provided by submitter.\\n', 'Geo based Long of the incident location\\n', 'Geo validated, X coordinate of the incident location. \\n', 'Date when responding agency last updated the SR.\\n', 'If the incident location is identified as a Landmark the name of the landmark will display here\\n', '', 'Type of incident location information available.\\n', '', 'If the incident location is a Parks Dept facility, the Name of the facility will appear here\\n', 'Geo validated,  Y coordinate of the incident location. \\n', 'This is the first level of a hierarchy identifying the topic of the incident or condition. Complaint Type may have a corresponding Descriptor (below) or may stand alone.', 'Acronym of responding City Government Agency\\n', 'Street name of incident address provided by the submitter\\n', 'Second Cross Street based on the geo validated incident location\\n', 'If the incident is identified as a Bridge/Highway, the name will be displayed here.\\n', 'If the incident is a taxi, this field describes the type of TLC vehicle. \\n', 'Borough Block and Lot, provided by geovalidation. Parcel number to identify the location of location of buildings and properties in NYC.', 'Combination of the geo based lat & long of the incident location\\n', 'Geo based Lat of the incident location\\n', 'If the incident is identified as a taxi, this field displays the taxi pick up location\\n', 'The borough of incident if it is a Parks Dept facility\\n', 'Provided by the submitter and confirmed by geovalidation.\\n', 'Status of SR submitted \\n', 'First intersecting street based on geo validated incident location\\n', 'First Cross street based on the geo validated incident location\\n', 'Incident location zip code, provided by geo validation.\\n', 'Full Agency name of responding City Government Agency', 'If the incident is identified as a taxi, this field will display the borough of the taxi company. \\n', 'If available, this field describes the type of city facility associated to the SR\\n', 'Indicates how the SR was submitted to 311.  i.e. By Phone, Online, Mobile, Other or Unknown.', '', '', \"This column was automatically created in order to record in what polygon from the dataset 'Police Precinct' (7mpf-4k6g) the point in column 'location' is located.  This enables the creation of region maps (choropleths) in the visualization canvas and data lens.\", '', '']\n",
      "\n",
      "columns_format: [{}, {}, {}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'view': 'date_time', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {}, {'view': 'date_time', 'align': 'left'}, {'view': 'date_time', 'align': 'left'}, {}, {}, {'displayStyle': 'plain', 'align': 'left'}, {}, {}, {'precisionStyle': 'standard', 'noCommas': 'false', 'align': 'right'}, {'precisionStyle': 'standard', 'noCommas': 'false', 'align': 'right'}, {'view': 'date_time', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {}, {}, {}, {'displayStyle': 'plain', 'align': 'left'}, {'precisionStyle': 'standard', 'noCommas': 'false', 'align': 'right'}, {'displayStyle': 'plain', 'align': 'left'}, {}, {}, {}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'view': 'coords', 'align': 'left'}, {'precisionStyle': 'standard', 'noCommas': 'false', 'align': 'right'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {}, {}, {}, {}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {'displayStyle': 'plain', 'align': 'left'}, {}, {}, {}, {}, {}]\n",
      "\n",
      "download_count: 434081\n",
      "\n",
      "provenance: official\n",
      "\n",
      "lens_view_type: tabular\n",
      "\n",
      "lens_display_type: table\n",
      "\n",
      "blob_mime_type: None\n",
      "\n",
      "hide_from_data_json: False\n",
      "\n",
      "publication_date: 2023-12-01T06:51:46.000Z\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# preview items for 311 dataset\n",
    "resource_311 = client.datasets()[dataset_index]['resource'].items()\n",
    "\n",
    "for key, value in resource_311:\n",
    "    print('{}: {}\\n'.format(key, value)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The 311 Service Requests from 2010 to Present data description:\n",
      "\n",
      "<b>Please note: Due to pandemic call handling modifications, the ‘Open Data Channel Type’ values may not accurately indicate the channel the Service Request was submitted in for the period starting March 2020.</b>\r\n",
      "<p>\r\n",
      "All 311 Service Requests from 2010 to present. This information is automatically updated daily.\r\n",
      "</p>.\n"
     ]
    }
   ],
   "source": [
    "name = client.datasets()[dataset_index]['resource']['name']\n",
    "desc = client.datasets()[dataset_index]['resource']['description']\n",
    "\n",
    "print('The {} data description:\\n\\n{}.'.format(name, desc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "created: 2011-10-10T05:52:17.000Z\n",
      "updated: 2024-04-23T01:34:04.000Z.\n"
     ]
    }
   ],
   "source": [
    "created = client.datasets()[dataset_index]['resource']['createdAt']\n",
    "updated = client.datasets()[dataset_index]['resource']['updatedAt']\n",
    "\n",
    "print('created: {}\\nupdated: {}.'.format(created, updated))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "page_views_last_week: 2173\n",
      "page_views_last_month: 9758\n",
      "page_views_total: 829836\n",
      "page_views_last_week_log: 11.08613622502731\n",
      "page_views_last_month_log: 13.252517607762288\n",
      "page_views_total_log: 19.66246845862453\n"
     ]
    }
   ],
   "source": [
    "# NYC 311 dataset page views information\n",
    "for key, value in client.datasets()[dataset_index]['resource']['page_views'].items():\n",
    "    print('{}: {}'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  5.2 .get()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`get` method: Read data from the requested resource. Options for content_type are JSON,\n",
    "CSV, and XML."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "unique_key: 60939475\n",
      "created_date: 2024-04-22T01:51:26.000\n",
      "agency: NYPD\n",
      "agency_name: New York City Police Department\n",
      "complaint_type: Noise - Residential\n",
      "descriptor: Loud Music/Party\n",
      "location_type: Residential Building/House\n",
      "incident_zip: 11375\n",
      "incident_address: 89-17 69 ROAD\n",
      "street_name: 69 ROAD\n",
      "cross_street_1: METROPOLITAN AVENUE\n",
      "cross_street_2: OLCOTT STREET\n",
      "intersection_street_1: METROPOLITAN AVENUE\n",
      "intersection_street_2: OLCOTT STREET\n",
      "address_type: ADDRESS\n",
      "city: FOREST HILLS\n",
      "landmark: 69 ROAD\n",
      "status: In Progress\n",
      "resolution_action_updated_date: 2024-04-22T02:23:56.000\n",
      "community_board: 06 QUEENS\n",
      "bbl: 4032060011\n",
      "borough: QUEENS\n",
      "x_coordinate_state_plane: 1024544\n",
      "y_coordinate_state_plane: 198466\n",
      "open_data_channel_type: ONLINE\n",
      "park_facility_name: Unspecified\n",
      "park_borough: QUEENS\n",
      "latitude: 40.71132832064505\n",
      "longitude: -73.8546568589395\n",
      "location: {'latitude': '40.71132832064505', 'longitude': '-73.8546568589395', 'human_address': '{\"address\": \"\", \"city\": \"\", \"state\": \"\", \"zip\": \"\"}'}\n",
      ":@computed_region_efsh_h5xi: 14786\n",
      ":@computed_region_f5dn_yrer: 40\n",
      ":@computed_region_yeji_bk3q: 3\n",
      ":@computed_region_92fq_4b7q: 28\n",
      ":@computed_region_sbqj_enih: 70\n",
      ":@computed_region_7mpf_4k6g: 70\n"
     ]
    }
   ],
   "source": [
    "# source domain for NYC Open Data on Socrata\n",
    "socrata_domain = 'data.cityofnewyork.us'\n",
    "\n",
    "# NYC 311 dataset identifier\n",
    "socrata_dataset_identifier = 'erm2-nwe9'\n",
    "\n",
    "client = Socrata(\n",
    "    socrata_domain,\n",
    "    app_token=None,\n",
    "    timeout=1000\n",
    ")\n",
    "\n",
    "# printing the column and value of the first record\n",
    "for key, value in client.get(socrata_dataset_identifier)[0].items():\n",
    "    print('{}: {}'.format(key, value))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "unique_key: 60939475\n",
      "created_date: 2024-04-22T01:51:26.000\n",
      "agency: NYPD\n",
      "agency_name: New York City Police Department\n",
      "complaint_type: Noise - Residential\n",
      "descriptor: Loud Music/Party\n",
      "location_type: Residential Building/House\n",
      "incident_zip: 11375\n",
      "incident_address: 89-17 69 ROAD\n",
      "street_name: 69 ROAD\n",
      "cross_street_1: METROPOLITAN AVENUE\n",
      "cross_street_2: OLCOTT STREET\n",
      "intersection_street_1: METROPOLITAN AVENUE\n",
      "intersection_street_2: OLCOTT STREET\n",
      "address_type: ADDRESS\n",
      "city: FOREST HILLS\n",
      "landmark: 69 ROAD\n",
      "status: In Progress\n",
      "resolution_action_updated_date: 2024-04-22T02:23:56.000\n",
      "community_board: 06 QUEENS\n",
      "bbl: 4032060011\n",
      "borough: QUEENS\n",
      "x_coordinate_state_plane: 1024544\n",
      "y_coordinate_state_plane: 198466\n",
      "open_data_channel_type: ONLINE\n",
      "park_facility_name: Unspecified\n",
      "park_borough: QUEENS\n",
      "latitude: 40.71132832064505\n",
      "longitude: -73.8546568589395\n",
      "location: {'latitude': '40.71132832064505', 'longitude': '-73.8546568589395', 'human_address': '{\"address\": \"\", \"city\": \"\", \"state\": \"\", \"zip\": \"\"}'}\n",
      ":@computed_region_efsh_h5xi: 14786\n",
      ":@computed_region_f5dn_yrer: 40\n",
      ":@computed_region_yeji_bk3q: 3\n",
      ":@computed_region_92fq_4b7q: 28\n",
      ":@computed_region_sbqj_enih: 70\n",
      ":@computed_region_7mpf_4k6g: 70\n"
     ]
    }
   ],
   "source": [
    "# printing the column and value of the first record\n",
    "for key, value in client.get(socrata_dataset_identifier)[0].items():\n",
    "    print('{}: {}'.format(key, value))     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape of data: (100, 44)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unique_key</th>\n",
       "      <th>created_date</th>\n",
       "      <th>agency</th>\n",
       "      <th>agency_name</th>\n",
       "      <th>complaint_type</th>\n",
       "      <th>descriptor</th>\n",
       "      <th>location_type</th>\n",
       "      <th>incident_zip</th>\n",
       "      <th>incident_address</th>\n",
       "      <th>street_name</th>\n",
       "      <th>...</th>\n",
       "      <th>:@computed_region_sbqj_enih</th>\n",
       "      <th>:@computed_region_7mpf_4k6g</th>\n",
       "      <th>taxi_pick_up_location</th>\n",
       "      <th>closed_date</th>\n",
       "      <th>resolution_description</th>\n",
       "      <th>vehicle_type</th>\n",
       "      <th>facility_type</th>\n",
       "      <th>bridge_highway_name</th>\n",
       "      <th>bridge_highway_segment</th>\n",
       "      <th>bridge_highway_direction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>60939475</td>\n",
       "      <td>2024-04-22T01:51:26.000</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Residential</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Residential Building/House</td>\n",
       "      <td>11375</td>\n",
       "      <td>89-17 69 ROAD</td>\n",
       "      <td>69 ROAD</td>\n",
       "      <td>...</td>\n",
       "      <td>70</td>\n",
       "      <td>70</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>60938407</td>\n",
       "      <td>2024-04-22T01:51:25.000</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Commercial</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Club/Bar/Restaurant</td>\n",
       "      <td>10458</td>\n",
       "      <td>621 CRESCENT AVENUE</td>\n",
       "      <td>CRESCENT AVENUE</td>\n",
       "      <td>...</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>60935450</td>\n",
       "      <td>2024-04-22T01:50:12.000</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Residential</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Residential Building/House</td>\n",
       "      <td>11235</td>\n",
       "      <td>1245 AVENUE X</td>\n",
       "      <td>AVENUE X</td>\n",
       "      <td>...</td>\n",
       "      <td>36</td>\n",
       "      <td>36</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60937446</td>\n",
       "      <td>2024-04-22T01:50:11.000</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Commercial</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Club/Bar/Restaurant</td>\n",
       "      <td>11373</td>\n",
       "      <td>92-02 CORONA AVENUE</td>\n",
       "      <td>CORONA AVENUE</td>\n",
       "      <td>...</td>\n",
       "      <td>68</td>\n",
       "      <td>68</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>60935436</td>\n",
       "      <td>2024-04-22T01:50:11.000</td>\n",
       "      <td>NYPD</td>\n",
       "      <td>New York City Police Department</td>\n",
       "      <td>Noise - Commercial</td>\n",
       "      <td>Loud Music/Party</td>\n",
       "      <td>Club/Bar/Restaurant</td>\n",
       "      <td>11423</td>\n",
       "      <td>184-15 JAMAICA AVENUE</td>\n",
       "      <td>JAMAICA AVENUE</td>\n",
       "      <td>...</td>\n",
       "      <td>61</td>\n",
       "      <td>61</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 44 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  unique_key             created_date agency                      agency_name  \\\n",
       "0   60939475  2024-04-22T01:51:26.000   NYPD  New York City Police Department   \n",
       "1   60938407  2024-04-22T01:51:25.000   NYPD  New York City Police Department   \n",
       "2   60935450  2024-04-22T01:50:12.000   NYPD  New York City Police Department   \n",
       "3   60937446  2024-04-22T01:50:11.000   NYPD  New York City Police Department   \n",
       "4   60935436  2024-04-22T01:50:11.000   NYPD  New York City Police Department   \n",
       "\n",
       "        complaint_type        descriptor               location_type  \\\n",
       "0  Noise - Residential  Loud Music/Party  Residential Building/House   \n",
       "1   Noise - Commercial  Loud Music/Party         Club/Bar/Restaurant   \n",
       "2  Noise - Residential  Loud Music/Party  Residential Building/House   \n",
       "3   Noise - Commercial  Loud Music/Party         Club/Bar/Restaurant   \n",
       "4   Noise - Commercial  Loud Music/Party         Club/Bar/Restaurant   \n",
       "\n",
       "  incident_zip       incident_address      street_name  ...  \\\n",
       "0        11375          89-17 69 ROAD          69 ROAD  ...   \n",
       "1        10458    621 CRESCENT AVENUE  CRESCENT AVENUE  ...   \n",
       "2        11235          1245 AVENUE X         AVENUE X  ...   \n",
       "3        11373    92-02 CORONA AVENUE    CORONA AVENUE  ...   \n",
       "4        11423  184-15 JAMAICA AVENUE   JAMAICA AVENUE  ...   \n",
       "\n",
       "  :@computed_region_sbqj_enih :@computed_region_7mpf_4k6g  \\\n",
       "0                          70                          70   \n",
       "1                          31                          31   \n",
       "2                          36                          36   \n",
       "3                          68                          68   \n",
       "4                          61                          61   \n",
       "\n",
       "  taxi_pick_up_location closed_date resolution_description vehicle_type  \\\n",
       "0                   NaN         NaN                    NaN          NaN   \n",
       "1                   NaN         NaN                    NaN          NaN   \n",
       "2                   NaN         NaN                    NaN          NaN   \n",
       "3                   NaN         NaN                    NaN          NaN   \n",
       "4                   NaN         NaN                    NaN          NaN   \n",
       "\n",
       "  facility_type bridge_highway_name bridge_highway_segment  \\\n",
       "0           NaN                 NaN                    NaN   \n",
       "1           NaN                 NaN                    NaN   \n",
       "2           NaN                 NaN                    NaN   \n",
       "3           NaN                 NaN                    NaN   \n",
       "4           NaN                 NaN                    NaN   \n",
       "\n",
       "  bridge_highway_direction  \n",
       "0                      NaN  \n",
       "1                      NaN  \n",
       "2                      NaN  \n",
       "3                      NaN  \n",
       "4                      NaN  \n",
       "\n",
       "[5 rows x 44 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# SoQL query string below:\n",
    "# retrieve all columns and limit our records to 100\n",
    "\n",
    "query = (\n",
    "    \"\"\"\n",
    "    SELECT *\n",
    "    LIMIT 100\n",
    "    \"\"\"\n",
    ")\n",
    "\n",
    "# returned as JSON from API / converted to Python list of dictionaries by sodapy\n",
    "results = client.get(\n",
    "    socrata_dataset_identifier,\n",
    "    query=query\n",
    ")\n",
    "\n",
    "# convert to pandas DataFrame\n",
    "results_df = pd.DataFrame.from_records(results)\n",
    "client.close()\n",
    "\n",
    "print('shape of data: {}'.format(results_df.shape))\n",
    "results_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.3  .get_metadata()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`get_metadata` method: Retrieve the metadata for a particular dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id 70\n",
      "name 70\n",
      "assetType 70\n",
      "attribution 70\n",
      "averageRating 70\n",
      "category 70\n",
      "createdAt 70\n",
      "description 70\n",
      "displayType 70\n",
      "downloadCount 70\n",
      "hideFromCatalog 70\n",
      "hideFromDataJson 70\n",
      "newBackend 70\n",
      "numberOfComments 70\n",
      "oid 70\n",
      "provenance 70\n",
      "publicationAppendEnabled 70\n",
      "publicationDate 70\n",
      "publicationGroup 70\n",
      "publicationStage 70\n",
      "rowClass 70\n",
      "rowIdentifierColumnId 70\n",
      "rowsUpdatedAt 70\n",
      "rowsUpdatedBy 70\n",
      "tableId 70\n",
      "totalTimesRated 70\n",
      "viewCount 70\n",
      "viewLastModified 70\n",
      "viewType 70\n",
      "approvals 70\n",
      "clientContext 70\n",
      "columns 70\n",
      "grants 70\n",
      "metadata 70\n",
      "owner 70\n",
      "query 70\n",
      "rights 70\n",
      "tableAuthor 70\n",
      "tags 70\n",
      "flags 70\n"
     ]
    }
   ],
   "source": [
    "# source domain for NYC Open Data on Socrata\n",
    "socrata_domain = 'data.cityofnewyork.us'\n",
    "\n",
    "# dataset id for NYC 311 on NYC Open Data on Socrata\n",
    "socrata_dataset_identifier = 'erm2-nwe9'\n",
    "\n",
    "client = Socrata(\n",
    "    socrata_domain,\n",
    "    app_token=None,\n",
    "    timeout=100\n",
    ")\n",
    "\n",
    "# previewing keys\n",
    "for key in client.get_metadata(socrata_dataset_identifier).keys():\n",
    "    print(key, value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "434088"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# preview download count\n",
    "client.get_metadata(socrata_dataset_identifier)['downloadCount']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}