Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
7 changed files
with
307 additions
and
546 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
167 changes: 167 additions & 0 deletions
167
bindings/python/pydeck/examples/04 - Plotting massive data sets.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Plotting massive data sets\n", | ||
| "\n", | ||
| "**This notebook is experimental in pydeck's beta version. It may not work on all devices.**\n", | ||
| "\n", | ||
| "This notebook plots 1.6 million points of LIDAR points around the Carnegie Mellon University campus. ([Source](https://github.com/ajduberstein/oakland_point_cloud)) The data points are labeled. With pydeck, we can render these points and interact with them. We'll downsample this data to 800,000 points–depending on your device, you may be able to run the full data set." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Cleaning the data\n", | ||
| "\n", | ||
| "First we need to import the data. We should expect about 1.6M points." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import pandas as pd\n", | ||
| "\n", | ||
| "URL = 'https://raw.githubusercontent.com/ajduberstein/oakland_point_cloud/master/%s'\n", | ||
| "DATA_URL_1 = URL % 'lidar_chunks_1.csv'\n", | ||
| "DATA_URL_2 = URL % 'lidar_chunks_2.csv'\n", | ||
| "LOOKUP_URL = URL % 'ground_truth_label.csv'\n", | ||
| "lidar = pd.concat([pd.read_csv(DATA_URL_1), pd.read_csv(DATA_URL_2)])\n", | ||
| "lookup = pd.read_csv(LOOKUP_URL)\n", | ||
| "lidar = lidar.merge(lookup)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "print('Number of points:', lidar.count()[0])" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "It does not appear to be in a standard coordinate format, so we'll scale it to make it easy to plot on a map. We'll also color objects by label type. The `data_utils.assign_random_colors` assigns a random RGB value to a vector of data labels." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from pydeck import data_utils\n", | ||
| "\n", | ||
| "color_lookup = data_utils.assign_random_colors(lidar['label_name'])\n", | ||
| "lidar['rgb'] = lidar.apply(lambda row: color_lookup.get(row['label_name']), axis=1)\n", | ||
| "# Scaling the points using min-max scaling\n", | ||
| "lidar[['x', 'y', 'z']] -= lidar[['x', 'y', 'z']].max()\n", | ||
| "lidar[['x', 'y', 'z']] /= lidar[['x', 'y', 'z']].min()\n", | ||
| "lidar[['x', 'y']] /= 1000\n", | ||
| "lidar[['z']] *= 10\n", | ||
| "lidar.head()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Plotting the data\n", | ||
| "\n", | ||
| "We'll define a single `PointCloudLayer` and plot it.\n", | ||
| "\n", | ||
| "pydeck by default expects the input of `get_position` to be a string name indicating a single position value. For convenience, you can pass in a string indicating the X/Y/Z coordinate, here `get_position='[x, y, z]'`.\n", | ||
| "\n", | ||
| "We'll zoom to the approximate center of the data by taking a mean of a few hundred points in pandas.\n", | ||
| "\n", | ||
| "This example may take 10-15 seconds to render." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import pydeck as pdk\n", | ||
| "\n", | ||
| "point_cloud = pdk.Layer(\n", | ||
| " 'PointCloudLayer',\n", | ||
| " lidar[['x', 'y', 'z', 'label_name', 'rgb']].sample(800000),\n", | ||
| " # You can specify the XYZ coordinate in a list as a string\n", | ||
| " get_position='[x, y, -1*z]',\n", | ||
| " get_normal=[0, 0, 1],\n", | ||
| " get_color='rgb',\n", | ||
| " pickable=True,\n", | ||
| " auto_highlight=True,\n", | ||
| " radius_pixels=3)\n", | ||
| "\n", | ||
| "\n", | ||
| "view_state = pdk.data_utils.compute_viewport(lidar[['x', 'y']])\n", | ||
| "view_state.max_zoom = 23\n", | ||
| "view_state.zoom = 22\n", | ||
| "view_state.pitch = 60\n", | ||
| "\n", | ||
| "\n", | ||
| "r = pdk.Deck(\n", | ||
| " point_cloud,\n", | ||
| " initial_view_state=view_state,\n", | ||
| " map_style='',\n", | ||
| " width=700)\n", | ||
| "r.show()" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%%javascript\n", | ||
| "// Use Javascript to change the cell background to black\n", | ||
| "document.getElementById(\"deckgl-overlay\").style.backgroundColor = \"black\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "#### Citations:\n", | ||
| "\n", | ||
| "Contextual Classification with Functional Max-Margin Markov Networks. \n", | ||
| "Daniel Munoz, J. Andrew (Drew) Bagnell, Nicolas Vandapel, and Martial Hebert. \n", | ||
| "IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June, 2009." | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.7.4" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
Oops, something went wrong.