Render millions of points on a map.
Go HTML JavaScript CSS
Latest commit 36024a1 Feb 2, 2017 @fogleman fixes
Permalink
Failed to load latest commit information.
loader fixes Apr 14, 2016
server fixes Feb 2, 2017
stitch fixes Feb 2, 2017
web use less ram by accumulating points in grid Apr 8, 2016
README.md Update README.md Apr 9, 2016
kernel.go Refactor: Tile, Kernel, Renderer Apr 8, 2016
layer.go fixes Apr 14, 2016
renderer.go fixes Feb 2, 2017
tile.go fixes Feb 2, 2017

README.md

High-density Point Maps

Render millions of points on a map.

Map

Demo Site

This demo shows 77 million taxi pickups in NYC - from January to June 2015.

https://www.michaelfogleman.com/static/density/

Dependencies

  • Go
  • Cassandra

Download

go get github.com/fogleman/density

Loading Data

Cassandra is used to store large amounts of data. Data is clustered by (zoom, x, y) so that all of the points inside of a tile can quickly be fetched for rendering, even faster than PostGIS with an index.

First, create a new keyspace and table to house the data.

create keyspace density
    with replication = {
        'class': 'SimpleStrategy',
        'replication_factor': 1
    }
    and durable_writes = false;

create table points (
    zoom int,
    x int,
    y int,
    lat double,
    lng double,
    primary key ((zoom, x, y), lat, lng)
);

Next, load points into the database from a CSV file using the loader script.

go run loader/main.go < input.csv

Several command line options are available:

Flag Default Description
-keyspace density Cassandra keyspace to load into
-table points Cassandra table to load into
-lat 0 CSV column index of latitude values
-lng 1 CSV column index of longitude values
-zoom 18 Zoom level to use for binning points

Just run the loader whenever you need to insert more data.

Serving Tiles

Once the data is loaded into Cassandra, the tile server can be run for rendering tiles on the fly.

go run server/main.go
Flag Default Description
-keyspace density Cassandra keyspace to load from
-table points Cassandra table to load from
-zoom 18 Zoom level that was used for binning points
-port 5000 Tile server port number
-cache cache Directory for caching tile images

Serving Maps

A simple Leaflet map is provided to display a base map with the point tiles on top in a separate layer.

cd web
python -m SimpleHTTPServer

Then visit http://localhost:8000/ in your browser!

TODO

  • tile rendering options
  • multiple layers