Latest commit 17629bd Oct 25, 2018
Permalink
..
Failed to load latest commit information.
assets/images Added Strange Attractors example (#632) Sep 10, 2018
dashboard Split OSM example into 1B and 2.7B versions, in and out of core May 19, 2017
getting_started Documentation simplifications for newer nbsite (#619) Sep 9, 2018
topics Updated attractors notebook to store colormaps (#662) Oct 6, 2018
user_guide Tiling (#636) Oct 25, 2018
FAQ.ipynb Documentation simplifications for newer nbsite (#619) Sep 9, 2018
README.md update environment activation instructions Sep 9, 2018
datasets.yml Fixed website issues (#558) Feb 2, 2018
environment.yml Updated and fixed test and example environments Dec 5, 2017
filetimes.py Conda build fixes (#435) Aug 10, 2017
filetimes.sh Fixed outdated field names Jul 14, 2017
filetimes.yml Update conda pkgs in filetimes.yml (#360) May 19, 2017
get_raster_data.sh added initial support for raster datasets May 16, 2016
index.ipynb Fixed link Oct 25, 2018
nyc_taxi-nongeo.ipynb Packaging, installation, and testing (#564) May 8, 2018
pcap_to_parquet.py Remove edge index from generated pcap data Sep 22, 2017
raster.py Packaging, installation, and testing (#564) May 8, 2018
small.yml Packaging, installation, and testing (#564) May 8, 2018
streaming-aggregation.ipynb Packaging, installation, and testing (#564) May 8, 2018
streaming.py Replace {get,set}() with getters and setters Sep 14, 2016
taxi_preprocessing_example.py Packaging, installation, and testing (#564) May 8, 2018
tiling.ipynb Tiling (#636) Oct 25, 2018

README.md

Datashader Examples

The best way to understand how Datashader works is to try out our extensive set of examples. Datashader.org includes static versions of the getting started guide, user manual, and topic examples, but for the full experience with dynamic updating you will need to install them on a live server.

These instructions assume you are using conda, but they can be adapted as needed to use pip and virtualenv if desired.

To get started, first go to your home directory and download the current list of everything needed for the examples:

Then run the following commands in your terminal (command) prompt, from wherever you saved environment.yml:

1. conda env create --file environment.yml
2. conda activate ds
3. datashader examples
3. cd datashader-examples

Step 1 will read environment.yml, create a new Conda environment named ds, and install of the libraries needed into that environment (including datashader itself). It will use Python 3.6 by default, but you can edit that file to specify a different Python version if you prefer (which may require changing some of the dependencies in some cases).

Step 2 will activate the ds environment, using it for all subsequent commands. You will need to re-run step 2 after closing your terminal or rebooting your machine, if you want to use anything in the ds environment. For older versions of conda, you may instead need to do source activate ds (mac/linux) or activate ds (windows).

Step 3 will copy the datashader examples from wherever Conda placed them into a subdirectory datashader-examples, and will then download the sample data required for the examples. (datashader examples is a shorthand for datashader copy-examples --path datashader-examples && datashader fetch-data --path datashader-examples.)

The total download size is currently about 4GB to transfer, requiring about 10GB on disk when unpacked, which can take some time depending on the speed of your connection. The files involved are specified in the text file datasets.yml in the datashader-examples directory, and you are welcome to edit that file or to download the individual files specified therein manually if you prefer, as long as you put them into a subdirectory data/ so the examples can find them. Once these steps have completed, you will be ready to run any of the examples listed on datashader.org.

Notebooks

Most of the examples are in the form of runnable Jupyter notebooks. Once you have obtained the notebooks and the data they require, you can run them on your own system using Jupyter:

cd datashader-examples
jupyter notebook

If you want the generated notebooks to work without an internet connection or with an unreliable connection (e.g. if you see Loading BokehJS ... but never BokehJS sucessfully loaded), then restart the Jupyter notebook server using:

BOKEH_RESOURCES=inline jupyter notebook --NotebookApp.iopub_data_rate_limit=100000000

Dashboard

An example interactive dashboard using bokeh server integrated with a datashading pipeline.

To start, launch it with one of the supported datasets specified:

python dashboard/dashboard.py -c dashboard/nyc_taxi.yml
python dashboard/dashboard.py -c dashboard/census.yml
python dashboard/dashboard.py -c dashboard/opensky.yml
python dashboard/dashboard.py -c dashboard/osm.yml

The '.yml' configuration file sets up the dashboard to use one of the datasets downloaded above. You can write similar configuration files for working with other datasets of your own, while adding features to dashboard.py itself if needed to support them.

For most of these datasets, if you have less than 16GB of RAM on your machine, you will want to add the "-o" option before "-c" to tell it to work out of core instead of loading all data into memory. However, doing so will make interactive use substantially slower than if sufficient memory were available.

To launch multiple dashboards at once, you'll need to add "-p 5001" (etc.) to select a unique port number for the web page to use for communicating with the Bokeh server. Otherwise, be sure to kill the server process before launching another instance.