No description, website, or topics provided.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
R bringing new C based implementation of nanocube into main github project Nov 9, 2017
ac_aux disable AM_MAINTAINER_MODE Aug 30, 2018
api updated doc May 22, 2018
data update May 25, 2018
doc
ide bringing new C based implementation of nanocube into main github project Nov 9, 2017
m4
misc bringing new C based implementation of nanocube into main github project Nov 9, 2017
old updated documentation on v4 May 22, 2018
scripts update May 24, 2018
src using xdoc.c to build docs Nov 13, 2018
templates
tests
web Added div by total for polygon maps Oct 15, 2018
EXTRA.md update May 24, 2018
MAPPING.md update May 25, 2018
Makefile.am update May 24, 2018
Makefile.in disable AM_MAINTAINER_MODE Aug 30, 2018
NOTES.md
README.md
aclocal.m4 disable AM_MAINTAINER_MODE Aug 30, 2018
config.h.in disable AM_MAINTAINER_MODE Aug 30, 2018
configure disable AM_MAINTAINER_MODE Aug 30, 2018
configure.ac disable AM_MAINTAINER_MODE Aug 30, 2018

README.md

Nanocubes: an in-memory data structure for spatiotemporal data cubes

Nanocubes are a fast data structure for in-memory data cubes developed at the Information Visualization department at AT&T Labs Research. Visualizations powered by nanocubes can be used to explore datasets with billions of elements at interactive rates in a web browser, and in some cases nanocubes uses sufficiently little memory that you can run a nanocube in a modern-day laptop.

About this branch

This branch (v4) contains a new implementation of Nanocubes in the C programming language. The goal with this new implementation was to get a much finer control in all aspects of the data structure and specially on its memory aspects (allocation, layout). In our original C++ template-based implementation of Nanocubes (up to version 3.3), we implemented the Nanocube data structure on top of C++ STL (standard library) and while this was a reasonable solution at the time, it had some important downsides: (1) complex serialization which made it hard to save/load Nanocube into files; (2) variations in the internal memory layout of a Nanocube based on the specific STL implementation we used.

Here is a link to the new API

Compiling on Linux or Mac

# Dependencies for Ubuntu 18.04
# sudo apt install build-essential curl unzip
#
# Dependencies for Mac OS X 10.13.4
# XCode

# get the v4 branch
curl -L -O https://github.com/laurolins/nanocube/archive/master.zip
unzip master.zip
cd nanocube-master

# modify INSTALL_DIR to point to another installation folder if needed
export INSTALL_DIR="$(pwd)/install"
./configure --with-polycover --prefix="$INSTALL_DIR"
make
make install

# Test if nanocubes is working
$INSTALL_DIR/bin/nanocube

# Add nanocube binaries to the PATH environment variable
export PATH="$INSTALL_DIR/bin":$PATH

Creating and serving a nanocube index

# create a nanocube index for the Chicago Crime dataset (small example included)
# Inputs: (1) CSV data file, (2) mapping file (data/crime50k.map)
# Output: (1) nanocube index called data/crime50k.nanocube
nanocube create <(gunzip -c data/crime50k.csv.gz) data/crime50k.map data/crime50k.nanocube

# serve the nanocube index just created on port 51234
nanocube serve 51234 crimes=data/crime50k.nanocube &

# test querying the schema of the index
curl "localhost:51234/schema()"

# test querying the number of indexed records
curl "localhost:51234/format('text');q(crimes)"

# test querying the number of records per crime type
curl "localhost:51234/format('text');q(crimes.b('type',dive(1),'name'))"

For more information on .map files go to mapping files

For more query examples go to API

Viewer

# If you need to install pip (e.g. on MacOS)
# python <(curl https://bootstrap.pypa.io/get-pip.py) --user
python -m pip install --user requests future

# Setup a web viewer on port 8000 for the crimes nanocube previously opened 
# on port 51234.
#
# Parameters:
#     -s         nanocube backend server (any http heachable machine)
#     --ncport   nanocube backend port
#     -p         port of the webviewer to be open in the localhost
#

nanocube_webconfig -s http://`hostname -f` --ncport 51234 -p 8000

Zoom into the Chicago region to see a heatmap of crimes.

image

Extra

For more advanced information follow this link: extra