Skip to content


Subversion checkout URL

You can clone with
Download ZIP
visualization server
JavaScript CSS Scala Other
Failed to load latest commit information.
avro-loader/src Move to 2.11
css upload dialog works better now
dashboard/demos angular installed w/basic demo on admin page
data-server/src using column path as column category, doing prefix check to find colu…
http-common/src Move to 2.11
kafka Move to 2.11
loader/src/main/scala/nest/sparkle/loader fixes and blacklist support for streaming loading flat avro records (…
log4j Add request time metrics on api server http requests and websocket re…
logback Add request time metrics on api server http requests and websocket re…
project bump to cassandra 2.1.5 for local builds
protocol-test-kit/src/main/scala/nest/sparkle/time/protocol Move to 2.11
protocol/src/main fix some scaladoc comments that the 2.11 compiler catches
sbt bump to sbt 0.13.8
servers create it tests for util-kafka
spark-avro/src/main/scala/nest/sparkle/loader/spark/avro TaggedBlock and TaggedBlock2 cleanup
spark/src Move to 2.11
sparkle-store-tests/src check which column paths really exist, when querying entity/leafDataS…
sparkle-tests Move to 2.11
sparkle improve handing for empty streams
store-test-kit/src/main confirm that we shouldn't wrap a try around test failures
store/src/main Move to 2.11
test-kit Add request time metrics on api server http requests and websocket re…
util-kafka/src Move to 2.11
util-tests/src/test Add request time metrics on api server http requests and websocket re…
util/src Move to 2.11
.gitignore ignore mac fs cruft use correct sg files in unit tests
LICENSE Initial commit spark console fixes
ToDo adjust styling to make remove button more prominent Basic support for draft 3 of the http data api a basic performance test for on/off interval sum queries Initial commit Initial commit be more clear in the getting-started doc about which sbt sub project … dashboard columnPicker entity search: document log levels in the server code.
scalastyle-config.xml Explicitly declare procedure return type as Unit. Basic support for draft 3 of the http data api
version.sbt ensure the correct directory when getting git commit hash


This is the 0.5.x version of Sparkle.


Sparkle is a small suite of libraries for making interactive data visualizations. Sparkle includes a javascript client library, a scala/jvm server library and a well defined protocol for requesting and serving data for visualization.

As a standalone tool, sparkle gives users an easy way to make visualize data from a directory of .csv files. The graphs are interactively zoomable in a d3 based web interface. With some easy customization, graphs can be aggregated into dashboards, and customized with simple declarative javascript, and data read in from other sources such as Apache Kafka.

See Sparkle Intro Talk for an introduction presentation. See Visualize the Things for a brief introduction to sparkle, lessons on garbage collection with reative streams, and comments on the future of the Lambda Architecture.


Sparkle contains three core components:

Sparkle-Time Server

sparkle-time is a mini web server that provides HTTP and websocket apis for data visualization. Sparkle-time collects data from an extensible set of inputs (Apache Kafka and .csv/.tsv data files, hadoop and netcat support coming soon). Sparkle-time stores data in Cassandra. Sparkle-time also hosts an extensible set of data transformations that transform data on demand (e.g. for aggregation).

Sparkle-time can be run as a library inside another server, or it can run standalone if no customization is needed.


sg.js is a d3.js based javascript library for displaying interactive charts of potentially large data sets. Charts are customizable with declarative javascript, and extensible with d3. sg.js currently offers support for line charts, scatter plots, bar charts, etc. and is easily extensible. All charts support a drag-based zooming UI.

sg.js also includes a dashboard component for constructing pages that aggregate many charts on the same page. Graphs in the dashboard are resizable. The browser's back button works to undo/redo zoom navigation and chart resizing.

sg.js supports the sparkle-time api, enabling zooming charts of potentially huge server hosted data sets, the option of offloading data transformations to the server, and (soon) server pushed updates to locally displayed charts.

Sparkle Data API

To enable interoperation of multiple clients and servers, the sparkle-time server and sg.js javascript library speak a well-defined data protocol: the Sparkle Data Protocol. The protocol allows clients to request data transformations and data streams from a comformant server.

See Sparkle Data Protocol for details of the protocol. See Sparkle Transforms for the current set of built in transforms.


Sparkle is currently at version 0.5.x. Sparkle 0.6 will:

  • clean out legacy code
  • more fully support the Sparkle Data Protocol between visualization client and visualizaton server.

Many more future features are listed in the todo list

Using sparkle

See Using-sg

Contributing to Sparkle

See Contributing, Building, and javascript component model.

Something went wrong with that request. Please try again.