Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
994 lines (732 sloc) 32.4 KB
**Unveil** is a data exploration and visualization toolkit that utilizes data-driven
software design.
It features generic data abstraction through Collections, a unified interface
for developing pluggable visualizations, and a Scenegraph implementation that
simplifies the usage of the the "HTML 5 Canvas API"[].
Although shipping with it's own little graphic library, the *Scene API*, Unveil.js
is designed to be used with any other visualization library. So in the first
place, it suggests a different approach on how you deal with domain data in a
visualization scenario.
This documentation is generated on the fly using the "Ndogen"[] documentation generator.
Since this instant doc-generation part is under heavy development, you'll
experience some glitches. We're working on it! Please have a look at the "raw markup"[] too.
The data visualization process
When faced with a data visualization task you'll usually go through a workflow
that looks like this:
1. Data aggregation
This includes querying and preparing data, which is not covered by Unveil.js.
However the outcome of this process should be a collection of data items
represented in a uniform Collection format (JSON), ideally served dynamically
through a webservice.
I've recently implemented such an aggregation service that uses data from
Freebase (in this example music artist data). Therefore I've used "Acre"[]
,a hosted platform that allows you to build applications or services from Freebase
"This simple Acre script"[!path=//] queries
the graph for music artists of certain genres. It processes that data and translate
it into the Unveil.js Collection format. The result is a web service that turns freebase data
into a Collection on the fly. And here is the "result"[].
Btw: I just used Freebase Acre for the first time. It's simply perfect for on-the-fly data-aggregation on the Freebase graph.
The collection of music artists I'm using for the stacks example is gained using this simple acre-script:!path=//
-> results in
2. Visualization
Now that you have a ready-to-be-visualized collection you can start implementing
a visualization, or use that aggregated data to power an existing vis.
The Unveil.js visualization should operate directly on the Collection passed to
the constructor. If I 've got it right you shouldn't need additional
data-structures for holding domain data (except for your visualization's state, of course) .
For querying within a Collection (which happens in memory) you can just traverse
the graph. Since every item / property / value has a unique key you can efficiently
access items in O(1).
Here's some more documentation regarding Collections I haven't migrated to the unveil doc page:
There's also a Criterion API for querying data within a collection.
And there's the concept of collection Transformers. Transformers turn collections into modified collections.
DataGraph API
You'd want to use the DataGraph API when it comes to visualizing linked data.
The DataGraph format takes the idea of the Semantic Web, tries to reduce the
complexity and applies the concepts if RDF to a local visualization scenario.
Collection API
Collections are the main building block of the toolkit. A *Collection* is a simple
data abstraction format where a data-set under investigation conforms to a
collection of data items that describes all facets of the underlying data in a
simple and universal way. You can think of a *Collection* as a table of data,
except it provides precise information about the data contained (meta-data).
*Collections* are represented as a JSON string, which serves as an exchange format.
Since JSON is heavily used in web-services, its perfectly suited for usage in a
cloud-like scenario. Your collections can be constructed on the fly using live
data from web services that either provides data directly in the Collection format
or through a translator service that serves as an adapter for an existing service.
After passing the raw format to the *Collection* constructor, they are
internally represented as a network of nodes (a graph so to say), that allows
efficient operations on the *Collection* by traversing the graph instead of
iterating over all items, values etc. Also operations like filtering, grouping
etc. are already supported.
$ var countries = new Collection({
$ "properties": "properties": {
$ "name": {
$ "name": "Country Name",
$ "type": "string",
$ "unique": true
$ },
$ "official_language": {
$ "name": "Official language",
$ "type": "string",
$ "unique": true
$ },
$ "form_of_government": {
$ "name": "Form of governmennt",
$ "type": "string",
$ "unique": false
$ },
$ "currency_used": {
$ "name": "Currency used",
$ "type": "string",
$ "unique": true
$ },
$ "population": {
$ "name": "Population",
$ "type": "number",
$ "unique": true
$ },
$ "gdp_nominal": {
$ "name": "GDP nominal",
$ "type": "number",
$ "unique": true
$ },
$ "area": {
$ "name": "Area",
$ "type": "number",
$ "unique": true
$ },
$ "date_founded": {
$ "name": "Date founded",
$ "type": "date",
$ "unqiue": true
$ }
$ },
$ "items": {
$ "arg": {
$ "name": "Argentina",
$ "official_language": "Spanish Language",
$ "form_of_government": [
$ "Federal republic",
$ "Presidential system"
$ ],
$ "currency_used": "Argentinian Peso",
$ "population": 39745613,
$ "gdp_nominal": 338700000000.0,
$ "area": 2780403.0,
$ "date_founded": "1816-07-09"
$ },
$ ...
$ },
$ });
The *Collection* constructor takes a collection represented as a JavaScript object,
which conforms to a parsed *JSON Collection String*. Once created you can query
the information contained.
The API features a simple object model for working with such in-memory
collections. The *Collection* object gives you access to the properties and
items contained.
Please make yourself familiar with the *Node API*, which is the data
structure used for modeling the internal structure of a collection, which is a
graph. Be aware that *Collection*, *Property*, *Item* and *Value* objects inherit
from *Node*, so the full *Node API* is available on such objects.
Meta-data (data about data) is represented as a set of properties that belongs
to a collection. A *Property* (cmp. a column in a table) holds a key, a name
(cmp. a header of a column) and a type (telling whether the data is numeric or
textual, etc.).
**Access the Collection's properties:**
$ var properties = countries.all('properties'); // => SortedHash
**You can also access them directly:**
$ var population = countries.get('properties', 'population');
**And you can use property inspection:**
$ population.type; // => 'string'
$; // => 'Population'
$ population.unique; // => true
An item of the collection conforms to a row in a data table, except one ‘cell’
can have arbitrary many values (non-unique attributes).
**List all items:**
$ var items = countries.all('items'); // => SortedHash containing Item objects
**Or access them directly:**
$ var argentina = countries.get('items', 'arg'); -> Item#arg
**Inspect item values:**
$ argentina.value('name'); // => 'Argentina'
$ argentina.value('currency_used'); // => 'Argentinian Peso'
**Non-unique properties have more that one value:**
$ argentina.values('form_of_government');
$ // => [String#Federal_Republic, String#Presidential_system]
**With *SortedHash#each* you can easily iterate over values:**
$ argentina.values('form_of_government').each(function(index, gf) {
$ gf; // => e.g. 'Presidential System'
$ });
You can not only view values on a raw level but also as Nodes, which provide
useful meta-information about connections etc.
**List all items:**
$ var items = countries.all('items'); // => SortedHash containing Item objects
**Inspect the value 'Federal republic':**
$ federal_republic = argentina.all('form_of_government').first();
$ // => Node#Federal_Republic
**Which other countries also have this government form?**
$ federal_republic.all('items'); // => [Node#austria, Node#usa]
Set operations
Since a *SortedHash* conforms to a set in mathematical respect, some interesting
operations can be performed.
$ var english = countries.all('items').select(function(key, i) {
$ return i.value('official_language') === 'English';
$ });
$ var french = countries.all('items').select(function(key, i) {
$ return i.value('official_language') === 'French';
$ });
$ // english and french speaking countries
$ var english_and_french = english.intersect(french); // => [Node#canada]
$ // english or french speaking countries
$ var english_or_french = english.union(french);
$ // => [Node#uk, Node#france, Node#Switzerland, ...]
Transformers [$TRANSFORMERS]
Transformers are used to do operations on the desired *Collection*. A commonly
used one is the group transformation.
The following code does a grouping by the *official_language* property and sums
up all numeric properties (like population, area). What you get is a modified
collection that shows aggregated values for all occurring languages.
$ var languages = countries.transform('group', {
$ property: 'official_language',
$ aggregator: 'SUM'
$ });
**The resulting collection looks like this:**
$ {
$ "properties": "properties": {
$ "official_language": {
$ "name": "Official language",
$ "type": "string",
$ "unique": true
$ },
$ "population": {
$ "name": "Population",
$ "type": "number",
$ "unique": true
$ },
$ "gdp_nominal": {
$ "name": "GDP nominal",
$ "type": "number",
$ "unique": true
$ },
$ "area": {
$ "name": "Area",
$ "type": "number",
$ "unique": true
$ }
$ },
$ "items": {
$ "English": {
$ "population": 397445613,
$ "gdp_nominal": 4341700000000.0,
$ "area": 2780403.0,
$ },
$ "German": {
$ "population": 91560000,
$ "gdp_nominal": 4341700000000.0,
$ "area": 2780403.0,
$ },
$ ...
$ }
$ };
Transformers are non-destructive. The original *Collection* is not modified unless
you re-assign the result.
Implement your own Transformer
All you have to do is to assign a new *Property* on the pv.Collection.transformers
object, which holds a transformer function. This function takes an input
collection and a params hash.
$ = function(c, params) {
$ // hardcore grouping action that yields a modified output collection
$ return outputColletion;
$ };
Have a look at the implementation of the "Group Transformer"[], to see how it's done.
**Transformer specification:**
You need to define a specification for your *Transformer*, which gives it a
human readable name and describes the parameters your transformation takes.
$ = "Group By";
$ = {
$ properties: {
$ name: "Property",
$ type: "property_list"
$ },
$ aggregator: {
$ name: "Aggregator Function",
$ type: "aggregator"
$ }
$ }
Available Collections
Some sample collections are available through "Collectionize"[], a dedicated
aggregator service, that translates interesting web services to the *Collection* format.
**Currently available:**
* "Countries fetched from"[]
* " Playlists"[]
Visualization API [$VIS]
The *Visualization API* provides a simple abstraction layer for visualizations to ease
the process of creating re-usable data-driven visualizations.
The appearance of a visualization is determined by the underlying data rather than by
user defined plotting options. Visualizations directly access data trough a well defined
interface, the *Collection API*, so there's no longer a gap between domain data and
data used by the visualization engine.
Such visualization can be re-used in terms of putting arbitrary data in.
This works as long as the data is a valid *Collection* and satisfies the
visualization specification (some visualizations exclusively use numbers as their
input, others use dates (e.g. Timeline plots), and so on...).
$ // load some data (represented as a Collection)
$ c = new uv.Collection(countries);
$ // construct a visualization based on that data
$ vis = new uv.Linechart(c, {
$ measures: ['population'],
$ params: {scale: 'linear'}
$ });
$ vis.render();
Creating visualizations
In order to create your own visualizations you need to setup a new class that
inherits from *uv.Visualization*.
$ var MyChart = uv.Visualization.extend({
$ constructor: function(collection, options) {
$, collection, options);
$ },
$ render: function() {
$ // hardcore rendering action
$ }
$ });
All you have to do is to implement *render()*. It's totally up to you whether you
want to use the included Scene API or any other visualization library (I can recommend "Protovis"[], "Processing.js"[] and "Scene.js"[]) or
work with the native API's (Canvas, SVG, WebGL).
Visualization Specification
There are various types of visualizations. Some exclusively use numbers as their
input, some use dates (e.g. Timeline plots) others visualize relationships between
data items (e.g. which countries share the same currency). There are further
visualizations that use various combinations of measure types.
In order to verify that the current selection of measures can be displayed by the
chosen visualization you have to define a specification like this:
$ // Displays 1..n number properties
$ uv.Barchart.spec = {
$ measures: [
$ {
$ types: ['number'],
$ unique: true,
$ cardinality: 1
$ },
$ {
$ types: ['number'],
$ unique: true,
$ cardinality: "*"
$ }
$ ]
$ };
**Have a look at the implementations of *Scatterplot* and *Linechart* for reference:**
* "Scatterplot"[]
* "Linechart"[]
Scene API [$SCENE]
Inspired by various great visualization libraries (Protovis, Processing.js), I
created my own rather low level graphical library that is basically an
implementation of a Scenegraph on top of the *HTML5 Canvas API*.
It's more a framework that helps you managing complexity rather than a full-featured
graphical visualization library.
**I focussed on two core principles:**
* Object oriented
* Declarative
Object oriented in terms of thinking in graphical objects and modularizing
code, declarative in terms of using, configuring and combining existing *Actors*
(graphical objects) and attach them to the Scenegraph.
**Creating a scene is easy:**
$ var scene = new uv.Scene({
$ fillStyle: '#ccc'
$ });
Once you have constructed your *Scene* object, you can start adding actors.
*Actors* can be primitive graphical objects (like a Bar, a Line, or a Dot) or higher
level objects that combine lower level ones (e.g. a Hyperbolic Tree). Actors
typically take properties in their constructor. However for higher level objects
you probably want them to be constructed with real domain data instead of simple
graphical oriented properties (width, height etc.). You can decide on your own
how you want to shape the interface.
**Add an Actor (Bar) to the scene:**
$ var bar = new uv.Bar({
$ x: 30,
$ y: 50,
$ width: 30,
$ height: 80,
$ fillStyle: 'darkblue'
$ });
**Add an additional bar as a child of the bar just created:**
$ bar.add(new uv.Bar({
$ x: 50,
$ y: 20,
$ width: 20,
$ height: 80,
$ fillStyle: 'lightblue'
$ }));
The x and y coordinates are relative to the parent object. So an object does not
know where it is located in the coordinate-space. It just renders itself at
position 0,0. The positioning is done through matrix transformations, where for
each object the current context (transformation matrix) is being calculated on
every frame. That's how a SceneGraph works.
**Specify an output display:**
$ scene.display({
$ container: $('#plotarea'),
$ width: 800,
$ height: 300,
$ zooming: true,
$ paning: true
$ });
The scene (world-coordinates) can be projected to one or many displays (canvas elements), that
have a local coordinate system. Each display can be modified (zoomed, paned, etc.) independently.
This conforms to a view transformation in computer-game-engine-like API's (camera analogy).
You can see multiple displays in action "here"[].
**Start the scene:**
$ // defaults to 10 frames per second
$ scene.start();
The scene automatically refreshes each attached display appropriately (on every frame).
**Create your own actors:**
Creating visualizations using the *Scene API* most often means implementing your own *Actors*.
I'd see the creation of custom Actors as a fundamental part of the visualization
creation process. So instead of combining only existing pre-implemented objects, you're
encouraged to create your own ones, suitable for your needs.
**Let's re-create the Bar Actor from scratch, and add some interaction and animation later:**
$ uv.Bar = function(properties) {
$ // super call
$, uv.extend({
$ width: 30,
$ height: 50,
$ strokeWeight: 2,
$ strokeStyle: '#000',
$ fillStyle: '#ccc'
$ }, properties));
$ };
**Every Actor inherits from uv.Actor:**
$ uv.Bar.prototype = uv.extend(uv.Actor);
**Implement a draw method, which takes a Canvas 2D context:**
$ uv.Bar.prototype.draw = function(ctx) {
$ ctx.fillRect(0, 0,,;
$ };
Interaction is key in visualizations. Therefore Unveil.js aims to provide an
abstraction for implementing interaction on your *Actors*. Unlike in SVG, with
the Canvas API you can't attach event handlers to shapes directly. Instead
you need to detect on your own, which objects are currently under the cursor.
Usually you'd do this with some math, but there's a simpler approach that
utilizes *isPointInPath()*.
With *isPointInPath()* you can check if the current mouse-position is inside
the current working path.
To make use of this you need to equip your Actors with an additional
*drawMask()* method.
**Let's add some interaction to our Bar:**
$ uv.Bar.prototype.drawMask = function(ctx) {
$ ctx.beginPath();
$ ctx.moveTo(0, 0);
$ ctx.lineTo(, 0);
$ ctx.lineTo(,;
$ ctx.lineTo(0,;
$ ctx.lineTo(0, 0);
$ };
This simply draws an invisible rectangle. If you have a more complex object like
a star-shape you can use a rectangle (also known as a bounding-box) as well, to
mask your object for interaction. However, that's for the lazy folks. ;-)
Actors that have a *drawMask()* implementation can then be easily checked
against the current cursor position.
**That's what is going on behind the curtain:**
$ uv.Actor.prototype.checkActive = function(ctx, mouseX, mouseY) {
$ if (this.drawMask && ctx.isPointInPath) {
$ this.drawMask(ctx);
$ if (ctx.isPointInPath(mouseX, mouseY))
$ = true;
$ else
$ = false;
$ }
$ };
It's easy to add animation to your objects too. I've added a "Tween class"[]
I created some time ago to animate properties over time.
**To animate the height property of a Bar you simply use a Tween for that:**
$ uv.Bar = function(properties) {
$ ...
$ this.t = new uv.Tween({
$ object:,
$ property: 'height'
$ });
$ };
**The *updateHeight()* method is used to trigger the Tween:**
$ uv.Bar.prototype.updateHeight = function(newHeight) {
$ this.t.continueTo(newHeight, 1.5);
$ };
**For now you need to manually trigger a Tween tick using the *Actor#update()* method:**
$ uv.Bar.prototype.update = function() {
$ this.t.tick();
$ };
**In your sketch you can use *setTimeout()* or *setInterval()* to trigger the *updateHeight()* method:**
$ setInterval(function() {
$ bar.updateHeight(70);
$ }, 1000);
Here's an example of "Moving Random Bars"[],
that feature interaction and animation.
*Commands* are used to modify properties on the scene. They can be executed
one or many times, and they can be unexecuted to recover the original state.
Such commands are a powerful abstraction for modifying state. Implement your own
commands whenever it makes sense.
What always annoyed me is the problem of high CPU-consumption that canvas-based visualizations
typically have. The reason for that is, that if your scene runs on 60 fps, then even when there's
no animation or interaction taking place, the *render()* method is called 60 times a second.
**I could solve this by implementing a *RequestFramerate* command:**
$ uv.cmds.RequestFramerate = function(scene, opts) {
$ this.scene = scene;
$ this.requests = 0;
$ this.framerate = opts.framerate;
$ this.originalFramerate = this.scene.framerate;
$ };
$ uv.cmds.RequestFramerate.className = 'RequestFramerate';
$ uv.cmds.RequestFramerate.prototype.execute = function() {
$ this.requests += 1;
$ this.scene.framerate = this.framerate;
$ };
$ uv.cmds.RequestFramerate.prototype.unexecute = function() {
$ this.requests -= 1;
$ if (this.requests <= 0) {
$ this.scene.framerate = this.originalFramerate;
$ }
$ };
**Having that command in place, we need to register it on the scene: **
$ scene.register(uv.cmds.RequestFramrate, { framerate: 60 });
**Then this command can be excuted and unexecued on demand.:**
$ t = new uv.Tween({...});
$ t.on('start', function() { that.scene.execute(uv.cmds.RequestFramerate); });
$ t.on('finish', function() { that.scene.unexecute(uv.cmds.RequestFramerate); });
**It works like this:**
The scene by default runs in idle mode (say 10fps),
which is enough to deal with mouse interaction. If an operation (in this case
a Motion Tween) needs high speed (60fps) it requests it, like if it would
request a resource.
The *Tween* class (like any other operation that needs full power) needs callbacks
for start and finish events, that can be used to request high speed (on start) and
release this demand when 'finish' is fired.
It basically works like a semaphore, where each request increments a *requests*
counter, and each release decrements it. When all motion tweens have finished
*requests* is zero. The current framerate then depends on the value of
*requests* (0 = idle mode, > 0 = highspeed / full throttle mode).
I've updated the "stacks example"[],
that now uses 60fps on demand (10fps per default). Thankfully lots of CPU cycles could
be saved. ;-)
Modification Matrix
While most often specifying properties (x, y, scaleX, scaleY, rotation) on the
Actor suffices, there may be cases where you need more control. Therefore
Actors expose a so called *Modification Matrix*, which can be directly modified
by the user.
**To scale and rotate an object around a point, you can specify a series of matrix transforms:**
$ var b = new uv.Bar({...});
$ // move the coordinate system to the desired point
$ b.translate(40,40);
$ // scale around this point (= new origin)
$ b.scale(1.5, 1.5);
$ // rotate 45°
$ b.rotate(Math.PI/4);
$ // move the coordinate system back
$ b.translate(-40, -40);
To calculate the resulting transformation matrix for drawing, it is
initialized with the values of the specified properties and then multiplied with
the modification matrix.
* "Stacks"[]
* "Random Bars"[]
* "Multiple displays"[]
The Scene API related code is pretty small (~ 400 LOC). I don't plan to provide
a full-featured graphic library with helper functions, since there are
existing libraries, that do a perfect job. I recommend using Protovis along with
the Scene API. Actually, this library was created because Protovis does not
support the canvas element. I also wanted an easier way to deal with interaction.
To prevent from verbose and messy code, I do not provide any backward
compatibility for older browsers. I'm trying to stay on the edge of technology.
Therefore I focus on supporting the most recent versions of Google Chrome,
Firefox, Safari. I'll probably support Internet Explorer 9 as well, since their
Canvas implementation looks promising. Currently mouse interaction is not working
in Firefox. Please use a webkit-based browser for the moment.
Node API [$NODE]
Node (not to be confused with Node.js) is a JavaScript Graph implementation that
hides graph complexity from the interface. It introduces properties, which group
types of edges together. Therefore multi-partit graphs are possible without any
hassle. Every Node simply contains properties which conform to outgoing edges.
It makes heavy use of hashing through JavaScript object properties to allow
random access whenever possible.
The Node API heavily relies on *SortedHash*, please have a look at
the documentation before you start.
$ // Initialize a plain Node
$ var austria = new Node(),
$ germany = new Node(),
$ uk = new Node();
$ // initialize with raw Value (raw values are typically stored in leave nodes)
$ var eu = new Node({value: 'European Union'}),
$ austrian = new Node({value: 'Austrian'}),
$ english = new Node({value: 'English'}),
$ german = new Node({value: 'German'}),
$ barroso = new Node({value: 'Barroso'});
Connect nodes through properties
$ austria.set('languages', 'at', austrian);
$ austria.set('languages', 'ger', german);
$ eu.set('president', 'barroso', barroso);
$ // Backlinks
$ german.set('spoken_in', {
$ 'at': austria,
$ 'de': germany
$ });
Get connected nodes
$ austria.all('languages') // => [Node#austrian, Node#german]
$ eu.first('president') // => [Node#barroso]
$ german.all('spoken_in') // => [Node#austria, Node#germany]
$ austria.all('languages').each(function(index, node) {
$ node; // => Node#at | Node#ger
$ index; // => 0 | 1
$ });
$ austria.all('languages').eachKey(function(key, node) {
$ node; // => Node#at | Node#ger
$ key; // => 'at' | 'ger'
$ });
Unveil.js features a *SortedHash* data structure that provides a simple layer of
abstraction for managing Sorted Hashes in JavaScript. It's heavily used
throughout the framework.
$ var items = new SortedHash();
$ items.set('at', 'Austria');
$ items.set('de', 'Germany');
**Hash Semantics:**
$ items.get('at') // => 'Austria';
$ items.get('de') // => 'Germany';
**Array Semantics:**
$; // => 'Austria'
$; // => 'Germany'
$ items.length; // => 2
$ items.each(function(index, value) {
$ value; // => 'Austria', 'Germany'
$ index; // => 0, 1
$ });
$ items.eachKey(function(key, value) {
$ value; // => 'Austria', 'Germany'
$ key; // => 'at', 'de'
$ })
$ var desc = function(item1, item2) {
$ return item1.value === item2.value
$ ? 0 : (item1.value > item2.value ? -1 : 1);
$ },
$ sortedItems;
$ sortedItems = items.sort(descendingOrder);
$ var mappedItems = (item) {
$ return item.slice(0, 3);
$ });
$ // leave original SortedHash untouched
$ // => 'Aus';
$ // => 'Ger';
This returns a SortedHash in descending order, while the original remains
$ var selectedItems = (key, item) {
$ return item === 'Austria';
$ });
$ // => 'Austria';
$ selectedItems.length; // => 1;
$ var items2 = new SortedHash(),
$ intersectedItems;
$ items2.set('fr', 'France');
$ items2.set('at', 'Austria');
$ intersectedItems = items.intersect(items2);
$ intersectedItems.length; // => 1
$ intersectedItems.get('at'); // => 'Austria'
$ var items2 = new SortedHash(),
$ unitedItems;
$ items2.set('fr', 'France');
$ items2.set('at', 'Austria');
$ intersectedItems = items.intersect(items2);
$ unitedItems.length; // => 3
$ unitedItems.get('at'); // => 'Austria'
$ unitedItems.get('de'); // => 'Germany'
$ unitedItems.get('fr'); // => 'France'
**Method chaining:**
$ var mappedAndSortedItems = (item) {
$ return item.slice(0, 3);
$ })
$ .sort(descendingOrder);
$ // => 'Ger';
$ // => 'Aus';