Streaming Data

Jeffrey Heer edited this page Mar 7, 2017 · 9 revisions

This wiki documents Vega version 2. For Vega 3 documentation, see vega.github.io/vega.

WikiDocumentationStreaming Data

Vega 2.0 supports streaming data through a set of operators that can be used within the specification, and API endpoints on the View component.

Streaming Operators

Within a dataset definition, a pipeline of streaming operators can be defined under the modify property. These operators take the following properties:

Property Type Description
type String Either insert, remove, or toggle.
signal String A signal name, with dot notation to reference nested properties. The value of this signal is inserted, removed, or toggled from the dataset.
field String | Array<String> A field name under which the signal value is inserted. When removing or toggling signal values, an array of field names may be specified to find existing matches within the dataset. Field names are optional if the specified signal contains an object value. If field names are provided, they are used as an equality criterion for the remove and toggle operations.

Streaming operators are only evaluated when a signal value changes. They run after all data transformations for the corresponding data set have been executed. In the event that multiple operators use the same signal, operators are evaluated in their definition order.

A special clear operator is also available, and takes a predicate property. This operator clears the dataset whenever the given predicate evaluates to true.

Here is an excerpt of the shift-click example that demonstrates the use of streaming operators in the specification. Clicking a point selects it, and multiple points can be selected when the shift key is pressed. Data values of the form {"point_id": 5} are added and removed from the selectedPts dataset based on user interaction.

{
  "signals": [
    {
      "name": "clickedPt",
      "streams": [{"type": "click", "expr": "datum._id"}]
    },
    {
      "name": "shift",
      "streams": [{"type": "click", "expr": "event.shiftKey"}]
    }
  ],

  "data": [
    {
      "name": "selectedPts",
      "modify": [
        {
          "type": "clear",
          "predicate": "clearPts"
        },
        {
          "type": "toggle",
          "signal": "clickedPt",
          "field": "point_id"
        }
      ]
    }
  ]
}

View Component Streaming API

Parsing a Vega specification, and rendering a visualization, produces a View Component which exposes an API. The following API methods are available to support streaming data externally:

Method Description
view.data(name) Returns a streaming API for the name dataset. The following methods are available, and can be invoked using a method chaining style:
data.insert(values) Inserts the given array of data values into the dataset.
data.update(where, field, modify) Updates the value of field for all data values that match the where condition. where is a function that is called with each data value, and must return true or false. modify is a function that is called with each matching data value, and must return the new field value.
data.remove(where) Removes all data values that match the where condition -- a function that is called for each data value, and must return true or false.
data.values() Returns all values currently in the dataset.

For convenience, an object can also be passed to the data API method where keys correspond to dataset names, and values are functions that receive the corresponding dataset's streaming API.

Once a dataset has been modified, a view.update call must be executed in order to update the visualization. For example,

view.data('table')
  .insert([{"x": 1}, {"x": 2}])
  .update(function(d) { return d.x > 5; }, 'y', function(d) { return d.y + 5; });

view.data({
  table2: function(data) {
    data.remove(function(d) { return d.x > 5; });
  },
  table3: function(data) {
    data.insert([{"x": 4}, {"x": 5}]);
  }
});

view.update();

Note: If it appears that executing streaming API methods has no effect on the visualization, ensure that the visual properties of marks are defined over the update property set (as opposed to enter). See this D3 tutorial for more information.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.