Skip to content

Datalets

Andrea Petta edited this page Oct 6, 2016 · 2 revisions

A datalet is a view WC, which is used to create rich, reusable visualization of open data. All the processing is performed client-side by the user browser, so that ensures scalability in terms of number of visualization (compared to others libraries as Google Chart). This is an architectural choice following the Performance Efficiency non-functional requirement (see Sec. 2.2.1.1). The datalet can query data providers provided that they return data in JSON or CSV format. All the datalets inherit from a base web-component called base-datalet. The datalet runs on the client-side within the user browser, thereby ensuring scalability on server-side: no additional heavy load is placed on the server, besides unloading few additional lines of JavaScript. In order to visualise a dataset, the following steps need to be performed. The user types the URL of a Web page which contains a datalet. The browser during the page content loading, takes the datalet URL and connects to the “datalets repository”. Then it loads the datalet source code to be executed (different visualization are rendered using different datalets). The datalet takes in input any external dataset URL and a query to recover the data. An optional filter can be applied to further refine data. Finally, the datalet renders an interactive visualization by using the filtered data.

Datalets guarantee the provenance of data by (1) loading the dataset from the external source without changing the data, (2) providing the URL of the data source so that the user can check the visualised data on the original source, (3) introducing a certified repository of datalets. Datalets have been designed to process any dataset as input. In the current implementation, datalets can visualise data from CKAN installations, and from commercial providers, such as OpenDataSoft2, and Enigma3.

The base datalet are divided in two categories: BaseDataletsBehaviors and CustomDataletBehaviors. The BaseDataletBehaviors define the lifecycle for all DATATLETs. Every lifecycle step in BaseDataletBehaviors is implemented in the CustomDataletBehaviors. This mechanism provides a components’ hierarchy that standardize datalet production and make reusable common behaviors. The BaseDataletBehaviors are:

  • BaseDataletBehavior;
  • WorkcycleBehavior.

The BaseDataletBehavior defines the mandatory attributes that all datalet must have:

  • data_url (String): the URL used to get the data from the open data Provider (e.g. the CKAN API).
  • fields (String): it represents a JSON array of user selected dataset fields.
  • data (array): data structure that store the data retrieved from open data Provider.

The WorkcycleBehavior implements the work-cycle of each datalets, composed by the following steps:

  • Get the data from a source: It is reponsible for data retrieving from an open data Platform.
  • Select a subset of fields from the data: this operation is data format dependent. It is responsible for extract a query related subset of information from the entire dataset. A multidimensional array will be made available for the transformation step.
  • Filter the data by applying one of the following operators (<, >, =, <=, >=, etc.).
  • Transform data in order to obtain a coherent data representation.
  • Present the data for a visualization.

Datalets in SPOD

In SPOD platform, everyone can, without no technical expertise, create and share visualization of a chosen dataset with a controllet, with the following simply steps:

  • Upload open data dataset from recommended datasets by SPOD, or a search suggested by a treemap, or external (remote) dataset via API;
  • Select the field of the chosen dataset;
  • Select from a variety of visualizations with customized chart (title of the chart, etc.);
  • Share your visualization over the SPOD.

The datalet life-cycle in SPOD can be described in the following way:

  1. SPOD provides a page with a datalet embedded (with all the parameters in the URL);
  2. The browser gets the datalet from the Dealer DEEP;
  3. The datalet gets the dataset from the provider and selects the data;
  4. Visualization is shown in the page.
You can’t perform that action at this time.