Viewing Resource Data

kindly edited this page Nov 9, 2013 · 4 revisions
Clone this wiki locally

Terminology

Version availability in brackets.

Resource: A Dataset in CKAN has many Resources. They consist of links to the data itself or sometimes link to information about the Dataset. (All versions)

Datastore: A feature of CKAN that allows storage of tabular data in CKAN. (1.8+)

DataProxy: A service which allows CKAN preview tabular data, without importing it to the Datastore. (1.7+)

Datastorer: This is a process that allows automatic uploading of tabular data to CKAN Datastore. (1.8+) https://github.com/okfn/ckanext-datastorer

DataPusher: A simple web service that allows you to automatically upload data to the Datastore. This is very similar to the Datastorer (see below) (2.2) https://github.com/okfn/datapusher

Previews: A feature of CKAN that previews CKAN resources. This includes tabular data. (1.7+)

Views: A feature of CKAN that allows you to configure views to be associated with a CKAN resource, these will be a super-set of the Previews currently available. This is very similar to Previews but allows you to assign multiple Views to a single resource. (2.3, poss 2.2)

Current Plans

For a while we have been trying to deprecate the user of the Dataproxy. This is because it has many limitations and issues (the main one only being able to preview small files before timeout).

We are trying to encourage everyone to try and import all their external tabular data into the Datastore, as it gets round these limitations and means we can do much more with the data. However, we realize that the process of getting the data into the Datastore could be easier as it is difficult to setup and install the Datatorer. As part of the 2.2 release we intend to replace the use of the Datastorer with the Datapusher.

The Datpushers aim is to make it much easier to deploy and also give the user more feedback if the process did not work. It also paves the way to making putting data into the Datastore much easier in general, by providing a nice UI for this. The Datastorer will still work for the foreseeable future nonetheless.

Currently when viewing data you use the Preview functionality. One of these previews currently looks very similar if you have the data in the Datastore of if you are getting data from the DataProxy.

The new Views functionality will offer more flexibility and a UI allowing the user to configure what is displayed against a resource. We are hoping to get rid of the Dataproxy for good at this point. In order to preview tabular data from other websites (without importing it into the Datastore) it will easy to add a new View type that does this. However this will be distinct from the new Views created to look the data within the Datastore.