Skip to content

Commit

Permalink
Merge pull request #111 from noi-techpark/issue-97-improve-APIv2-desc…
Browse files Browse the repository at this point in the history 
…ription

Issue 97 improve API v2 description
  • Loading branch information
Peter Moser committed Apr 1, 2020
2 parents bd25246 + e1f673e commit f5313dc
Show file tree
Hide file tree
Showing 2 changed files with 244 additions and 82 deletions.
164 changes: 90 additions & 74 deletions source/datasets.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,14 @@
Datasets
========

The goal of the |odh| project is to make available datasets containing
The goal of the Open Data Hub project is to make available datasets containing
data about the South Tyrolean ecosystem, to allow third parties to
develop novel applications on top of them, consuming the exposed
data. These applications may range from a simple processing of
datasets to extract statistical data and to display the result in
different graphic formats like pie-charts, to far more complex
applications that combine data from different datasets and correlate
them in some useful way.

.. note:: This page was last updated on |today| and all information
about the availability of datasets is correct as of this date. This
page will be updated in due time as soon as more material will be
made available.

As seen in :numref:`domains-diagram`, data originate from different
domains (Mobility, Tourism, and so on); they are gathered from sensors
Expand All @@ -31,34 +26,28 @@ endpoint is given along with other information in the description of
each dataset, see the lists of datasets in the remainder of this
section.

.. note:: There are currently two versions of the API, :strong:`v1`
and :strong:`v2`; :strong:`v1` is available for :strong:`both`
Tourism and Mobility domains, but deprecated within the Mobility
domain, for which the new :strong:`v2` is available and
recommended, see :ref:`the description of the new API <ninja api>`.

.. _data-providers:

Data Providers
--------------

A :strong:`Data Provider` is any entity that shares their Open Data
with the |odh| project, allowing their free reuse (ideally under a
with the Open Data Hub project, allowing their free reuse (ideally under a
free licence like |cc0| or |bysa|) from any third-party that relies on
the |odh| to build their application. These entities can be private
the Open Data Hub to build their application. These entities can be private
companies or enterprises, public bodies, and even private citizen, if
they have interesting data about South Tyrol to share.

The Open Data exposed by the |odh| originate from data and datasets
The Open Data exposed by the Open Data Hub originate from data and datasets
owned by different actors (called :strong:`Data Providers`) which are
at this time mostly local public bodies. Since there is no direct
1-to-1 correspondence between Data Providers and datasets, we
currently offer a list of data providers whose data can be pulled
from |odh|\. Indeed, an |odh| dataset can be composed of data deriving
from different providers, while a provider can submit to |odh|
from Open Data Hub\. Indeed, an Open Data Hub dataset can be composed of data deriving
from different providers, while a provider can submit to Open Data Hub
multiple types of data that will belong to more than one dataset.

The |odh|\'s Data Providers are:
The Open Data Hub\'s Data Providers are:

* IDM Südtirol/Alto Adige
* SIAG, Südtirol Informatica AG - Informatica Alto Adige
Expand All @@ -79,56 +68,110 @@ The |odh|\'s Data Providers are:

.. topic:: A note about datasets.

The |odh| contains many datasets: a few have been provided for
The Open Data Hub contains many datasets: a few have been provided for
testing purposes, other are meant for internal use only, and other
contain only a part of their data that is available as Open Data.

While the goal of the |odh| project is to expose :strong:`only Open
Data` and the |odh| team members always suggest to use |CC0| to
While the goal of the Open Data Hub project is to expose :strong:`only Open
Data` and the Open Data Hub team members always suggest to use |CC0| to
third-parties releasing datasets, it is not yet possible for the
|odh| team to guarantee the availability as open data of all the
Open Data Hub team to guarantee the availability as open data of all the
data in the datasets, because the data licensing and its
distribution rights are decided by the copyright holder of each
dataset.

Since some of the datasets may contain data that can not be
distributed by the |odh| team under an open licence like, e.g.,
distributed by the Open Data Hub team under an open licence like, e.g.,
|cc0| or |bysa|, a user will be able to retrieve from each dataset
only those data that are distributed as :strong:`Open Data`.

At the date of writing, datasets in the :ref:`Mobility
<mobility-datasets>` and :ref:`Tourism <tourism-datasets>` are available.
<mobility-datasets>` and :ref:`Tourism <tourism-datasets>` domains are
available.

Accessing data in the |odh|
Accessing data in the Open Data Hub
---------------------------

There are different modalities to access data that are provided by the
|odh|, that are listed here. Currently, data from the
Open Data Hub, that are listed here. Currently, data from the
:strong:`Mobility` and :strong:`Tourism` domains can be accessed, both
from the command line and using a browser. Various dedicated tutorials
are available in the :ref:`howto-list` section; while in section
from the command line and using a browser. Non-interactive access
using APIs is also available. Various dedicated tutorials are
available in the :ref:`howto-list` section; while in section
:ref:`getting-involved` you can find additional ways to interact with
the data and the |odh| team.
the data and the Open Data Hub team. The remainder of this section describes
all the possibilities to access the Open Data Hub's datasets and their
content.

.. _ninja api:

API
~~~

Programmatic and non-interactive access to the Open Data Hub's dataset
is possible using the APIs made available by the Open Data Hub Team.

The APIs are composed of a few generic methods, that can be combined
with many parameters to retrieve only the relevant data and then
post-processed in the preferred way.

The following table summarises how the two versions of the API can be
used within the Open Data Hub's domains.

=== ========= =============
API Tourism Mobility
=== ========= =============
v1 OK Deprecated
v2 -- Recommended
=== ========= =============


There are currently two versions of the API, v1 and v2, with the
former now :strong:`deprecated` for the Mobility domain and marked as
such |deprecated| throughout the Open Data Hub documentation. New users are
recommended to use the new API v2, while users of the API v1 are
encouraged to plan a migration to the new API.

The new API v2 has a different approach compared to the previous
version, and therefore is not compatible with the API v1, the main
difference being that all data stored in the Open Data Hub can now be
retrieved `from a single endpoint`, while with API v1 there was an
endpoint for each dataset.

This change in approach requires also a breaking change for the users
of API v1. The initial step, indeed, will not be to open the URL of
the dataset and start exploring, but to retrieve the
:literal:`stationType`\s and then retrieve additional data about each
station. A :literal:`stationType` is the main object of a datasets,
about which all the information in a dataset relate to; a dataset
includes at least one :literal:`stationType`. A new, dedicated howto
describing in detail the new API v2 and a few basic examples is
:ref:`already available <get-started-mobility>` in the dedicated section
of this documentation.

.. note:: It is important to remark that the API v2 is :strong:`only
available` for datasets in the :strong:`Mobility` Domain.


Browser access
~~~~~~~~~~~~~~

Accessing data in the |odh| by using a browser is useful on different
Accessing data in the Open Data Hub by using a browser is useful on different
levels: for the casual user, who can have a look at the type and
quality of data provided; for a developer, that can use the
:term:`REST API` implemented by the |odh| or even check if the results
:term:`REST API` implemented by the Open Data Hub or even check if the results
of his app are coherent with those retrieved with the API; for
everyone in order to get acquainted with the various methods to
retrieve data.

More in detail, these are the possibilities to interact with |odh|\'s
More in detail, these are the possibilities to interact with Open Data Hub's
data by using a browser:

#. Go to the :ref:`applist` section of the documentation, particularly
sub-sections :ref:`production-stage-apps` and
:ref:`beta-stage-apps`, and choose one of the web sites and portals
that are listed there. Each of them uses the data gathered from one
or more |ODH|\'s datasets to display a number of useful
or more OPEN DATA HUB's datasets to display a number of useful
information. You can then see how data are exposed and browse them.

#. In the same :ref:`applist` section, you can also check the list of
Expand All @@ -154,11 +197,12 @@ data by using a browser:
:ref:`tourism-data-howto` that will guide you in the first steps.

#. Access the :strong:`Swagger interface` of the datasets in the
Mobility domain. Check the link for each of them in section
:ref:`mobility-datasets`. Like in the case of the tourism' Swagger
interface, you can learn REST API call for that domain and fetch
data for your application. There is a dedicated howto to learn more
how to interact with this interface: ref:`mobility-data-howto`
Mobility domain, located at
https://mobility.api.opendatahub.bz.it/v2/swagger-ui.html. Like in
the case of the tourism' Swagger interface, you can learn REST API
call for that domain and fetch data for your application. There is
a dedicated howto to learn more how to interact with this
interface: ref:`get-started-mobility`

#. Open the :strong:`Analytics for Mobility` web page, at
https://analytics.mobility.bz.it/. This portal uses data in the
Expand Down Expand Up @@ -188,7 +232,7 @@ used with a variety of options and can save the contents it downloads,
which can them be send to other applications and manipulated.

The number of options required by :program:`curl` to retrieve data
from |odh|\'s dataset is limited, usually they are not more than 3 or
from Open Data Hub's dataset is limited, usually they are not more than 3 or
4, but their syntax and content might become long and not easily
readable by a human, due to the number of :ref:`filters
<common-filters>` available. For example, to retrieve the list of all
Expand All @@ -211,12 +255,12 @@ Authentication
~~~~~~~~~~~~~~

The authentication layer is currently intended for :strong:`internal
use only`. All data in the dataset that you can receive from the |odh|
use only`. All data in the dataset that you can receive from the Open Data Hub
are free to use and do not require any type of authentication.


The authentication layer can be of interest for developers who want to
collaborate in the development of |odh|\; Details on the implementation
collaborate in the development of Open Data Hub; Details on the implementation
are available in section :ref:`authentication`.

.. _mobility-datasets:
Expand All @@ -229,37 +273,8 @@ Datasets in the Mobility Domain
:local:

This section contains information about the datasets and how to access
them using the API that the |odh| team developed and made available.
them using the API that the Open Data Hub team developed and made available.

.. _ninja api:

.. topic:: API v1 vs API v2.

There are currently two versions of the API, v1 and v2, with the
former now :strong:`deprecated` and marked as such |deprecated|
throughout the |odh| documentation. New users are strongly
suggested to use the new API v2, while users of the API v1 are
encouraged to plan a migration to the new API.

The new API v2 has a different approach compared to the previous
version, and therefore is not compatible with the API v1, the main
difference being that all data stored in the |odh| can now be
retrieved `from a single endpoint`, while with API v1 there was an
endpoint for each dataset.

This change in approach requires also a breaking change for the
users of API v1. The initial step, indeed, will not be to open the
URL of the dataset and start exploring, but to retrieve the
:literal:`stationType`\s and then retrieve additional data about
each station. A :literal:`stationType` is the main object of a
datasets, about which all the information in a dataset relate to; a
dataset includes at least one :literal:`stationType`. A new,
dedicated howto will describe with examples the new workflow for
the use of the API v2.

It is important to note that the API v2 is :strong:`only available`
for datasets in the :strong:`Mobility` Domain.

The description of each dataset includes the following information:

* The output format of the API call
Expand Down Expand Up @@ -382,9 +397,10 @@ for each of the above-listed dataset:
* The versions of the API that can be used to access the dataset.
* The swagger URL of the APIs.

.. note:: There is one :literal:`StationType`, namely :strong:`MobileStation` which
is a mobile probe no longer active. It will always return an empty
set of values, because historical data are not available in the |odh|\.
.. note:: There is one :literal:`StationType`, namely
:strong:`MobileStation` which is a mobile probe no longer
active. It will always return an empty set of values, because
historical data are not available in the Open Data Hub.

.. _accommodation-dataset:

Expand Down

0 comments on commit f5313dc

Please sign in to comment.