Merge pull request #42 from transportenergy/data-terminology

Add Glossary page to documentation

.github/workflows/pytest.yml

@@ -37,7 +37,7 @@ jobs:
         curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip
         popd
         unzip $HOME/.cache/rclone-current-linux-amd64.zip
-        echo "::add-path::$(ls -d rclone-v*)"
+        ls -d rclone-v* > $GITHUB_PATH
         echo "$service_account_json" >ci/item-historical-database.json
 
     - name: Upgrade pip, wheel

doc/glossary.rst

@@ -0,0 +1,56 @@
+Glossary
+********
+
+Attribute
+  Any information associated with an observation or group of observations. *Example:* the attribute ‘status’ might have a value of “Provisional” or “Final”, related to a statistical agency's process of publishing preliminary and then final values.
+
+Assumptions
+  Quantities used in data processing.
+
+Concept
+  Both background concepts and specific, systematic, defined meanings. *Example:* ‘energy demand’ and ‘fuel use’.
+
+Conversion factor
+  Used to convert between alternate measures of the same concept.
+  *Example:* energy content of fuel is used to convert ‘fuel use’ from volume to energy units.
+
+Data
+  Collective noun for of observations of specific measures for general  concepts, organized in one or more dimensions, with attributes.
+
+  Modelled data
+    Data that are produced from an existing model or calculations.
+
+Data processing
+  Algorithms that combine raw data and assumptions to produce a dataset with greater coverage or quality; or to derive certain measures from raw data.
+
+Data set
+  A collection of individual observations.
+
+Data source
+  A person, agency, or web service that provides data.
+
+  National source
+    National organizations such as national statistical agencies, ministries of transport or energy, etc. who directly measure quantities or collect measurements from subsidiary organizations, and provide these as data.
+
+  Aggregator
+    An agency or person who collects and assembles data into larger data sets.
+    These may include data from multiple upstream sources (such as national sources), with or without any cleaning, adjustment, or harmonization.
+
+Dimension
+  A named list of labels or values used to organize multiple observations in a set of data. *Example:* ‘year’ (a sequential list of annual periods), ‘country’ (names or codes for countries).
+
+Measure
+  An operational definition, including units, of a systematic concept.
+  Multiple measures may exist for the same concept.
+  *Example:* ‘fuel use’ may be measured in terms of the volume of fuel (litres) or its energy content (joule).
+
+Observation
+  A single value for a measure.
+
+Parameter
+  Used to derive one measure from another in a data processing calculation.
+  *Example:* ‘occupancy of passenger vehicles’ (persons per vehicle) is used to calculate ‘passenger travel’ (in kilometres) from ‘vehicle travel’ (kilometers).
+
+Upstream
+  Data (or software) used as an input.
+  Sometimes the term “raw data” is incorrectly used for “upstream data”.

doc/historical.rst

@@ -1,7 +1,9 @@
 Historical & statistical data (:mod:`item.historical`)
 ******************************************************
 
-This module contains the code that implements the `iTEM Open Data project <https://transportenergy.org/data/historical/>`_, the broader aims of which are described on the main iTEM website.
+This module contains the code that implements the `iTEM Open Data project <https://transportenergy.org/open-data/>`_, the broader aims of which are described on the main iTEM website.
+
+See also the :doc:`glossary`, which gives precise terminology used on this page.
 
 .. contents::
    :local:

doc/index.rst

@@ -3,14 +3,14 @@ International Transport Energy Modeling (iTEM) toolkit
 
 `iTEM`_ maintains two databases:
 
-1. A **model database** of transport energy projections assembled as part of
-   the `iTEM workshops`_, of which there have been four so far, with a fifth
-   planned for early 2020.
+1. A **historical database** to form a common, public, “best available” baseline for model calibration and projections.
+   The historical database is under continuous development.
 
-2. A **historical database** to form a common, public, “best available”
-   baseline for model calibration and projections.
+2. A **model database** of transport energy projections assembled as part of the iTEM model intercomparison projects (MIPs) linked to `iTEM workshops`_.
+   To meet the intellectual property concerns of workshop participants, the model database is currently not public, and only available on request; however, the tools used to prepare it are public.
+   These tools are developed periodically, during sequential MIPs.
 
-This documentation, built automatically from the `transportenergy/database GitHub repository <https://github.com/transportenergy/database>`_, describes the Python and R toolkit for maintaining these databases.
+This documentation, built automatically from the `transportenergy/database GitHub repository <https://github.com/transportenergy/database>`_, describes the Python and R code for maintaining these databases.
 
 .. _iTEM: http://transportenergy.org
 .. _iTEM workshops: http://transportenergy.org/workshops
@@ -28,6 +28,7 @@ This documentation, built automatically from the `transportenergy/database GitHu
    remote
    cli
    metadata
+   glossary
    whatsnew
    developing
 

doc/whatsnew.rst

@@ -11,6 +11,7 @@ What's new?
 Next release
 ============
 
+- Add the :doc:`glossary` page.
 - Correct an error in the input data for :mod:`.T001` (:issue:`32`, :pull:`40`).