Various updates to the README

README.md

@@ -8,7 +8,9 @@ The black box has two major components:
 * `data logger`: A set of interfaces for reading data from different data sources and logging those data
 * `query_interface`: An interface for black box data retrieval
 
-Both the logger and the query interface are exposed in both Python and C++; the Python versions are generally more up-to-date than the C++ versions. Note: We develop and test under Python 3; there are no plans for backwards compatibility with Python 2.
+Both the logger and the query interface are exposed in both Python and C++; the Python versions are however newer and generally more up-to-date than the C++ versions (the C++ version of the query interface is in particular outdated currently). Note: We develop and test under Python 3; there are no plans for backwards compatibility with Python 2.
+
+A collection of tools for working with data from the black box - aimed at offline data analysis - can be found in our separate [`black-box-tools`](https://github.com/ropod-project/black-box-tools) repository.
 
 ## Black Box Design Principles
 
@@ -25,11 +27,15 @@ which have however evolved in a few respects since that paper was published.
 
 ### Data Format
 
+#### General Principles
+
 The black box stores data in a standard format, such that
 1. **logged data are always associated with a timestamp** and
-2. **all data types are logged in a nested manner that reflects the underlying data type** (in the original version of the black box introduced in the above paper, data were logged at a single level of abstraction, i.e. in an unrolled manner, but we've abandoned that design since it required manual conversion of data to the standard format, which was inconvenient particularly in the case of logging ROS messages).
+2. **all data types are logged in a nested manner that reflects the underlying data type** (in the original version of the black box introduced in the above paper, data were logged at a single level of abstraction, i.e. in an unrolled manner, but we've abandoned that design since it required manual conversion of data to the standard format, which was quite inconvenient and required a fair bit of manual processing - particularly in the case of logging ROS messages).
+
+#### Example
 
-This is best illustrated by an example. If we want to log a ROS topic of type [geometry_msgs/Pose](http://docs.ros.org/lunar/api/geometry_msgs/html/msg/Pose.html), which is defined as
+The standard format is best illustrated by an example. If we want to log a ROS topic of type [geometry_msgs/Pose](http://docs.ros.org/lunar/api/geometry_msgs/html/msg/Pose.html), which is defined as
 
 ```
 geometry_msgs/Point position
@@ -71,11 +77,38 @@ ros_cmd_vel/orientation/z
 ros_cmd_vel/orientation/w
 ```
 
-The full names of the variables are the ones we use for querying data.
+The full variable names the ones we use for querying data.
+
+#### Data Types With Lists
+
+In the case of data types that contain lists, the full variable names unroll the items represented by the data.
+
+For instance, let's say that we have a robot with four ultrasonic sensor and we publish their distances as a [std_msgs/Float32MultiArray](http://docs.ros.org/lunar/api/std_msgs/html/msg/Float32MultiArray.html) message on a `distances` topic, which will be stored in the format
+
+```
+double timestamp
+double data
+    [
+        distance_1,
+        distance_2,
+        distance_3,
+        distance_4
+    ]
+```
+
+(in this example, we ignore the `layout` field of the message for convenience), the full variable names will be given as follows:
+
+```
+ros_distances/timestamp
+ros_distances/data/0
+ros_distances/data/1
+ros_distances/data/2
+ros_distances/data/3
+```
 
 ### Data Filtering and Configurability
 
-In order to deal with data that come from different sources (e.g. ROS, EtherCAT, Zyre) and have different formats, each data source is associated with its own data filter that converts the data into the previously described standard format and allows data preprocessing/filtering. These filters are configured through a configuration file, an example of which can be found in [config/data_sources.yaml](config/data_sources.yaml).
+In order to deal with data that come from different sources (e.g. ROS, EtherCAT, Zyre) and have different formats, each data source is associated with its own data filter that - whenever necessary - converts the data into the previously described standard format and allows data preprocessing/filtering. These filters are configured through a configuration file, an example of which can be found in [config/data_sources.yaml](config/data_sources.yaml).
 
 ### Query Interface
 
@@ -105,12 +138,6 @@ The logger additionally depends on:
 * `rospy`
 * [`rospy_message_converter`](https://github.com/ropod-project/rospy_message_converter) (a slightly customised version so that it works with Python 3)
 
-## Class diagrams (C++)
-
-A UML-like diagram of the major data logger components is given below; the diagram is not exactly UML because the boxes contain more casual versions of the actual class names.
-
-![ROPOD data logger - UML diagram](docs/images/ropod_datalogger.png)
-
 ## Usage (Python)
 
 The black box functionalities are exposed through a `black_box` package that contains two subpackages - `datalogger` and `query interface`. To install the package, run the command
@@ -192,3 +219,9 @@ To launch the component monitors, run the command
 ```
 
 inside the previously created `build` directory.
+
+## Class diagrams (C++)
+
+A UML-like diagram of the major data logger components is given below; the diagram is not exactly UML because the boxes contain more casual versions of the actual class names. Not all the components in the diagram are actively used and developed (e.g. the EtherCAT and ZMQ readers are not actively supported).
+
+![ROPOD data logger - UML diagram](docs/images/ropod_datalogger.png)