Building MobilityDB and MEOS

Interplay between MobilityDB and MEOS

MobilityDB is a database management system for moving object geospatial trajectories, such as GPS traces. It adds support for temporal and spatio-temporal objects to the PostgreSQL database and its spatial extension PostGIS.

MEOS (Mobility Engine, Open Source) is a C library which enables the manipulation of temporal and spatio-temporal data. It is the core component of MobilityDB.

The relationship between MobilityDB and MEOS is represented in the following figure.

MobilityDB extends PostgreSQL and PostGIS types with temporal features thus enabling to represent phenomena that evolve on time. For this, MobilityDB defines the following temporal types: tbool, tint, tfloat, and ttext, extending the corresponding PostgreSQL base types bool, int, float, and text, as well as tgeompoint and tgeogpoint, extending the corresponding PostGIS base types geometry and geography. Each of these types have a rich set of associated operations. On top of that, MobilityDB defines GiST and SP-GiST indexes for temporal types as well the statistics and histogram mechanisms enabling the query optimizer to assess the cost of SQL queries to choose between alternative query plans.

On the other hand, many applications require to manipulate temporal features in programs, that is, independently of any database. A typical example is stream processing, in which the more recent observations of mobility data are processed in main memory. When these observations are no longer needed, they can be sent to a MobilityDB database to store them and free the buffers to continue processing the latest observations. This is role of the MEOS library. As can be seen in the figure above, MEOS contains the type definitions and corresponding operations for MobilityDB temporal types, as well as the corresponding base types from PostgreSQL and PostGIS. One way to see MEOS, is that it contains the components of MobilityDB that are independent of PostgreSQL.

An important aspect to be emphasized is that MobilityDB and MEOS share the same codebase, so that the same operation performed in a C program running in main memory and in a SQL query in MobilityDB on top of PostgreSQL will return exactly the same result. Furthermore, bindings of the MEOS library for various programming languages are being developed: PyMEOS for Python, as well as similar bindings for Java and C#.

Software Architecture

We explain next the architecture enabling the generation of both MobilityDB and MEOS libraries using the same codebase.

The figure above shows the three main code directories of the MobilityDB repository.

• meos contains code specific to MEOS
• postgis contains code borrowed from PostGIS
• mobilitydb contains code specific to MobilityDB

The figure also shows how these directories are used for building the MobilityDB and the MEOS libraries.

• MEOS includes code from the meos and postgis directories
• MobilityDB includes code from the meos, postgis, and mobilitydb directories

As can be seen in the figure, some code in the meos directory is not used for MobilityDB. This includes

• the postgres directory containing code borrowed from PostgreSQL. The directory contains code for manipulating base data types (boolean, integer, float, and text), time data types (timestamp, timestamptz, date, time, and interval), as well as hash and sort functions.
• function wrappers that hide the internal implementation details. Such code can be found in files whose suffix is _meos.c or functions that are conditionally compiled using #if MEOS ... #endif.

Similarly, some code in the postgis directory is not used for MEOS. This includes

• the libpgcommon directory containing code specific to PostgreSQL.

The advantage of this architecture is to enable the use of a single basecode for both libraries MobilityDB and MEOS. The MobilityDB library is generated by default. To generate the MEOS library it suffices to pass the additional parameter -DMEOS=on to cmake.

Detailed Directory Structure

The directory meos has the following subdirectories

• examples containing the MEOS tutorial programs.

• include containing the MEOS include files

• postgres containing the code borrowed from PostgreSQL

• src containing the MEOS source code, which it is divided into the following subdirectories

  • general containing generic temporal functionality. This includes functionality for the temporal types tbool, tint, tfloat, and ttext, the bounding box type tbox, the span types intspan, floatspan, and period, as well as the lifting infrastructure that temporally extend traditional functions such as arithmetic operators for numbers or distance operators for geometries.
  • npoint containing specific functionality for temporal network points, that is, the type tnpoint
  • point containing specific functionality for temporal points, that is, the temporal types tgeompoint and tgeogpoint and the bounding box type stbox.

The directory mobilitydb has the following subdirectories

• pg_include containing the MobilityDB include files
• sql containing the PostgreSQL definitions of MobilityDB types and functions
• src containing the MobilityDB source code, which it is divided into the directories general, npoint and point as for the meos directory above
• test containing the regression tests

Separating Functionality between the Directories

As stated above, the directory meos contains generic functionality pertaining to both MEOS and MobilityDB, while the directory mobilitydb has specific MobilityDB functionality. For example, the lifting infrastructure mentioned above is located in the meos directory. On the other hand, the temporal indexing and temporal aggregation functionality is located in the mobilitydb directory, since it builds upon built-in indexing and aggregation infrastructure provided by PostgreSQL.

However, most of the functionality of MEOS and MobilityDB is split into both directories meos and mobilitydb. The general approach for this is that the functionality is entirely provided in the meos directory using standard C data types as well as the basic C data types borrowed from PostgreSQL such as timestamptz and interval. This allows us to provide in the mobilitydb directory thin PostgreSQL wrappers that input the parameters of the function, call the corresponding MEOS function, and return the output of the function to PostgreSQL.

This is explained next with the example of the function that outputs temporal types in Well-Known Text (WKT) format, a temporal extension of the Open Geospatial Consortium's WKT standard. The SQL function definition found in the mobilitydb/sql/point directory is given next.

CREATE FUNCTION asText(tgeompoint)
  RETURNS text
  AS 'MODULE_PATHNAME', 'Tpoint_as_text'
  LANGUAGE C IMMUTABLE STRICT PARALLEL SAFE;
CREATE FUNCTION asText(tgeogpoint)
  RETURNS text
  AS 'MODULE_PATHNAME', 'Tpoint_as_text'
  LANGUAGE C IMMUTABLE STRICT PARALLEL SAFE;

The MobilityDB C function found in the mobilitydb/src/point directory is as follows.

PGDLLEXPORT Datum
Tpoint_as_text(PG_FUNCTION_ARGS)
{
  /* Input parameters */
  Temporal *temp = PG_GETARG_TEMPORAL_P(0);
  int dbl_dig_for_wkt = OUT_DEFAULT_DECIMAL_DIGITS;
  if (PG_NARGS() > 1 && ! PG_ARGISNULL(1))
     dbl_dig_for_wkt = PG_GETARG_INT32(1);
  /* Call the MEOS function */
  char *str = tpoint_as_text(temp, dbl_dig_for_wkt);
  /* Construct the result */
  text *result = cstring2text(str);
  /* Clean-up and return */
  pfree(str);
  PG_FREE_IF_COPY(temp, 0);
  PG_RETURN_TEXT_P(result);
}

As can be seen, the function starts by getting the input parameters, that is, the temporal geometric or geographic point and the optional parameter specifying the number of digits used for printing the floating point values of the coordinates. Then, a call of the MEOS function tpoint_as_text is performed to obtain the result as a C string (the function is shown below). This string is converted into a text value and then returned to PostgreSQL. Notice also that the convention used is that the names of the MobilityDB functions start with an uppercase character, while the corresponding MEOS functions start with a lowercase character.

Finally, the MEOS function found in the meos/src/point directory is shown next.

char *
tpoint_as_text(const Temporal *temp, int maxdd)
{
  /* Ensure validity of the arguments */
  assert(temp != NULL);
  ensure_non_negative(maxdd);

  char *result;
  assert(temptype_subtype(temp->subtype));
  if (temp->subtype == TINSTANT)
    result = tinstant_to_string((TInstant *) temp, maxdd, &wkt_out);
  else if (temp->subtype == TSEQUENCE)
    result = tsequence_to_string((TSequence *) temp, maxdd, false, &wkt_out);
  else /* temp->subtype == TSEQUENCESET */
    result = tsequenceset_to_string((TSequenceSet *) temp, maxdd, &wkt_out);
  return result;
}

As explained above, one advantage of this architecture is that it enables to have the same functionality in MEOS and in MobilityDB. It is worth noting that this requires that the verification of the validity of the input parameters of a function should be done in the MEOS function and not in the MobilityDB function. For example, this means that the tests verifying the validity of the arguments should be done in the MEOS function tpoint_as_text and not in the MobilityDB function Tpoint_as_text.