GeoRDFBench Framework: Geospatial Semantic Benchmarking Simplified

GeoRDFBench can be used by researchers and practitioners interested in Semantic Benchmarking with SPARQL language. The framework however provides unique features for the geospatial aspects of RDF stores that support the GeoSPARQL standard. Documentation for the project is maintained at our GeoRDFBench web site.

For a quick acquaintance with the platform, please run the dockerized example at dockerized example which exhibits running a GeoSPARQL experiment with the RDF4J RDF store against the Scalability 10K dataset of the Geographica 2 benchmark.

The GeoRDFBench Framework Samples is a maven project using GeoRDFBench's runtime as dependence and showcases the use of the JSON Generator's API for constructing the workload specification for the LUBM(1, 0) benchmark.

The GeoRDFBench web site also provides the Multiple stores against LUBM(1, 0) Workload dockerized example showcasing the use of GeoRDFBench to run experiments with RDF4J, GraphDB and JenaGeoSPARQL against LUBM(1, 0) workload.

News

The new v2.2.0 JSON Specification Library Endpoint module. Its main features are:

• It's a JavaScript module running on Node.js JS runtime environment and takes advantage of the Express web framework.
• It allows the deployment of the JSON Specification Library to a remote server, different from the experiments server, therefore promoting a more distributed approach to the GeoRDFBench Framework's operation.
• The user can choose the JSON Specification Library location which holds the json_defs structure, the endpoint's listening port and configure which CRUD operations are enabled for each specification category. The implemented operations are: (C)reate, (R)ead and (D)elete. Since the (U)pdate operation can impair the desired immutability of reference specifications, a more safe (Copy/Clone) operation is offered by the new JSON Specification UI Endpoint to subtitute that.
• The experiment JSON specification arguments can now also be REST API URLs from the endpoint instead of file paths. An example can be found in the RunRDF4JExperimentRemoteSpecsTest test class of the RDF4JSUT module.
• The user can find details for the API in the Swagger (OpenAPI Specification) documentation
• The endpoint implements a REST API which currently allows GET operations, such as:
  • Get a list of available Specification categories
  • Get the Endpoint's configuration
  • Get a list of available Datasets
  • Get the JSON spec for dataset LUBM_1_0DSoriginal
  • Get a list of available Querysets
  • Get the JSON spec for queryset scalabilityFuncQSoriginal
  • Get a list of available Execution Specs
  • Get the JSON spec for execution specification scalabilityESoriginal
  • ...

The new v2.4.0 JSON Specification UI Endpoint module. Its main features are:

• It's a JavaScript module running on Node.js JS runtime environment and takes advantage of the Express web framework.
• It provides a user interface through which the user can manage a JSON Specification Library (through the Library Endpoint).
• It uses Pug (formerly known as Jade) high performance template engine for Node.js to server dynamic forms to the user.
• It can be hosted to the same or different server hosting the JSON Specification Library Endpoint. It can be hosted to the same or different server from the experiments server, therefore promoting a more distributed approach to the GeoRDFBench Framework's operation.
• The user can configure the JSON Specification Library Endpoint which is managed.
• The user can choose the endpoint's listening port.
• The endpoint offers the following operations, such as:
  • Get the status of the JSON Specification Library and UI Endpoints.
  • Get the Swagger (OpenAPI Specification) documentation of the JSON Specification Library Endpoint.
  • Get the list of available Specification categories
  • Get the List, Create new, Copy from existing, Delete of Report Specs, Hosts, Execution Specs, Datasets and Querysets.
  • ...(Workloads to be added shortly!)

Short Term Next Steps

• New capabilities:

  • add multi-user capabilities (user authentication and authorization) to the JSON Specification UI Endpoint which will allow each user their own copy of the JSON Specification Library,

• New RDF Modules for SPARQL/GeoSPARQL:

  • Neo4J with neosemantics (n10s) plugin

Medium Term Next Steps

• New capabilities:

  • inference support for RDF modules
  • support for the Hadoop file system,
  • a fourth Spark-based framework API, so that Spark-based distributed GeoSPARQL solutions, such as Strabo 2 can be tested, and

• New RDF Modules for SPARQL/GeoSPARQL:

  • Anzo Graph by Cambridge Semantics (now Altair),

• New Semantic Languages:

  • Graph Query Language (GQL) for running benchmarks on native LPGs

Sustainability plan and limitations

A full list of issues can be found here. Their status can be followed up in a GitHub Project.

GeoRDFBench's runtime was created to support:

• RDF store integration: fast integration of new or updated RDF stores by incorporating the common logic of working with RDF repositories through any of the well-known RDF Java Frameworks: OpenRDF Sesame, Eclipse RDF4J, Apache Jena.
• Benchmark specification Ser/Der: easy construction of GeoSPARQL/SPARQL benchmark specifications and even easier serialization/deserialization of these JSON specification files, increasing the reusability of these specifications and equally important improving the repeatability of experiments and confidence on experiment results.

Current limitations include the non automatic serialization/deserialization of the benchmark environment (host, os, RDF store components) specifications and the lack of support for distributed RDF stores.