Home

Introduction

OBIS Schema Extractor (v2) is a Java-based web application for data schema extraction from SPARQL endpoints.

OBIS Schema Extractor artifact is a packaged executable JAR file and needs Java 17 to run.

For running OBIS Schema Extractor locally:

1. install Java
2. copy build/schema-extractor-exec.jar, together with the entire build folder contents, to any folder
3. run executable JAR file java -jar schema-extractor-exec.jar
4. navigate to http://server:port/swagger-ui.html (default URL - http://localhost:8080/swagger-ui.html)
5. configure the parameters, as described on the browser page, or use create the configuration file (a sample configuration file is available in the build folder)

Example Execution

We provide a resulting Schema file, a Configuration file and the Execution log for extracting the schema from Nobel Prizes SPARQL API endpoint (overall schema extraction time is below 3 minutes).

Further possible use of the schema is to upload it into Data Shape Server that can be set up either independently, or as a part of ViziQuer Tools container system. The schema upload is done by DSS Schema import utility.

Extractor Parameters

The parameters available in Swagger-based extractor execution are most conveniently accessible from the browser page for running the extractor.

The specifications of all parameters, including the ones currently hidden (@ApiParam(hidden = true)) from the browser UI (but available for use from the configuration file), are available in the parameter description code.

An example configuration file is provided along with the extractor, as well (when extracting all classes and/or properties from an endpoint, set the includedClasses and/or includedProperties parameters to []).

The Extractor extracts the schema from a SPARQL Endpoint by means of (possibly long) series of SPARQL SELECT queries issued against it. The query templates are provided in a separate Source File.

The following table describes some important parameters

Parameter Name	Datatype / Enum	Description
endpointUrl	String	SPARQL Endpoint URL, for example, http://localhost:8890/sparql
graphName	String	Named Graph (optional, recommended). If no graph name is provided, the search will involve all graphs from the endpoint
calculateSubClassRelations	Boolean	Calculate subclass relation (strongly recommended)
calculateMultipleInheritanceSuperclasses	Boolean	Include multiple inheritance check, strongly recommended. If set to false, no more than a single direct super-class for a class shall be found. Use this option only for very large endpoints, if the processing of subclass relations is stuck in the extractor for a longer time.
minimalAnalyzedClassSize	Integer	Minimal analyzed class size. The classes below the specified size shall be included in the schema, still they shall not be considered as possible super-classes of other classes and their links to properties shall not be analyzed individually. If 1, all classes should be analyzed. Values above 1, e.g. 10 or 100, may be essential for endpoints with more than several thousand classes.
requireClasses	Boolean	At least one class is required for schema extraction. If no classes are found, extractor work is aborted (Swagger default = true)
calculatePropertyPropertyRelations	Boolean	Calculate property-property adjacency: following properties, same subject, same object. Important for visual and textual query auto-completion use case; is used in schema visualization only if property sources and targets are considered as potential nodes (work in progress). Can be time-consuming for larger schemas (Swagger default = true)
propertyPropertyLinkCheckBackupMode	none, limits, limitsPlus, limitsPlusSimple, details	Set backup validation queries to calculate property-property relations in the case of a failing property co-occurrence query for a given base property. details, limitsPlus and limitsPlusSimple guarantee trying a single-line pattern of two property co-occurrence, if larger patterns fail. details gives exact statistics for two property co-occurrence, whenever possible. limitsPlus attempts at least a limit-based estimate, limitsPlusSimple can stay with yes/no answer for co-occurrence for performance reasons (with some count information imputed at later schema import steps). none and limits are faster, but can result in partially incomplete information. (Swagger default = limitsPlusSimple)
calculateSourceAndTargetPairs	Boolean	Calculate pairs of source and target classes for properties. Creates finer-grained schemas and is used as a source of statistics in visual schema diagrams (Swagger default = true)
calculateDomainsAndRanges	Boolean	Calculate domain and range classes for properties (Swagger default = true)
calculateImportanceIndexes	basic, unionBased, classCoverage, no	Calculate ascription points / principal classes for properties. Strongly recommended if schema diagrams are envisaged. Use classCoverage if all class-to-property connections should be marked for the class itself or for some superclass/subclass (Swagger default = basic)
calculateClosedClassSets	Boolean	Check property source and target class set closure; essential for future SHACL export (Swagger default = true)
calculateCardinalities	none, propertyLevelOnly, propertyLevelAndClassContext	Calculate min and max cardinalities for properties (Swagger default = propertyLevelAndClassContext)
calculateDataTypes	none, propertyLevelOnly, propertyLevelAndClassContext	Calculate data types for attributes (Swagger default = propertyLevelAndClassContext)
sampleLimitForDataTypeCalculation	Long	Instance sample size for datatype calculation
calculateInstanceNamespaces	no, detailed, overview	Calculate instance namespace URIs (Swagger default = no)
sampleLimitForInstanceNamespacesCalculation	Long	Limit of instances to use in namespace calculation. Swagger description says no value or 0 means all data will be used (Swagger default = 1000)
addedLabels	List	Properties for class and property labels. rdfs:label and skos:prefLabel are assumed by default. For labels in specific languages use @{en;de} notation after the label property IRI or short form. To exclude a default labeling property, use property@{-} notation
addIntersectionClasses	yes, no	Add intersection classes to the result schema (Swagger default = yes)
excludedNamespaces	List	List of excluded namespaces, e.g. http://www.openlinksw.com/schemas/virtrdf#
includedClassesFile	CSV file	Valid CSV file with the list of included classes. If not specified, all classes will be analyzed
includedPropertiesFile	CSV file	Valid CSV file with the list of included properties. If not specified, all properties will be analyzed
enableLogging	Boolean	Enable SPARQL query logging to file (Swagger/request default = true)
saveThisConfig	Boolean	Save this configuration to file (Swagger/request default = true)