D2R Server is a tool for publishing relational databases on the Semantic Web.
- It enables RDF and HTML browsers to navigate the content of the database,
- and allows applications to query the database using the SPARQL query language.
2009-08-10: Version 0.7 released. This version provides several bugfixes, better dump performance, several new features as well as new optimizations that must be enabled using the new --fast switch.
-
2009-02-19: Version 0.6 released. This version introduces serving of vocabulary data and includes D2RQ 0.6, which provides significantly improved performance and memory usage, new features and several bugfixes.
-
2007-11-03: Version 0.4 released. This version can be run as a J2EE web application inside existing servlet containers and fixes several bugs.
-
2007-02-13: Version 0.3.2 released, featuring more configurability, easy installation as a Windows service, and many small improvements and bugfixes.
-
2007-01-22: dbpedia.org uses D2R Server to publish structured data extracted from Wikipedia.
D2R Server is a tool for publishing relational databases on the Semantic Web.
+ It enables RDF and HTML browsers to navigate the content of the database,
+ and allows applications to query the database using the SPARQL query language.
2009-08-10: Version 0.7 released. This version provides several bugfixes, better dump performance, several new features as well as new optimizations that must be enabled using the new --fast switch.
+
2009-02-19: Version 0.6 released. This version introduces serving of vocabulary data and includes D2RQ 0.6, which provides significantly improved performance and memory usage, new features and several bugfixes.
+
2007-11-03: Version 0.4 released. This version can be run as a J2EE web application inside existing servlet containers and fixes several bugs.
+
2007-02-13: Version 0.3.2 released, featuring more configurability, easy installation as a Windows service, and many small improvements and bugfixes.
+
2007-01-22: dbpedia.org uses D2R Server to publish structured data extracted from Wikipedia.
D2R Server is a tool for publishing the content of relational databases
- on the Semantic Web, a global
- information space consisting of
- linked data.
-
-
Data on the Semantic Web is modelled and represented in
- RDF.
- D2R Server uses a customizable
- D2RQ mapping
- to map database content into this format, and allows the RDF data to be
- browsed and searched – the two main access paradigms
- to the Semantic Web.
-
-
D2R Server's Linked Data interface makes RDF descriptions of individual resources available over the HTTP protocol. An RDF description can be retrieved simply by accessing the resource's URI over the Web. Using a Semantic Web browser like Tabulator (slides) or Disco, you can follow links from one resource to the next, surfing the Web of Data.
-
-
The SPARQL interface enables applications to search and query the database using the SPARQL query language over the SPARQL protocol.
-
-
A traditional HTML interface offers access to the familiar Web browsers.
-
-
-
-
Requests from the Web are rewritten into SQL queries via the mapping. This on-the-fly translation allows publishing of RDF from large live databases and eliminates the need for replicating the data into a dedicated RDF triple store.
-
-
Read more about the interfaces offered by D2R Server, including example HTTP requests and responses, in the Technical Note Publishing Databases on the Semantic Web.
-
-
Quick start
-
-
You need:
-
-
Java 1.4 or newer on the path (check with
- java -version if you're not sure),
-
A supported database. D2R Server works with Oracle,
- MySQL, PostgreSQL, Microsoft SQL Server, and any SQL-92 compatible database. Microsoft
- Access can be used with some restrictions. More information is
- available in the
- D2RQ manual.
-
a modern browser like Firefox, Opera or Safari
- for using D2R Server's AJAX SPARQL Explorer. Internet Explorer can
- only browse the HTML pages, but not use the SPARQL Explorer.
-
Optionally, a J2EE servlet container as a
- deployment target. D2R Server can be run either as a stand-alone
- web server or inside an existing servlet container.
-
-
-
What to do:
-
-
Download and extract the
- archive into a suitable location.
-
Download a JDBC driver from your database vendor.
- Place the driver's JAR file into D2R Server's /lib directory.
- A list of JDBC drivers
- from different vendors is maintained by Sun. Also take
- note of the driver class name (e.g.
- org.postgresql.Driver for PostgreSQL or
- oracle.jdbc.driver.OracleDriver for Oracle) and
- JDBC URL pattern (e.g.
- jdbc:mysql://servername/database
- for MySQL) from the driver's documentation. Drivers for MySQL and PostgreSQL are
- already included with D2R Server.
-
Generate a mapping file for your database schema.
- Change into the D2R Server directory and run:
You can browse the database content or use the SPARQL Explorer to execute
- queries and display results in a number of formats.
-
To test the data in an RDF browser, open any resource URI in
- Tabulator.
- You may have to tweak your Firefox settings first – see the Tabulator help.
-
-
-
Customizing the database mapping
-
D2R Server uses the D2RQ Mapping Language to map the content of a relational database to RDF. A D2RQ mapping specifies how resources are identified and which properties are used to describe the resources.
-
The generate-mapping script automatically generates a D2RQ mapping from the table structure of a database. The tool generates a new RDF vocabulary for each database, using table names as class names and column names as property names. Semantic Web client applications will understand more of your data if you customize the mapping and replace the auto-generated terms with terms from well-known and publicly accessible RDF vocabularies.
-
The mapping file can be edited with any text editor. Its syntax is described in the
- D2RQ Manual.
- D2R Server will automatically detect changes to the mapping file and
- reload appropriately when you hit the browser's refresh button.
-
Note: The HTML and RDF browser interfaces only work
- for URI patterns that are relative and do not contain the
- hash (#) character. For example, a URI pattern such as
- entries/@@mytable.id@@ is browsable, but
- http://example.com/entries#@@mytable.id@@ is not. The mapping
- generator only creates browsable patterns. Non-browsable patterns still
- work in the SPARQL interface and in RDF dumps.
-
-
Serving vocabulary classes and properties
-
A D2R deployment often requires the introduction of custom classes and properties.
-In the spirit of Linked Data, vocabulary data should be dereferencable by clients. D2RQ infers types of classes and properties, and allows the user to provide labels, comments
-and additional properties.
-
-
D2R Server automatically serves data for vocabularies placed under http://baseURI/vocab/resource/,
-with RDF and HTML representations located at http://baseURI/vocab/data/ and http://baseURI/vocab/page/, respectively.
-The following mapping illustrates the intended usage:
-
-
-
@prefix vocabClass: <http://localhost:2020/vocab/resource/class/> .
-@prefix vocabProperty: <http://localhost:2020/vocab/resource/property/> .
-
-map:offer a d2rq:ClassMap;
- d2rq:classDefinitionLabel "Offer"@en;
- d2rq:classDefinitionComment "This is an offer"@en;
- d2rq:class vocabClass:Offer;
- .
-
-
-
When dereferenced, http://localhost:2020/vocab/resource/class/Offer will return the specified label and comment as well as the automatically inferred type rdfs:Class. Note that the prefixes are bound to absolute URIs because relative URIs would be based under http://localhost:2020/resource/.
-
-
D2RQ offers several constructs to provide data for classes and properties. Please refer to the D2RQ Language Specification for more details.
-
-
Server configuration
-
-
The server can be configured by adding a configuration block to the mapping file (all parts are optional):
-
-
D2R Server is a tool for publishing the content of relational databases
+ on the Semantic Web, a global
+ information space consisting of
+ linked data.
+
+
Data on the Semantic Web is modelled and represented in
+ RDF.
+ D2R Server uses a customizable
+ D2RQ mapping
+ to map database content into this format, and allows the RDF data to be
+ browsed and searched – the two main access paradigms
+ to the Semantic Web.
+
+
D2R Server's Linked Data interface makes RDF descriptions of individual resources available over the HTTP protocol. An RDF description can be retrieved simply by accessing the resource's URI over the Web. Using a Semantic Web browser like Tabulator (slides) or Disco, you can follow links from one resource to the next, surfing the Web of Data.
+
+
The SPARQL interface enables applications to search and query the database using the SPARQL query language over the SPARQL protocol.
+
+
A traditional HTML interface offers access to the familiar Web browsers.
+
+
+
+
Requests from the Web are rewritten into SQL queries via the mapping. This on-the-fly translation allows publishing of RDF from large live databases and eliminates the need for replicating the data into a dedicated RDF triple store.
+
+
Read more about the interfaces offered by D2R Server, including example HTTP requests and responses, in the Technical Note Publishing Databases on the Semantic Web.
+
+
Quick start
+
+
You need:
+
+
Java 1.4 or newer on the path (check with
+ java -version if you're not sure),
+
A supported database. D2R Server works with Oracle,
+ MySQL, PostgreSQL, Microsoft SQL Server, and any SQL-92 compatible database. Microsoft
+ Access can be used with some restrictions. More information is
+ available in the
+ D2RQ manual.
+
a modern browser like Firefox, Opera or Safari
+ for using D2R Server's AJAX SPARQL Explorer. Internet Explorer can
+ only browse the HTML pages, but not use the SPARQL Explorer.
+
Optionally, a J2EE servlet container as a
+ deployment target. D2R Server can be run either as a stand-alone
+ web server or inside an existing servlet container.
+
+
+
What to do:
+
+
Download and extract the
+ archive into a suitable location.
+
Download a JDBC driver from your database vendor.
+ Place the driver's JAR file into D2R Server's /lib directory.
+ A list of JDBC drivers
+ from different vendors is maintained by Sun. Also take
+ note of the driver class name (e.g.
+ org.postgresql.Driver for PostgreSQL or
+ oracle.jdbc.driver.OracleDriver for Oracle) and
+ JDBC URL pattern (e.g.
+ jdbc:mysql://servername/database
+ for MySQL) from the driver's documentation. Drivers for MySQL and PostgreSQL are
+ already included with D2R Server.
+
Generate a mapping file for your database schema.
+ Change into the D2R Server directory and run:
You can browse the database content or use the SPARQL Explorer to execute
+ queries and display results in a number of formats.
+
To test the data in an RDF browser, open any resource URI in
+ Tabulator.
+ You may have to tweak your Firefox settings first – see the Tabulator help.
+
+
+
Customizing the database mapping
+
D2R Server uses the D2RQ Mapping Language to map the content of a relational database to RDF. A D2RQ mapping specifies how resources are identified and which properties are used to describe the resources.
+
The generate-mapping script automatically generates a D2RQ mapping from the table structure of a database. The tool generates a new RDF vocabulary for each database, using table names as class names and column names as property names. Semantic Web client applications will understand more of your data if you customize the mapping and replace the auto-generated terms with terms from well-known and publicly accessible RDF vocabularies.
+
The mapping file can be edited with any text editor. Its syntax is described in the
+ D2RQ Manual.
+ D2R Server will automatically detect changes to the mapping file and
+ reload appropriately when you hit the browser's refresh button.
+
Note: The HTML and RDF browser interfaces only work
+ for URI patterns that are relative and do not contain the
+ hash (#) character. For example, a URI pattern such as
+ entries/@@mytable.id@@ is browsable, but
+ http://example.com/entries#@@mytable.id@@ is not. The mapping
+ generator only creates browsable patterns. Non-browsable patterns still
+ work in the SPARQL interface and in RDF dumps.
+
+
Serving vocabulary classes and properties
+
A D2R deployment often requires the introduction of custom classes and properties.
+In the spirit of Linked Data, vocabulary data should be dereferencable by clients. D2RQ infers types of classes and properties, and allows the user to provide labels, comments
+and additional properties.
+
+
D2R Server automatically serves data for vocabularies placed under http://baseURI/vocab/resource/,
+with RDF and HTML representations located at http://baseURI/vocab/data/ and http://baseURI/vocab/page/, respectively.
+The following mapping illustrates the intended usage:
+
+
+
@prefix vocabClass: <http://localhost:2020/vocab/resource/class/> .
+@prefix vocabProperty: <http://localhost:2020/vocab/resource/property/> .
+
+map:offer a d2rq:ClassMap;
+ d2rq:classDefinitionLabel "Offer"@en;
+ d2rq:classDefinitionComment "This is an offer"@en;
+ d2rq:class vocabClass:Offer;
+ .
+
+
+
When dereferenced, http://localhost:2020/vocab/resource/class/Offer will return the specified label and comment as well as the automatically inferred type rdfs:Class. Note that the prefixes are bound to absolute URIs because relative URIs would be based under http://localhost:2020/resource/.
+
+
D2RQ offers several constructs to provide data for classes and properties. Please refer to the D2RQ Language Specification for more details.
+
+
Server configuration
+
+
The server can be configured by adding a configuration block to the mapping file (all parts are optional):
+
+
The rdfs:label is the server name displayed throughout the HTML interface.
-
The d2r:baseURI and d2r:port must be changed to make the D2R Server accessible to remote machines.
-
The d2r:vocabularyIncludeInstances setting controls whether the RDF and HTML representations of vocabulary classes and properties will also list related instances (defaults to true).
-As an alternative to disabling the serving of vocabulary instances, d2rq:resultSizeLimit may be used to limit the amount of returned data.
-Vocabulary serving is a feature of D2RQ and may be controlled using the d2rq:serveVocabulary property.
-
-
The d2r:autoReloadMapping setting specifies whether changes to the mapping file should be detected automatically (defaults to true). This feature has performance implications, so this value should be set to false for high-traffic production systems or when running benchmarks.
-
-
With large databases, some queries will produce too many results. Adding a d2rq:resultSizeLimit to the d2rq:Database section of the mapping file will add a LIMIT clause to all generated statements.
The rdfs:label is the server name displayed throughout the HTML interface.
+
The d2r:baseURI and d2r:port must be changed to make the D2R Server accessible to remote machines.
+
The d2r:vocabularyIncludeInstances setting controls whether the RDF and HTML representations of vocabulary classes and properties will also list related instances (defaults to true).
+As an alternative to disabling the serving of vocabulary instances, d2rq:resultSizeLimit may be used to limit the amount of returned data.
+Vocabulary serving is a feature of D2RQ and may be controlled using the d2rq:serveVocabulary property.
+
+
The d2r:autoReloadMapping setting specifies whether changes to the mapping file should be detected automatically (defaults to true). This feature has performance implications, so this value should be set to false for high-traffic production systems or when running benchmarks.
+
+
With large databases, some queries will produce too many results. Adding a d2rq:resultSizeLimit to the d2rq:Database section of the mapping file will add a LIMIT clause to all generated statements.
+
+
d2rq:resultSizeLimit 500;
+
Not that this effectively “cripples” the server and can cause somewhat unpredictable results. We still recommend it until we can offer a better solution.
-
d2r:metadataTemplate is used to specify a metadata template file within the directory WEB-INF/templates(more details).
+
d2r:metadataTemplate is used to specify a metadata template file within the directory WEB-INF/templates(more details).
-
All statements inside the d2r:documentMetadata block will be added as document metadata to all RDF documents.
+
All statements inside the d2r:documentMetadata block will be added as document metadata to all RDF documents.
Publishing metadata
D2R Server provides a mechanism for adding metadata such as licensing and provenance information to the RDF graphs served by its Linked Data interface. The supplied metadata will be shown in a "Metadata" table below the usual property-value table, which provides a tree-like visualization of the metadata. The (more) links in the visualization allow to expand the view to show the properties of the corresponding entities.
@@ -376,100 +376,100 @@
Optimizing performance
-
-
Running D2R Server as a service on Windows
-
-
To run D2R Server as a service on Windows, use the install-service script:
-
-
install-service servicename mapping.n3
-
-
The service can be started from the Services panel in the management console. The service will log all output including startup errors to the files stdout.log and stderr.log.
-
-
To uninstall the service, use uninstall-service. You have to uninstall and re-install the service when upgrading to a later version of D2R Server.
-
-
Running D2R Server in an existing Servlet Container
-
-
By default, D2R Server is a stand-alone server application that includes
- its own web server. But D2R Server can also be run as a J2EE web application
- inside an existing servlet container, such as Tomcat:
-
-
-
Make sure that your mapping file includes a configuration block, as
- described in the server configuration section.
- Set the port number to that of the servlet container, and set the base
- URI to something like http://servername/webappname/.
-
Change the configFile param in /webapp/WEB-INF/web.xml
- to the name of your configuration file. For deployment, we recommend placing
- the mapping file into the /webapp/WEB-INF/ directory.
-
In D2R Server's main directory, Run ant war. This creates the
- d2r-server.war file. You need Apache Ant for this step.
-
Optionally, if you want a different name for your web application,
- rename the file to webappname.war
-
Deploy the war file into your servlet container, e.g. by copying it
- into Tomcat's webapps directory.
-
-
-
Creating RDF dumps
-
-
Sometimes it is useful to create an RDF dump of the entire database. The dump-rdf script can be used for this purpose. See the D2RQ manual for details.
-
-
Pre-built mapping files
-
-
We collect mapping files for popular database-driven applications.
To run D2R Server as a service on Windows, use the install-service script:
+
+
install-service servicename mapping.n3
+
+
The service can be started from the Services panel in the management console. The service will log all output including startup errors to the files stdout.log and stderr.log.
+
+
To uninstall the service, use uninstall-service. You have to uninstall and re-install the service when upgrading to a later version of D2R Server.
+
+
Running D2R Server in an existing Servlet Container
+
+
By default, D2R Server is a stand-alone server application that includes
+ its own web server. But D2R Server can also be run as a J2EE web application
+ inside an existing servlet container, such as Tomcat:
+
+
+
Make sure that your mapping file includes a configuration block, as
+ described in the server configuration section.
+ Set the port number to that of the servlet container, and set the base
+ URI to something like http://servername/webappname/.
+
Change the configFile param in /webapp/WEB-INF/web.xml
+ to the name of your configuration file. For deployment, we recommend placing
+ the mapping file into the /webapp/WEB-INF/ directory.
+
In D2R Server's main directory, Run ant war. This creates the
+ d2r-server.war file. You need Apache Ant for this step.
+
Optionally, if you want a different name for your web application,
+ rename the file to webappname.war
+
Deploy the war file into your servlet container, e.g. by copying it
+ into Tomcat's webapps directory.
+
+
+
Creating RDF dumps
+
+
Sometimes it is useful to create an RDF dump of the entire database. The dump-rdf script can be used for this purpose. See the D2RQ manual for details.
+
+
Pre-built mapping files
+
+
We collect mapping files for popular database-driven applications.
As Semantic Web technologies are getting mature, there is a growing need for
- RDF applications to access the content of huge, live, non-RDF, legacy databases
- without having to replicate the whole database into RDF. This document describes
- the D2RQ mapping language for treating non-RDF
- relational databases as virtual RDF graphs, and the D2RQ Platform that enables applications to access these graphs through the Jena and
- Sesame
- APIs, as well as over the Web via the SPARQL Protocol and as Linked Data.
As Semantic Web technologies are getting mature, there is a growing need for
+ RDF applications to access the content of huge, live, non-RDF, legacy databases
+ without having to replicate the whole database into RDF. This document describes
+ the D2RQ mapping language for treating non-RDF
+ relational databases as virtual RDF graphs, and the D2RQ Platform that enables applications to access these graphs through the Jena and
+ Sesame
+ APIs, as well as over the Web via the SPARQL Protocol and as Linked Data.
This document describes the D2RQ Platform for accessing
- non-RDF, relational databases as virtual, read-only RDF graphs. D2RQ offers a variety of different RDF-based access mechanisms
- to the content of huge, non-RDF databases without having to replicate the database
- into RDF.
-
-
Using D2RQ you can:
-
-
query a non-RDF database using SPARQL or find(spo) queries,
-
access information in a non-RDF database using the Jena API or the Sesame API,
-
access the content of the database as Linked Data over the Web,
the D2RQ Mapping Language, a declarative mapping language
- for describing the relation between an ontology and an relational data model.
-
-
the D2RQ Engine, a plug-in for the Jena and Sesame Semantic Web toolkits,
- which uses the mappings to rewrite Jena and Sesame API calls to SQL queries against the database and passes
- query results up to the higher layers of the frameworks.
-
D2R Server, an HTTP server that can be used to provide a Linked Data view, a HTML view for debugging and a SPARQL Protocol endpoint over the database.
-
-
-
The figure below depicts the architecture of the D2RQ Platform:
-
-
The D2RQ Engine is implemented as a Jena graph, the basic information representation object within the Jena framework. A D2RQ graph wraps a local relational databases into a virtual, read-only RDF graph. It rewrites Jena or Sesame API calls, find() and SPARQL queries to application-data-model specific SQL queries. The result sets of these SQL queries are transformed into RDF triples or SPARQL result sets that are passed up to the higher layers of the framework. The D2RQ Sesame interface wraps the D2RQ Jena graph implementation behind a Sesame RDF source interface. It provides a read-only Sesame repository interface for querying and reasoning with RDF and RDF Schema.
-
D2R Server is a tool for publishing relational databases on the Semantic Web. It enables RDF and HTML browsers to navigate the content of the database,
-and allows applications to query the database using the SPARQL query language. D2R Server builds on the D2RQ Engine. For detailed information on how to set up D2R Server please refer to the
-separate D2R Server website.
-
-
Example
-
We are using an example database which stores information about conferences,
- papers, authors and topics throughout this manual. The database is mapped to the
- International Semantic Web Community (ISWC) Ontology.
The D2RQ Plaffrom has been tested with these database engines:
-
-
-
Oracle
-
Works with Oracle.
-
-
MySQL
-
Works with MySQL.
-
-
PostgreSQL
-
Works with PostgreSQL.
-
-
Microsoft SQL Server
-
Works with Microsoft SQL Server.
-
-
ODBC data sources (e.g. Microsoft Access)
-
D2RQ can connect to ODBC data sources over the Java's ODBC-JDBC bridge. This works
- with the following limitations:
-
-
The mapping generator does not work.
-
The automatic detection of column types does not work. Therefore, column types
- for all database columns must be provided in the mapping file
- by adding d2rq:textColumn,
- d2rq:numericColumn, d2rq:dateColumn or
- d2rq:timestampColumn statements to the d2rq:Database.
-
- If possible, a dedicated JDBC driver for the database engine should be used
- instead of the ODBC-JDBC bridge to avoid these problems.
-
-
-
Other SQL-92 compatible databases
-
By default, D2RQ will interact with the database using the SQL-92 standard.
- Any compatible database should work out of the box.
-
-
Other databases
-
Not tested; may or may not work. We are interested in feedback about D2RQ running with other database engines.
-
-
-
-
4. The Command Line Tools
-
-
The D2RQ Platform comes with
-two command line tools: a mapping generator that creates a default mapping file
-by analyzing the schema of an existing database, and a dump script that writes
-the complete contents of a database into a single RDF file. The scripts work
-on Windows and Unix systems.
-
-
4.1 Auto-generate Mapping Files
-
-
The generate-mapping script creates a default mapping file by
-analyzing the schema of an existing database. This mapping file can be used
-as-is or can be customized.
This document describes the D2RQ Platform for accessing
+ non-RDF, relational databases as virtual, read-only RDF graphs. D2RQ offers a variety of different RDF-based access mechanisms
+ to the content of huge, non-RDF databases without having to replicate the database
+ into RDF.
+
+
Using D2RQ you can:
+
+
query a non-RDF database using SPARQL or find(spo) queries,
+
access information in a non-RDF database using the Jena API or the Sesame API,
+
access the content of the database as Linked Data over the Web,
the D2RQ Mapping Language, a declarative mapping language
+ for describing the relation between an ontology and an relational data model.
+
+
the D2RQ Engine, a plug-in for the Jena and Sesame Semantic Web toolkits,
+ which uses the mappings to rewrite Jena and Sesame API calls to SQL queries against the database and passes
+ query results up to the higher layers of the frameworks.
+
D2R Server, an HTTP server that can be used to provide a Linked Data view, a HTML view for debugging and a SPARQL Protocol endpoint over the database.
+
+
+
The figure below depicts the architecture of the D2RQ Platform:
+
+
The D2RQ Engine is implemented as a Jena graph, the basic information representation object within the Jena framework. A D2RQ graph wraps a local relational databases into a virtual, read-only RDF graph. It rewrites Jena or Sesame API calls, find() and SPARQL queries to application-data-model specific SQL queries. The result sets of these SQL queries are transformed into RDF triples or SPARQL result sets that are passed up to the higher layers of the framework. The D2RQ Sesame interface wraps the D2RQ Jena graph implementation behind a Sesame RDF source interface. It provides a read-only Sesame repository interface for querying and reasoning with RDF and RDF Schema.
+
D2R Server is a tool for publishing relational databases on the Semantic Web. It enables RDF and HTML browsers to navigate the content of the database,
+and allows applications to query the database using the SPARQL query language. D2R Server builds on the D2RQ Engine. For detailed information on how to set up D2R Server please refer to the
+separate D2R Server website.
+
+
Example
+
We are using an example database which stores information about conferences,
+ papers, authors and topics throughout this manual. The database is mapped to the
+ International Semantic Web Community (ISWC) Ontology.
The D2RQ Plaffrom has been tested with these database engines:
+
+
+
Oracle
+
Works with Oracle.
+
+
MySQL
+
Works with MySQL.
+
+
PostgreSQL
+
Works with PostgreSQL.
+
+
Microsoft SQL Server
+
Works with Microsoft SQL Server.
+
+
ODBC data sources (e.g. Microsoft Access)
+
D2RQ can connect to ODBC data sources over the Java's ODBC-JDBC bridge. This works
+ with the following limitations:
+
+
The mapping generator does not work.
+
The automatic detection of column types does not work. Therefore, column types
+ for all database columns must be provided in the mapping file
+ by adding d2rq:textColumn,
+ d2rq:numericColumn, d2rq:dateColumn or
+ d2rq:timestampColumn statements to the d2rq:Database.
+
+ If possible, a dedicated JDBC driver for the database engine should be used
+ instead of the ODBC-JDBC bridge to avoid these problems.
+
+
+
Other SQL-92 compatible databases
+
By default, D2RQ will interact with the database using the SQL-92 standard.
+ Any compatible database should work out of the box.
+
+
Other databases
+
Not tested; may or may not work. We are interested in feedback about D2RQ running with other database engines.
+
+
+
+
4. The Command Line Tools
+
+
The D2RQ Platform comes with
+two command line tools: a mapping generator that creates a default mapping file
+by analyzing the schema of an existing database, and a dump script that writes
+the complete contents of a database into a single RDF file. The scripts work
+on Windows and Unix systems.
+
+
4.1 Auto-generate Mapping Files
+
+
The generate-mapping script creates a default mapping file by
+analyzing the schema of an existing database. This mapping file can be used
+as-is or can be customized.
JDBC connection URL for the database. Refer to your JDBC driver documentation
+for the format for your database engine. Examples:
+
MySQL: jdbc:mysql://servername/databasename
+PostgreSQL: jdbc:postgresql://servername/databasename
Oracle: jdbc:oracle:thin:@servername:1521:databasename
-Microsoft SQL Server: jdbc:sqlserver://servername;databaseName=databasename (due to the semicolon, the URL must be put in quotes when passed as a command-line argument in Linux/Unix shells)
-
-
-
-u username
-
The login name of the database user
-
-
-p password
-
The password of the database user
-
-
-d driverclass
-
The fully qualified Java class name of the database driver. The jar file
-containing the JDBC driver has to be in D2RQ's /lib/db-drivers/
-directory. Drivers for MySQL and PostgreSQL are provided with the
-download, for other databases a driver has to be downloaded from the vendor
-or a third party. To find the driver class name, consult the driver
-documentation. Examples:
-
MySQL: com.mysql.jdbc.Driver
-PostgreSQL: org.postgresql.Driver
+Microsoft SQL Server: jdbc:sqlserver://servername;databaseName=databasename (due to the semicolon, the URL must be put in quotes when passed as a command-line argument in Linux/Unix shells)
+
+
+
-u username
+
The login name of the database user
+
+
-p password
+
The password of the database user
+
+
-d driverclass
+
The fully qualified Java class name of the database driver. The jar file
+containing the JDBC driver has to be in D2RQ's /lib/db-drivers/
+directory. Drivers for MySQL and PostgreSQL are provided with the
+download, for other databases a driver has to be downloaded from the vendor
+or a third party. To find the driver class name, consult the driver
+documentation. Examples:
The generated mapping will be stored in this file in N3 syntax. If this
-parameter is omitted, the mapping will be written to standard out.
-
-
-b baseURI
-
The base URI is used to construct a vocabulary namespace that will automatically be served
-as Linked Data by D2R Server, following the convention http://baseURI/vocab/resource/.
-This should be the same base URI that is used when invoking the server. Defaults to http://localhost:2020/.
-For more information on vocabulary serving, see section 7.4.
The dump-rdf script provides a way of dumping the contents of the
-whole database into a single RDF file. This can be done with or without a
-mapping file. If a mapping file is specified, then the script will use it
-to translate the database contents to RDF. If no mapping file is specified,
-then the script will invoke generate-mapping and use its default
-mapping for the translation.
-
-
dump-rdf -m mapping.n3 [output parameters]
-
-
If no mapping file is provided, then the database connection must be
-specified on the command line. With the exception of fetchSize, the meaning of all parameters is the
-same as for the generate-mapping script.
-
-
Several optional parameters control the RDF output:
-
-
-
-f format
-
The RDF syntax to use for output. Supported syntaxes are "RDF/XML" (the default),
- "RDF/XML-ABBREV", "N3", "N-TRIPLE". "N-TRIPLE" works best for large
+Microsoft SQL Server: com.microsoft.sqlserver.jdbc.SQLServerDriver
+
+
+
-o outfile.n3
+
The generated mapping will be stored in this file in N3 syntax. If this
+parameter is omitted, the mapping will be written to standard out.
+
+
-b baseURI
+
The base URI is used to construct a vocabulary namespace that will automatically be served
+as Linked Data by D2R Server, following the convention http://baseURI/vocab/resource/.
+This should be the same base URI that is used when invoking the server. Defaults to http://localhost:2020/.
+For more information on vocabulary serving, see section 7.4.
The dump-rdf script provides a way of dumping the contents of the
+whole database into a single RDF file. This can be done with or without a
+mapping file. If a mapping file is specified, then the script will use it
+to translate the database contents to RDF. If no mapping file is specified,
+then the script will invoke generate-mapping and use its default
+mapping for the translation.
+
+
dump-rdf -m mapping.n3 [output parameters]
+
+
If no mapping file is provided, then the database connection must be
+specified on the command line. With the exception of fetchSize, the meaning of all parameters is the
+same as for the generate-mapping script.
+
+
Several optional parameters control the RDF output:
+
+
+
-f format
+
The RDF syntax to use for output. Supported syntaxes are "RDF/XML" (the default),
+ "RDF/XML-ABBREV", "N3", "N-TRIPLE". "N-TRIPLE" works best for large
databases.
-
-s fetchSize
+
-s fetchSize
The number of rows to retrieve with every database request. This value is particularily important to control memory resources of both the D2RQ and the database server when performing dumps. dump-rdf sets this value to 500 by default, or to Integer.MIN_VALUE for MySQL in order to enable streaming mode.
This value may alternatively be specified in the mapping using d2rq:fetchSize.
-
-
-
-b baseURI
-
A base URI for resolving relative URI patterns.
-
-
-o outfile
-
Name of the destination file. Defaults to standard output.
At the time of writing, the latest Jena release is version 2.4. D2RQ requires
- a more recent custom-built version of Jena, the version that ships with
- ARQ 1.4. All required jar files
- are included in the D2RQ distribution. (Jena 2.4 and 2.3 may
- work to some extent.)
-
-
Installation
-
-
Add the d2rq-X.X.jar file from D2RQ's /lib directory to your
- application's classpath.
-
Add all jar files from the /lib/arq-1.4 directory
- to your application's classpath.
-
Add a JDBC driver for your database to your application's
- classpath. Drivers for some popular databases are found
- in D2RQ's /lib/db-drivers directory.
-
-
-
Debugging
-
D2RQ uses the Apache Commons - Logging
- API for logging. To enable D2RQ debug messages, set the log level for logger
- de.fuberlin.wiwiss.d2rq to ALL. An easy way to do this is:
-
The ModelD2RQ class provides a Jena
-Model
-view on the data
-in a D2RQ-mapped database. The example shows how a ModelD2RQ is set up
-using a mapping file, and how Jena API calls are used to extract
-information about papers and their authors from the model.
-
-
The ISWC and FOAF classes have been created with Jena's
-schemagen
-tool. The DC and RDF classes are part of Jena.
-
-
// Set up the ModelD2RQ using a mapping file
-Model m = new ModelD2RQ("file:doc/example/mapping-iswc.n3");
-
-// Find anything with an rdf:type of iswc:InProceedings
-StmtIterator paperIt = m.listStatements(null, RDF.type, ISWC.InProceedings);
-
-// List found papers and print their titles
-while (paperIt.hasNext()) {
- Resource paper = paperIt.nextStatement().getSubject();
- System.out.println("Paper: " + paper.getProperty(DC.title).getString());
-
- // List authors of the paper and print their names
- StmtIterator authorIt = paper.listProperties(DC.creator);
- while (authorIt.hasNext()) {
- Resource author = authorIt.nextStatement().getResource();
- System.out.println("Author: " + author.getProperty(FOAF.name).getString());
- }
- System.out.println();
-}
-
-
-
5.2 Using Jena's Graph API
-
-
In some situations, it is better to use Jena's low-level
-Graph
-API instead of the Model API. D2RQ provides an implementation
-of the Graph interface, the GraphD2RQ.
-
-
The following example shows how the Graph API is used to find all papers
-that have been published in 2003.
-
-
// Load mapping file
-Model mapping = FileManager.get().loadModel("doc/example/mapping-iswc.n3");
-
-// Set up the GraphD2RQ
-GraphD2RQ g = new GraphD2RQ(mapping, "http://localhost:2020/");
-
-// Create a find(spo) pattern
-Node subject = Node.ANY;
-Node predicate = DC.date.asNode();
-Node object = Node.createLiteral("2003", null, XSDDatatype.XSDgYear);
-Triple pattern = new Triple(subject, predicate, object);
-
-// Query the graph
-Iterator it = g.find(pattern);
-
-// Output query results
-while (it.hasNext()) {
- Triple t = (Triple) it.next();
- System.out.println("Published in 2003: " + t.getSubject());
-}
-
-
5.2.1 The CachingGraphD2RQ
-
-
In addition to the GraphD2RQ, there is a CachingGraphD2RQ which supports
-the same API and uses a LRU cache to remember a number of recent query
-results. This will improve performance for repeated queries, but will
-report inconsistent results if the database is updated during the lifetime
-of the CachingGraphD2RQ.
-
-
-
5.3 Using SPARQL
-
D2RQ can answer SPARQL queries against a D2RQ model. The SPARQL queries
-are processed by Jena's ARQ query engine. The example shows how a D2RQ model
-is set up, how a SPARQL query is executed, and how the results are written
-to the console.
D2RQ comes with a Jena assembler. Jena assembler specifications are RDF
-configuration files that describe how to construct a Jena model. For more
-information on Jena assemblers, see the
-Jena Assembler quickstart page.
-
-
The following example shows an assembler specification for a D2RQ model:
D2RQ model specifications support these two properties:
-
-
-
d2rq:mappingFile (required)
-
The URI of a D2RQ mapping file to use for setting up the model
-
d2rq:resourceBaseURI (optional)
-
The base URI for turning relative URI patterns into full URIs.
- If not specified, D2RQ will pick an appropriate base URI.
-
-
-
This usage example will create a D2RQ model from a model specification, and write it to the console:
-
-
// Load assembler specification from file
-Model assemblerSpec = FileManager.get().loadModel("doc/example/assembler.n3");
-
-// Get the model resource
-Resource modelSpec = assemblerSpec.createResource(assemblerSpec.expandPrefix(":myModel"));
-
-// Assemble a model
-Model m = Assembler.general.openModel(modelSpec);
-
-// Write it to System.out
-m.write(System.out);
-
-
-
5.5. Running the D2RQ unit tests
-
-
Some of the D2RQ unit tests are using the ISWC example database from the /doc/example directory of the D2RQ distribution. To run the tests:
-
-
-
Set up a MySQL database and import the database dump from /doc/example/iswc-mysql.sql.
-
Change the JDBC database connection string, username and password in /doc/example/mapping-iswc.n3
- to match your configuration.
-
Run the D2RQTestSuite class as a JUnit test.
-
-
-
-
-
6. Using D2RQ within Sesame
-
This section describes how the D2RQ Engine is used within the Sesame 1.2 RDF API.
-
Jena 2 can be downloaded from http://jena.sourceforge.net/downloads.html (The Jena package is neccessary, because the Sesame interface is just providing a wrapper around the Jena D2RQ interface)
You have to add the "d2rq.jar" and "d2rq-to-sesame.jar" files from the "bin" directory
- of the D2RQ distribution together with the Jena 2 and Sesame 1.2 jar files to your classpath. To run D2RQ only the jar files
-
-
jena.jar
-
antlr-2.7.5.jar
-
commons-logging.jar
-
concurrent.jar
-
xercesImpl.jar
-
and xml-apis.jar
-
- from the "lib" folder of the Jena 2.3 distribution are required.
-
-
6.1 Using D2RQ as Sesame repository
-
The following example shows how RDQL is used to get all information about the paper with the URI
- "http://www.conference.org/conf02004/paper#Paper1" out of a D2RQRepository.
-
-import de.fuberlin.wiwiss.d2rq.sesame.D2RQRepository;
-import de.fuberlin.wiwiss.d2rq.sesame.D2RQSource;
-
-import org.openrdf.model.Value;
-import org.openrdf.sesame.Sesame;
-import org.openrdf.sesame.constants.QueryLanguage;
-import org.openrdf.sesame.query.QueryResultsTable;
-import org.openrdf.sesame.repository.SesameRepository;
-
-...
-
-try{
- // Initialize repository
- D2RQSource source = new D2RQSource("file:///where/you/stored/the/d2rq-mapping.n3", "N3");
- SesameRepository repos = new D2RQRepository("urn:youRepository", source, Sesame.getService());
-
- // Query the repository
- String query = "SELECT ?x, ?y WHERE (<http://www.conference.org/conf02004/paper#Paper1>, ?x, ?y)";
- QueryResultsTable result = repos.performTableQuery(QueryLanguage.RDQL, query);
-
- // print the result
- int rows = result.getRowCount();
- int cols = result.getColumnCount();
- for(int i = 0; i < rows; i++){
- for(int j = 0; j < cols; j++){
- Value v = result.getValue(i,j);
- System.out.print(v.toString() + " ");
- }
- System.out.println();
- }
-} catch(Exception e){
- // catches D2RQException from D2RQSource construcor
- // catches java.io.IOException,
- // org.openrdf.sesame.query.MalformedQueryException,
- // org.openrdf.sesame.query.QueryEvaluationException,
- // org.openrdf.sesame.config.AccessDeniedException
- // from performTableQuery
- e.printStackTrace();
-}
-
-
-
-
-
7. Language Specification
-
The D2RQ mapping language is a declarative language for describing the relation between
- a relational database schemata and RDFS vocabularies or OWL ontologies. A D2RQ map is an RDF document.
-
-
The language is formally defined by the D2RQ RDFS Schema.
-
- The D2RQ namespace is http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/0.1#
-
An ontology is mapped to a database schema using d2rq:ClassMaps and
- d2rq:PropertyBridges.
- The central object within D2RQ and also the
- object to start with when writing a new D2RQ map is the ClassMap. A ClassMap
- represents a class or a group of similar classes of the ontology. A ClassMap
- specifies how instances of the class are identified. It has a set of PropertyBridges,
- which specify how the properties of an instance are created.
-
The figure below shows the structure of an example D2RQ map:
-
-
The following example D2RQ map relates the table conferences in a database to the class
- conference in an ontology. You can use the map as a template for writing your own
- maps.
-
# D2RQ Namespace
-@prefix d2rq: <http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/0.1#> .
-# Namespace of the ontology
-@prefix : <http://annotation.semanticweb.org/iswc/iswc.daml#> .
-
-# Namespace of the mapping file; does not appear in mapped data
-@prefix map: <file:///Users/d2r/example.n3#> .
-
-# Other namespaces
-@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
-@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
-
-map:Database1 a d2rq:Database;
- d2rq:jdbcDSN "jdbc:mysql://localhost/iswc";
- d2rq:jdbcDriver "com.mysql.jdbc.Driver";
- d2rq:username "user";
- d2rq:password "password";
- .
-# -----------------------------------------------
-# CREATE TABLE Conferences (ConfID int, Name text, Location text);
-
-map:Conference a d2rq:ClassMap;
- d2rq:dataStorage map:Database1.
- d2rq:class :Conference;
- d2rq:uriPattern "http://conferences.org/comp/confno@@Conferences.ConfID@@";
- .
-map:eventTitle a d2rq:PropertyBridge;
- d2rq:belongsToClassMap map:Conference;
- d2rq:property :eventTitle;
- d2rq:column "Conferences.Name";
- d2rq:datatype xsd:string;
- .
-map:location a d2rq:PropertyBridge;
- d2rq:belongsToClassMap map:Conference;
- d2rq:property :location;
- d2rq:column "Conferences.Location";
- d2rq:datatype xsd:string;
- .
-
-
The constructs of the D2RQ mapping language are described in detail below.
-
-
-
7.1 Database
-
A d2rq:Database defines a JDBC or ODBC connection to a local relational database
- and specifies the type of the database columns used by D2RQ. A D2RQ map can
- contain several d2rq:Databases for accessing different local databases.
-
Properties
-
-
-
d2rq:jdbcDSN
-
The JDBC database URL. This is a string of the form
- jdbc:subprotocol:subname. For a MySQL database,
- this is something like jdbc:mysql://hostname:port/dbname.
- Examples for other databases
-
-
-
d2rq:jdbcDriver
-
The JDBC driver class name for the database. Used together with d2rq:jdbcDSN.
- Example: com.mysql.jdbc.Driver for MySQL.
-
-
-
d2rq:odbcDSN
-
The ODBC data source name of the database.
-
-
-
d2rq:username
-
A username if required by the database.
-
-
-
d2rq:password
-
A password if required by the database.
-
-
-
d2rq:resultSizeLimit
-
An integer value that will be added as a LIMIT clause to all
- generated SQL queries. This sets an upper bound for the number
- of results returned from large databases. Note that this effectively
- “cripples” the server and can cause unpredictable
+
+
+
-b baseURI
+
A base URI for resolving relative URI patterns.
+
+
-o outfile
+
Name of the destination file. Defaults to standard output.
At the time of writing, the latest Jena release is version 2.4. D2RQ requires
+ a more recent custom-built version of Jena, the version that ships with
+ ARQ 1.4. All required jar files
+ are included in the D2RQ distribution. (Jena 2.4 and 2.3 may
+ work to some extent.)
+
+
Installation
+
+
Add the d2rq-X.X.jar file from D2RQ's /lib directory to your
+ application's classpath.
+
Add all jar files from the /lib/arq-1.4 directory
+ to your application's classpath.
+
Add a JDBC driver for your database to your application's
+ classpath. Drivers for some popular databases are found
+ in D2RQ's /lib/db-drivers directory.
+
+
+
Debugging
+
D2RQ uses the Apache Commons - Logging
+ API for logging. To enable D2RQ debug messages, set the log level for logger
+ de.fuberlin.wiwiss.d2rq to ALL. An easy way to do this is:
+
The ModelD2RQ class provides a Jena
+Model
+view on the data
+in a D2RQ-mapped database. The example shows how a ModelD2RQ is set up
+using a mapping file, and how Jena API calls are used to extract
+information about papers and their authors from the model.
+
+
The ISWC and FOAF classes have been created with Jena's
+schemagen
+tool. The DC and RDF classes are part of Jena.
+
+
// Set up the ModelD2RQ using a mapping file
+Model m = new ModelD2RQ("file:doc/example/mapping-iswc.n3");
+
+// Find anything with an rdf:type of iswc:InProceedings
+StmtIterator paperIt = m.listStatements(null, RDF.type, ISWC.InProceedings);
+
+// List found papers and print their titles
+while (paperIt.hasNext()) {
+ Resource paper = paperIt.nextStatement().getSubject();
+ System.out.println("Paper: " + paper.getProperty(DC.title).getString());
+
+ // List authors of the paper and print their names
+ StmtIterator authorIt = paper.listProperties(DC.creator);
+ while (authorIt.hasNext()) {
+ Resource author = authorIt.nextStatement().getResource();
+ System.out.println("Author: " + author.getProperty(FOAF.name).getString());
+ }
+ System.out.println();
+}
+
+
+
5.2 Using Jena's Graph API
+
+
In some situations, it is better to use Jena's low-level
+Graph
+API instead of the Model API. D2RQ provides an implementation
+of the Graph interface, the GraphD2RQ.
+
+
The following example shows how the Graph API is used to find all papers
+that have been published in 2003.
+
+
// Load mapping file
+Model mapping = FileManager.get().loadModel("doc/example/mapping-iswc.n3");
+
+// Set up the GraphD2RQ
+GraphD2RQ g = new GraphD2RQ(mapping, "http://localhost:2020/");
+
+// Create a find(spo) pattern
+Node subject = Node.ANY;
+Node predicate = DC.date.asNode();
+Node object = Node.createLiteral("2003", null, XSDDatatype.XSDgYear);
+Triple pattern = new Triple(subject, predicate, object);
+
+// Query the graph
+Iterator it = g.find(pattern);
+
+// Output query results
+while (it.hasNext()) {
+ Triple t = (Triple) it.next();
+ System.out.println("Published in 2003: " + t.getSubject());
+}
+
+
5.2.1 The CachingGraphD2RQ
+
+
In addition to the GraphD2RQ, there is a CachingGraphD2RQ which supports
+the same API and uses a LRU cache to remember a number of recent query
+results. This will improve performance for repeated queries, but will
+report inconsistent results if the database is updated during the lifetime
+of the CachingGraphD2RQ.
+
+
+
5.3 Using SPARQL
+
D2RQ can answer SPARQL queries against a D2RQ model. The SPARQL queries
+are processed by Jena's ARQ query engine. The example shows how a D2RQ model
+is set up, how a SPARQL query is executed, and how the results are written
+to the console.
D2RQ comes with a Jena assembler. Jena assembler specifications are RDF
+configuration files that describe how to construct a Jena model. For more
+information on Jena assemblers, see the
+Jena Assembler quickstart page.
+
+
The following example shows an assembler specification for a D2RQ model:
D2RQ model specifications support these two properties:
+
+
+
d2rq:mappingFile (required)
+
The URI of a D2RQ mapping file to use for setting up the model
+
d2rq:resourceBaseURI (optional)
+
The base URI for turning relative URI patterns into full URIs.
+ If not specified, D2RQ will pick an appropriate base URI.
+
+
+
This usage example will create a D2RQ model from a model specification, and write it to the console:
+
+
// Load assembler specification from file
+Model assemblerSpec = FileManager.get().loadModel("doc/example/assembler.n3");
+
+// Get the model resource
+Resource modelSpec = assemblerSpec.createResource(assemblerSpec.expandPrefix(":myModel"));
+
+// Assemble a model
+Model m = Assembler.general.openModel(modelSpec);
+
+// Write it to System.out
+m.write(System.out);
+
+
+
5.5. Running the D2RQ unit tests
+
+
Some of the D2RQ unit tests are using the ISWC example database from the /doc/example directory of the D2RQ distribution. To run the tests:
+
+
+
Set up a MySQL database and import the database dump from /doc/example/iswc-mysql.sql.
+
Change the JDBC database connection string, username and password in /doc/example/mapping-iswc.n3
+ to match your configuration.
+
Run the D2RQTestSuite class as a JUnit test.
+
+
+
+
+
6. Using D2RQ within Sesame
+
This section describes how the D2RQ Engine is used within the Sesame 1.2 RDF API.
+
Jena 2 can be downloaded from http://jena.sourceforge.net/downloads.html (The Jena package is neccessary, because the Sesame interface is just providing a wrapper around the Jena D2RQ interface)
You have to add the "d2rq.jar" and "d2rq-to-sesame.jar" files from the "bin" directory
+ of the D2RQ distribution together with the Jena 2 and Sesame 1.2 jar files to your classpath. To run D2RQ only the jar files
+
+
jena.jar
+
antlr-2.7.5.jar
+
commons-logging.jar
+
concurrent.jar
+
xercesImpl.jar
+
and xml-apis.jar
+
+ from the "lib" folder of the Jena 2.3 distribution are required.
+
+
6.1 Using D2RQ as Sesame repository
+
The following example shows how RDQL is used to get all information about the paper with the URI
+ "http://www.conference.org/conf02004/paper#Paper1" out of a D2RQRepository.
+
+import de.fuberlin.wiwiss.d2rq.sesame.D2RQRepository;
+import de.fuberlin.wiwiss.d2rq.sesame.D2RQSource;
+
+import org.openrdf.model.Value;
+import org.openrdf.sesame.Sesame;
+import org.openrdf.sesame.constants.QueryLanguage;
+import org.openrdf.sesame.query.QueryResultsTable;
+import org.openrdf.sesame.repository.SesameRepository;
+
+...
+
+try{
+ // Initialize repository
+ D2RQSource source = new D2RQSource("file:///where/you/stored/the/d2rq-mapping.n3", "N3");
+ SesameRepository repos = new D2RQRepository("urn:youRepository", source, Sesame.getService());
+
+ // Query the repository
+ String query = "SELECT ?x, ?y WHERE (<http://www.conference.org/conf02004/paper#Paper1>, ?x, ?y)";
+ QueryResultsTable result = repos.performTableQuery(QueryLanguage.RDQL, query);
+
+ // print the result
+ int rows = result.getRowCount();
+ int cols = result.getColumnCount();
+ for(int i = 0; i < rows; i++){
+ for(int j = 0; j < cols; j++){
+ Value v = result.getValue(i,j);
+ System.out.print(v.toString() + " ");
+ }
+ System.out.println();
+ }
+} catch(Exception e){
+ // catches D2RQException from D2RQSource construcor
+ // catches java.io.IOException,
+ // org.openrdf.sesame.query.MalformedQueryException,
+ // org.openrdf.sesame.query.QueryEvaluationException,
+ // org.openrdf.sesame.config.AccessDeniedException
+ // from performTableQuery
+ e.printStackTrace();
+}
+
+
+
+
+
7. Language Specification
+
The D2RQ mapping language is a declarative language for describing the relation between
+ a relational database schemata and RDFS vocabularies or OWL ontologies. A D2RQ map is an RDF document.
+
+
The language is formally defined by the D2RQ RDFS Schema.
+
+ The D2RQ namespace is http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/0.1#
+
An ontology is mapped to a database schema using d2rq:ClassMaps and
+ d2rq:PropertyBridges.
+ The central object within D2RQ and also the
+ object to start with when writing a new D2RQ map is the ClassMap. A ClassMap
+ represents a class or a group of similar classes of the ontology. A ClassMap
+ specifies how instances of the class are identified. It has a set of PropertyBridges,
+ which specify how the properties of an instance are created.
+
The figure below shows the structure of an example D2RQ map:
+
+
The following example D2RQ map relates the table conferences in a database to the class
+ conference in an ontology. You can use the map as a template for writing your own
+ maps.
+
# D2RQ Namespace
+@prefix d2rq: <http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/0.1#> .
+# Namespace of the ontology
+@prefix : <http://annotation.semanticweb.org/iswc/iswc.daml#> .
+
+# Namespace of the mapping file; does not appear in mapped data
+@prefix map: <file:///Users/d2r/example.n3#> .
+
+# Other namespaces
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+
+map:Database1 a d2rq:Database;
+ d2rq:jdbcDSN "jdbc:mysql://localhost/iswc";
+ d2rq:jdbcDriver "com.mysql.jdbc.Driver";
+ d2rq:username "user";
+ d2rq:password "password";
+ .
+# -----------------------------------------------
+# CREATE TABLE Conferences (ConfID int, Name text, Location text);
+
+map:Conference a d2rq:ClassMap;
+ d2rq:dataStorage map:Database1.
+ d2rq:class :Conference;
+ d2rq:uriPattern "http://conferences.org/comp/confno@@Conferences.ConfID@@";
+ .
+map:eventTitle a d2rq:PropertyBridge;
+ d2rq:belongsToClassMap map:Conference;
+ d2rq:property :eventTitle;
+ d2rq:column "Conferences.Name";
+ d2rq:datatype xsd:string;
+ .
+map:location a d2rq:PropertyBridge;
+ d2rq:belongsToClassMap map:Conference;
+ d2rq:property :location;
+ d2rq:column "Conferences.Location";
+ d2rq:datatype xsd:string;
+ .
+
+
The constructs of the D2RQ mapping language are described in detail below.
+
+
+
7.1 Database
+
A d2rq:Database defines a JDBC or ODBC connection to a local relational database
+ and specifies the type of the database columns used by D2RQ. A D2RQ map can
+ contain several d2rq:Databases for accessing different local databases.
+
Properties
+
+
+
d2rq:jdbcDSN
+
The JDBC database URL. This is a string of the form
+ jdbc:subprotocol:subname. For a MySQL database,
+ this is something like jdbc:mysql://hostname:port/dbname.
+ Examples for other databases
+
+
+
d2rq:jdbcDriver
+
The JDBC driver class name for the database. Used together with d2rq:jdbcDSN.
+ Example: com.mysql.jdbc.Driver for MySQL.
+
+
+
d2rq:odbcDSN
+
The ODBC data source name of the database.
+
+
+
d2rq:username
+
A username if required by the database.
+
+
+
d2rq:password
+
A password if required by the database.
+
+
+
d2rq:resultSizeLimit
+
An integer value that will be added as a LIMIT clause to all
+ generated SQL queries. This sets an upper bound for the number
+ of results returned from large databases. Note that this effectively
+ “cripples” the server and can cause unpredictable
results. Also see d2rq:limit and d2rq:limitInverse, which may be used to impose
- result limits on individual property bridges.
-
-
-
d2rq:fetchSize
+ result limits on individual property bridges.
+
+
+
d2rq:fetchSize
An integer value that specifies the number of rows to retrieve
with every database request. This value is particularily important to control memory resources
of both the D2RQ and the database server when performing dumps.
dump-rdf sets this value to 500 by default, or to Integer.MIN_VALUE for MySQL in order to
- enable streaming mode.
-
-
-
d2rq:allowDistinct
-
Specifies the databases ability to handle DISTINCT correctly.
- Value: "true" or "false". For example MSAccess cuts fields longer than 256 chars.
These properties are used to declare the column type of database columns.
- Values are column names in Table_name.column_name notation.
- These properties do not need to be specified unless
- the engine is for some reason unable to determine the correct
- column type by itself. The d2rq:timestampColumn
- is for column types that combine a date and a time.
-
Most JDBC drivers offer a range of JDBC connection properties,
- which specify advanced configuration options for the JDBC database connection.
- A D2RQ mapping file can be made to use arbitrary connection properties when
- setting up the JDBC connection. This is done through the jdbc: namespace
- (namespace URI: http://d2rq.org/terms/jdbc/). RDF properties in that
- namespace will be passed as connection properties.
- Consult your JDBC driver's documentation for a list of available properties.
These properties are used to declare the column type of database columns.
+ Values are column names in Table_name.column_name notation.
+ These properties do not need to be specified unless
+ the engine is for some reason unable to determine the correct
+ column type by itself. The d2rq:timestampColumn
+ is for column types that combine a date and a time.
+
Most JDBC drivers offer a range of JDBC connection properties,
+ which specify advanced configuration options for the JDBC database connection.
+ A D2RQ mapping file can be made to use arbitrary connection properties when
+ setting up the JDBC connection. This is done through the jdbc: namespace
+ (namespace URI: http://d2rq.org/terms/jdbc/). RDF properties in that
+ namespace will be passed as connection properties.
+ Consult your JDBC driver's documentation for a list of available properties.
The example uses two connection properties which are understood by the MySQL
+JDBC driver: autoReconnect=true and
+zeroDateTimeBehavior=convertToNull.
Keep-alive long-term connections
@@ -841,292 +841,292 @@
7.1 Database
By default the noop query is "SELECT 1", which may not work with some DBMS.
For this purpose, the default query may be overridden with a custom noop query.
-
-
7.2 ClassMap
-
A d2rq:ClassMap represents a class or a group of similar classes
- of an OWL ontology or RDFS schema. A class map defines how instances of the
- class are identified. It is connected to a d2rq:Database and has
- a set of d2rq:PropertyBridges which attach properties to the instances.
-
-
7.2.1 Resource Identity
-
-
D2RQ provides four different mechanisms of assigning identifiers to the instances
- in the database:
-
-
-
URI patterns
-
A URI pattern is instantiated by inserting values of
- certain database columns into a pattern. Examples:
The parts between @@'s mark database columns in
- Table.Column notation. URI patterns are used
- with the d2rq:uriPattern property.
-
Certain characters, like spaces or the hash sign, are not allowed
- in URIs or have special meaning. Columns that contain such characters
- need to be encoded before their values can be inserted into a URI
- pattern:
-
-
If a column is given as @@Table.Column|urlencode@@,
- then URL encoding
- is applied before the string is inserted.
-
If a column is given as @@Table.Column|urlify@@,
- then URL encoding is applied, with an additional rule that spaces are
- converted to underscores (_). Some people find that this
- produces friendlier URIs.
-
-
-
-
Relative URI patterns
-
A relative URI pattern is a URI pattern that generates
- relative URIs:
-
persons/@@Persons.ID@@
-
They will be combined with a base URI provided by
- the processing environment to form full URIs. Relative
- URI patterns allow the creation of portable mappings that
- can be used for multiple instances of the same database
- schema. Relative URI patterns are generated
- with the d2rq:uriPattern property.
-
-
-
URI columns
-
In some cases, the database may already contain URIs that
- can be used as resource identifiers, such as web page and
- document URLs. URI are generated from columns
- with the d2rq:uriColumns property.
-
-
-
Blank nodes
-
RDF also has the concept of blank nodes, existential qualifiers
- that denote some resource that exists and has certain properties,
- but is not named. In D2RQ, blank nodes can be generated from one or
- more columns. A distinct blank node will be generated for each distinct
- set of values of these columns. The columns are specified using the
- with the d2rq:bNodeIdColumns property.
-
-
-
-
Singleton classmaps
-
A d2rq:ClassMap usually produces many resources. Sometimes
- it is desirable to have a class map that only produces a single resource
- with fixed, static identity. In that case, one can use the
- d2rq:constantValue property to name the single instance.
-
-
Properties of d2rq:ClassMap
-
-
-
d2rq:dataStorage
-
Reference to a d2rq:Database where the instance data is stored.
-
-
-
d2rq:class
-
An RDF-S or OWL class. All resources generated by this
- ClassMap are instances of this class.
-
-
-
d2rq:uriPattern
-
Specifies a URI pattern that will be used to
- identify instances of this class map.
-
-
-
d2rq:uriColumn
-
A database column containing URIrefs for identifying
- instances of this class map. The column name has to be in the form
- "TableName.ColumnName".
-
-
-
d2rq:bNodeIdColumns
-
A comma-seperated list of column names in
- "TableName.ColumnName" notation. The instances of
- this class map will be blank nodes, one distinct blank node
- per distinct tuple of these columns.
-
-
-
d2rq:constantValue
-
This class map will only have a single instance, which is
- named by the value of this property. This can be a blank node or a URI.
-
-
-
d2rq:translateWith
-
Assigns a d2rq:TranslationTable to the class map. Values
- from the d2rq:uriColumn or d2rq:uriPattern will be translated by the table
- before a resource is generated. See below
- for details.
-
-
-
d2rq:containsDuplicates
-
Must be specified if a class map uses information from tables
- that are not fully normalized. If the d2rq:containsDuplicates property value
- is set to "true", then D2RQ adds a DISTINCT clause to all queries using
- this classMap. "False" is the default value, which doesn't have
- to be explicitly declared. Adding this property to class maps based on normalized
- database tables degrades query performance, but doesn't affect query results.
-
-
-
d2rq:additionalProperty
-
Adds an AdditionalProperty
- to all instances of this class.
- This might be useful for adding rdfs:seeAlso properties or other fixed
- statements to all instances of the class.
-
-
-
d2rq:condition
-
Specifies an SQL WHERE condition. An instance of this class
- will only be generated for database rows that satisfy the condition. Conditions
- can be used to hide parts of the database from D2RQ, e.g. deny access to
- data which is older or newer than a certain date. See section Conditional
- Mappings for details.
-
-
-
d2rq:classDefinitionLabel
-
Specifies a label that will be served as rdfs:label
- for all associated class definitions. Multiple lables, e.g. in several languages, are supported.
-
-
-
d2rq:classDefinitionComment
-
Specifies a comment that will be served as rdfs:comment
- for all associated class definitions. Multiple comments are supported.
Inverse of d2rq:class and unnecessary if d2rq:class is used. Specifies
- that a d2rq:classMap is used to create instances of an OWL or RDF-S class.
-
-
-
-
Example: ClassMap where instances are identified using an URI pattern
-
map:PaperClassMap a d2rq:ClassMap;
- d2rq:uriPattern "http://www.conference.org/conf02004/paper#Paper@@Papers.PaperID@@";
- d2rq:class :Paper;
- d2rq:classDefinitionLabel "paper"@en;
- d2rq:classDefinitionComment "A conference paper."@en;
- d2rq:dataStorage map:Database1.
-
The d2rq:class property is used to state that all resources generated by the
- d2rq:ClassMap are instances of an RDFS or OWL class. D2RQ automatically creates
- the necessary rdf:type triples.
-
-
Example: ClassMap where instances are identified using blank nodes
In order to recognize bNodes across several find() calls and to be able to
- map bNodes to instance data in the database, D2RQ encodes the classMap name
- together with the primary key values in the bNode label. The map above could
- produce the bNode label "http://www.example.org/dbserver01/db01#Topic@@6",
- where the number "6" is a primary key value and "http://www.example.org/dbserver01/db01#Topic"
- is the ClassMap name.
-
-
Example: ClassMap for a group of classes
- with the same properties
-
If you want to use one ClassMap for a group of classes with the same properties
- (like Person, Professor, Researcher, Student) that all come from the same table,
- you must create the rdf:type statements with an object property bridge instead of
- using d2rq:class.
Here, the class of each person is obtained by prefixing the values of the Persons.Type
- column with an ontology namespace. If the class names within the ontology can't
- be constructed directly from values of the Persons.Type column, then a TranslationTable
- could be used for aligning class names and database values.
-
-
7.3 Property Bridges
-
-
Property Bridges relate database table columns to RDF properties. They are
- used to attach properties to the RDF resources created by a class map.
- The values of these properties are often literals, but can also be URIs
- or blank nodes that relate the resource to other resources, e.g. the value
- of a paper's :author property could be a URI representing a person.
-
-
If the one of the columns used in a property bridge is NULL
- for some database rows, then no property is created for the resources
- corresponding to these rows.
-
-
Properties
-
-
-
-
d2rq:belongsToClassMap
-
Specifies that the property bridge belongs to a d2rq:ClassMap.
- Must be specified for every property bridge.
-
-
-
d2rq:property
-
The RDF property that connects the ClassMap
- with the object or literal created by the bridge. Must be specified for
- every property bridge. If multiple d2rq:properties are
- specified, then one triple with each property is generated per resource.
-
-
-
d2rq:dynamicProperty
-
A URI pattern that is used to generate the property URI at runtime. If multiple d2rq:dynamicProperty are specified, then one triple with each property is generated per resource.
-
-
-
d2rq:column
-
For properties with literal values.
- The database column that contains the literal values.
- Column names have to be given in the form "TableName.ColumnName".
-
-
-
-
d2rq:pattern
-
For properties with literal values.
- Can be used to extend and combine column values before
- they are used as a literal property value. If
- a pattern contains more than one column, then a separating string, which
- cannot occur in the column values, has to be used between the column names,
- in order to allow D2RQ reversing given literals into column values.
-
-
-
d2rq:sqlExpression
-
For properties with literal values. Generates
- literal values by evaluating a SQL expression. Note that querying
- for such a computed value might put a heavy load on the database.
- See example below.
-
-
-
d2rq:datatype
-
For properties with literal values. Specifies the RDF datatype of the literals.
-
-
-
d2rq:lang
-
For properties with literal values. Specifies the language tag of the literals.
-
-
-
d2rq:uriColumn
-
For properties with URI values. Database column that contains URIs.
- Column names
- have to be given in the form "TableName.ColumnName".
-
-
-
d2rq:uriPattern
-
For properties with URI values.
- Can be used to extend and combine column values before
- they are used as a URI property values. If
- a pattern contains more than one column, then a separating string, which
- cannot occur in the column values, has to be used between the column names,
- in order to allow D2RQ reversing given literals into column values.
- See example below.
-
+
+
7.2 ClassMap
+
A d2rq:ClassMap represents a class or a group of similar classes
+ of an OWL ontology or RDFS schema. A class map defines how instances of the
+ class are identified. It is connected to a d2rq:Database and has
+ a set of d2rq:PropertyBridges which attach properties to the instances.
+
+
7.2.1 Resource Identity
+
+
D2RQ provides four different mechanisms of assigning identifiers to the instances
+ in the database:
+
+
+
URI patterns
+
A URI pattern is instantiated by inserting values of
+ certain database columns into a pattern. Examples:
The parts between @@'s mark database columns in
+ Table.Column notation. URI patterns are used
+ with the d2rq:uriPattern property.
+
Certain characters, like spaces or the hash sign, are not allowed
+ in URIs or have special meaning. Columns that contain such characters
+ need to be encoded before their values can be inserted into a URI
+ pattern:
+
+
If a column is given as @@Table.Column|urlencode@@,
+ then URL encoding
+ is applied before the string is inserted.
+
If a column is given as @@Table.Column|urlify@@,
+ then URL encoding is applied, with an additional rule that spaces are
+ converted to underscores (_). Some people find that this
+ produces friendlier URIs.
+
+
+
+
Relative URI patterns
+
A relative URI pattern is a URI pattern that generates
+ relative URIs:
+
persons/@@Persons.ID@@
+
They will be combined with a base URI provided by
+ the processing environment to form full URIs. Relative
+ URI patterns allow the creation of portable mappings that
+ can be used for multiple instances of the same database
+ schema. Relative URI patterns are generated
+ with the d2rq:uriPattern property.
+
+
+
URI columns
+
In some cases, the database may already contain URIs that
+ can be used as resource identifiers, such as web page and
+ document URLs. URI are generated from columns
+ with the d2rq:uriColumns property.
+
+
+
Blank nodes
+
RDF also has the concept of blank nodes, existential qualifiers
+ that denote some resource that exists and has certain properties,
+ but is not named. In D2RQ, blank nodes can be generated from one or
+ more columns. A distinct blank node will be generated for each distinct
+ set of values of these columns. The columns are specified using the
+ with the d2rq:bNodeIdColumns property.
+
+
+
+
Singleton classmaps
+
A d2rq:ClassMap usually produces many resources. Sometimes
+ it is desirable to have a class map that only produces a single resource
+ with fixed, static identity. In that case, one can use the
+ d2rq:constantValue property to name the single instance.
+
+
Properties of d2rq:ClassMap
+
+
+
d2rq:dataStorage
+
Reference to a d2rq:Database where the instance data is stored.
+
+
+
d2rq:class
+
An RDF-S or OWL class. All resources generated by this
+ ClassMap are instances of this class.
+
+
+
d2rq:uriPattern
+
Specifies a URI pattern that will be used to
+ identify instances of this class map.
+
+
+
d2rq:uriColumn
+
A database column containing URIrefs for identifying
+ instances of this class map. The column name has to be in the form
+ "TableName.ColumnName".
+
+
+
d2rq:bNodeIdColumns
+
A comma-seperated list of column names in
+ "TableName.ColumnName" notation. The instances of
+ this class map will be blank nodes, one distinct blank node
+ per distinct tuple of these columns.
+
+
+
d2rq:constantValue
+
This class map will only have a single instance, which is
+ named by the value of this property. This can be a blank node or a URI.
+
+
+
d2rq:translateWith
+
Assigns a d2rq:TranslationTable to the class map. Values
+ from the d2rq:uriColumn or d2rq:uriPattern will be translated by the table
+ before a resource is generated. See below
+ for details.
+
+
+
d2rq:containsDuplicates
+
Must be specified if a class map uses information from tables
+ that are not fully normalized. If the d2rq:containsDuplicates property value
+ is set to "true", then D2RQ adds a DISTINCT clause to all queries using
+ this classMap. "False" is the default value, which doesn't have
+ to be explicitly declared. Adding this property to class maps based on normalized
+ database tables degrades query performance, but doesn't affect query results.
+
+
+
d2rq:additionalProperty
+
Adds an AdditionalProperty
+ to all instances of this class.
+ This might be useful for adding rdfs:seeAlso properties or other fixed
+ statements to all instances of the class.
+
+
+
d2rq:condition
+
Specifies an SQL WHERE condition. An instance of this class
+ will only be generated for database rows that satisfy the condition. Conditions
+ can be used to hide parts of the database from D2RQ, e.g. deny access to
+ data which is older or newer than a certain date. See section Conditional
+ Mappings for details.
+
+
+
d2rq:classDefinitionLabel
+
Specifies a label that will be served as rdfs:label
+ for all associated class definitions. Multiple lables, e.g. in several languages, are supported.
+
+
+
d2rq:classDefinitionComment
+
Specifies a comment that will be served as rdfs:comment
+ for all associated class definitions. Multiple comments are supported.
Inverse of d2rq:class and unnecessary if d2rq:class is used. Specifies
+ that a d2rq:classMap is used to create instances of an OWL or RDF-S class.
+
+
+
+
Example: ClassMap where instances are identified using an URI pattern
+
map:PaperClassMap a d2rq:ClassMap;
+ d2rq:uriPattern "http://www.conference.org/conf02004/paper#Paper@@Papers.PaperID@@";
+ d2rq:class :Paper;
+ d2rq:classDefinitionLabel "paper"@en;
+ d2rq:classDefinitionComment "A conference paper."@en;
+ d2rq:dataStorage map:Database1.
+
The d2rq:class property is used to state that all resources generated by the
+ d2rq:ClassMap are instances of an RDFS or OWL class. D2RQ automatically creates
+ the necessary rdf:type triples.
+
+
Example: ClassMap where instances are identified using blank nodes
In order to recognize bNodes across several find() calls and to be able to
+ map bNodes to instance data in the database, D2RQ encodes the classMap name
+ together with the primary key values in the bNode label. The map above could
+ produce the bNode label "http://www.example.org/dbserver01/db01#Topic@@6",
+ where the number "6" is a primary key value and "http://www.example.org/dbserver01/db01#Topic"
+ is the ClassMap name.
+
+
Example: ClassMap for a group of classes
+ with the same properties
+
If you want to use one ClassMap for a group of classes with the same properties
+ (like Person, Professor, Researcher, Student) that all come from the same table,
+ you must create the rdf:type statements with an object property bridge instead of
+ using d2rq:class.
Here, the class of each person is obtained by prefixing the values of the Persons.Type
+ column with an ontology namespace. If the class names within the ontology can't
+ be constructed directly from values of the Persons.Type column, then a TranslationTable
+ could be used for aligning class names and database values.
+
+
7.3 Property Bridges
+
+
Property Bridges relate database table columns to RDF properties. They are
+ used to attach properties to the RDF resources created by a class map.
+ The values of these properties are often literals, but can also be URIs
+ or blank nodes that relate the resource to other resources, e.g. the value
+ of a paper's :author property could be a URI representing a person.
+
+
If the one of the columns used in a property bridge is NULL
+ for some database rows, then no property is created for the resources
+ corresponding to these rows.
+
+
Properties
+
+
+
+
d2rq:belongsToClassMap
+
Specifies that the property bridge belongs to a d2rq:ClassMap.
+ Must be specified for every property bridge.
+
+
+
d2rq:property
+
The RDF property that connects the ClassMap
+ with the object or literal created by the bridge. Must be specified for
+ every property bridge. If multiple d2rq:properties are
+ specified, then one triple with each property is generated per resource.
+
+
+
d2rq:dynamicProperty
+
A URI pattern that is used to generate the property URI at runtime. If multiple d2rq:dynamicProperty are specified, then one triple with each property is generated per resource.
+
+
+
d2rq:column
+
For properties with literal values.
+ The database column that contains the literal values.
+ Column names have to be given in the form "TableName.ColumnName".
+
+
+
+
d2rq:pattern
+
For properties with literal values.
+ Can be used to extend and combine column values before
+ they are used as a literal property value. If
+ a pattern contains more than one column, then a separating string, which
+ cannot occur in the column values, has to be used between the column names,
+ in order to allow D2RQ reversing given literals into column values.
+
+
+
d2rq:sqlExpression
+
For properties with literal values. Generates
+ literal values by evaluating a SQL expression. Note that querying
+ for such a computed value might put a heavy load on the database.
+ See example below.
+
+
+
d2rq:datatype
+
For properties with literal values. Specifies the RDF datatype of the literals.
+
+
+
d2rq:lang
+
For properties with literal values. Specifies the language tag of the literals.
+
+
+
d2rq:uriColumn
+
For properties with URI values. Database column that contains URIs.
+ Column names
+ have to be given in the form "TableName.ColumnName".
+
+
+
d2rq:uriPattern
+
For properties with URI values.
+ Can be used to extend and combine column values before
+ they are used as a URI property values. If
+ a pattern contains more than one column, then a separating string, which
+ cannot occur in the column values, has to be used between the column names,
+ in order to allow D2RQ reversing given literals into column values.
+ See example below.
+
d2rq:uriSqlExpression
For properties with URI values and similar to d2rq:sqlExpression. Generates
@@ -1134,184 +1134,184 @@
7.3 Property Bridges
for such a computed value might put a heavy load on the database.
See example below.
-
-
d2rq:refersToClassMap
-
For properties that correspond to a foreign key.
- References another
- d2rq:ClassMap that creates the instances which are used
- as the values of this bridge. One or more d2rq:join properties
- must be specified to select the correct instances. See
- example below.
-
-
-
d2rq:constantValue
-
For properties that have the same constant value on all
- instances of the class map. The value can be a literal, blank node, or
- URI. See example below.
-
-
-
d2rq:join
-
If the columns used to create the literal value or object
- are not from the database table(s) that contains the ClassMap's columns,
- then the tables have to be joined together using one or more d2rq:join
- properties.
- See example below.
-
-
-
d2rq:alias
-
Aliases take the form "Table AS Alias"
- and are used when a table needs to be joined to itself.
- The table can be referred to using the alias within the
- property bridge. See example below.
-
-
-
d2rq:condition
-
Specifies an SQL WHERE condition. The bridge will only generate
- a statement if the condition holds. A common usage is to suppress triples
- with empty literal values: d2rq:condition "Table.Column <> ''".
- See section Conditional Mappings for details.
-
-
-
-
d2rq:translateWith
-
Assigns a d2rq:TranslationTable to the property bridge. Values
- from the d2rq:column or d2rq:pattern will be translated by the table. See
- section TranslationTables for details.
-
-
-
d2rq:valueMaxLength
-
Asserts that all values of this bridge are not longer than
- a number of characters. This allows D2RQ to speed up queries. See section
- Performance Optimization for details.
-
-
-
d2rq:valueContains
-
Asserts that all values of this bridge always contain a given
- string. This allows D2RQ to speed up queries. Most useful in conjunction
- with d2rq:column. See section Performance Optimization
- for details.
-
-
-
d2rq:valueRegex
-
Asserts that all values of this bridge match a given regular
- expression. This allows D2RQ to speed up queries. Most useful in conjunction
- with d2rq:column on columns whose values are very different from other columns
- in the database. See section Performance Optimization
- for details.
-
-
-
d2rq:propertyDefinitionLabel
-
Specifies a label that will be served as rdfs:label
- for all associated property definitions. Multiple lables, e.g. in several languages, are supported.
-
-
-
d2rq:propertyDefinitionComment
-
Specifies a comment that will be served as rdfs:comment
- for all associated property definitions. Multiple comments are supported.
The maximum number of results to retrieve from the database for this PropertyBridge. Also see d2rq:resultSizeLimit.
-
-
-
d2rq:limitInverse
-
The maximum number of results to retrieve from the database for the inverse statements for this PropertyBridge.
-
-
-
d2rq:orderAsc
-
The column after which to sort results in ascending order for this PropertyBridge.
-
-
-
d2rq:orderDesc
-
The column after which to sort results in descending order for this PropertyBridge.
-
-
-
-
PropertyBridge property:
-
-
-
d2rq:propertyBridge
-
Inverse of d2rq:property and not needed if d2rq:property is used. The
- d2rq:propertyBridge property specifies which property bridge is used for
- an RDF property. If the same RDF property is used by several RDF classes,
- then several property bridges are used to relate the RDF property to the
- different class maps.
This attaches a :title property to all resources generated
- by the map:Paper class map. The property values are
- taken from the Papers.Title column. The generated literals
- will have a language tag of "en".
-
-
Example: Property bridge using information
- from another database table
This property bridge adds the names of authors to papers.
- If a paper has several authors, then several :authorName
- properties are added. The tables Papers
- and Persons are in an n:m relation. The d2rq:join
+
+
d2rq:refersToClassMap
+
For properties that correspond to a foreign key.
+ References another
+ d2rq:ClassMap that creates the instances which are used
+ as the values of this bridge. One or more d2rq:join properties
+ must be specified to select the correct instances. See
+ example below.
+
+
+
d2rq:constantValue
+
For properties that have the same constant value on all
+ instances of the class map. The value can be a literal, blank node, or
+ URI. See example below.
+
+
+
d2rq:join
+
If the columns used to create the literal value or object
+ are not from the database table(s) that contains the ClassMap's columns,
+ then the tables have to be joined together using one or more d2rq:join
+ properties.
+ See example below.
+
+
+
d2rq:alias
+
Aliases take the form "Table AS Alias"
+ and are used when a table needs to be joined to itself.
+ The table can be referred to using the alias within the
+ property bridge. See example below.
+
+
+
d2rq:condition
+
Specifies an SQL WHERE condition. The bridge will only generate
+ a statement if the condition holds. A common usage is to suppress triples
+ with empty literal values: d2rq:condition "Table.Column <> ''".
+ See section Conditional Mappings for details.
+
+
+
+
d2rq:translateWith
+
Assigns a d2rq:TranslationTable to the property bridge. Values
+ from the d2rq:column or d2rq:pattern will be translated by the table. See
+ section TranslationTables for details.
+
+
+
d2rq:valueMaxLength
+
Asserts that all values of this bridge are not longer than
+ a number of characters. This allows D2RQ to speed up queries. See section
+ Performance Optimization for details.
+
+
+
d2rq:valueContains
+
Asserts that all values of this bridge always contain a given
+ string. This allows D2RQ to speed up queries. Most useful in conjunction
+ with d2rq:column. See section Performance Optimization
+ for details.
+
+
+
d2rq:valueRegex
+
Asserts that all values of this bridge match a given regular
+ expression. This allows D2RQ to speed up queries. Most useful in conjunction
+ with d2rq:column on columns whose values are very different from other columns
+ in the database. See section Performance Optimization
+ for details.
+
+
+
d2rq:propertyDefinitionLabel
+
Specifies a label that will be served as rdfs:label
+ for all associated property definitions. Multiple lables, e.g. in several languages, are supported.
+
+
+
d2rq:propertyDefinitionComment
+
Specifies a comment that will be served as rdfs:comment
+ for all associated property definitions. Multiple comments are supported.
The maximum number of results to retrieve from the database for this PropertyBridge. Also see d2rq:resultSizeLimit.
+
+
+
d2rq:limitInverse
+
The maximum number of results to retrieve from the database for the inverse statements for this PropertyBridge.
+
+
+
d2rq:orderAsc
+
The column after which to sort results in ascending order for this PropertyBridge.
+
+
+
d2rq:orderDesc
+
The column after which to sort results in descending order for this PropertyBridge.
+
+
+
+
PropertyBridge property:
+
+
+
d2rq:propertyBridge
+
Inverse of d2rq:property and not needed if d2rq:property is used. The
+ d2rq:propertyBridge property specifies which property bridge is used for
+ an RDF property. If the same RDF property is used by several RDF classes,
+ then several property bridges are used to relate the RDF property to the
+ different class maps.
This attaches a :title property to all resources generated
+ by the map:Paper class map. The property values are
+ taken from the Papers.Title column. The generated literals
+ will have a language tag of "en".
+
+
Example: Property bridge using information
+ from another database table
This property bridge adds the names of authors to papers.
+ If a paper has several authors, then several :authorName
+ properties are added. The tables Papers
+ and Persons are in an n:m relation. The d2rq:join
is used to join the tables over the Rel_Person_Paper. The join condition contains
directed arrows that indicate the foreign key relationship and are used as an optimization hint.
In the example above, the arrow directions indicate that all possible values of Rel_Person_Paper.PaperID and
Rel_Person_Paper.PersonID are present in Papers.PaperID and Persons.PerID, respectively.
Where this is unclear, a simple equation sign (=) may be used.
-
The pattern mailto:@@Persons.Email@@ is used to
- attach a mailto: prefix to the values of the
- "Persons.Email" column. The example uses
- d2rq:uriPattern instead of d2rq:pattern
- because the bridge should produce URIs, not literals.
-
-
Example: A property bridge that computes mailbox hashes
-
The popular FOAF vocabulary has a
- property foaf:mbox_sha1sum for publishing email addresses in an
- encoded form. This prevents spammers from harvesting the address, while still
- letting us recognize if the same email address is used in two different
- places.
The values of the foaf:mbox_sha1sum are computed by evaluating
- the d2rq:sqlExpression. We first create a mailto: URI
- from the email address, as required by FOAF. Then we apply the SHA1
- hash function, again as required by FOAF. The result will be a literal value.
-
Note that querying for a specific foaf:mbox_sha1sum value will
- put a heavy load on the database because the hash has to be computed for every
- user in the database. For a large database, it would be better to store the
- encoded values in a column in the database.
The pattern mailto:@@Persons.Email@@ is used to
+ attach a mailto: prefix to the values of the
+ "Persons.Email" column. The example uses
+ d2rq:uriPattern instead of d2rq:pattern
+ because the bridge should produce URIs, not literals.
+
+
Example: A property bridge that computes mailbox hashes
+
The popular FOAF vocabulary has a
+ property foaf:mbox_sha1sum for publishing email addresses in an
+ encoded form. This prevents spammers from harvesting the address, while still
+ letting us recognize if the same email address is used in two different
+ places.
The values of the foaf:mbox_sha1sum are computed by evaluating
+ the d2rq:sqlExpression. We first create a mailto: URI
+ from the email address, as required by FOAF. Then we apply the SHA1
+ hash function, again as required by FOAF. The result will be a literal value.
+
Note that querying for a specific foaf:mbox_sha1sum value will
+ put a heavy load on the database because the hash has to be computed for every
+ user in the database. For a large database, it would be better to store the
+ encoded values in a column in the database.
Example: A property bridge with URIs generated by an SQL expression
map:HomepageURL a d2rq:PropertyBridge;
@@ -1324,130 +1324,130 @@
7.3 Property Bridges
"Persons.Email" column. The example uses
d2rq:uriPattern instead of d2rq:pattern
because the bridge should produce URIs, not literals.
-
-
Example: Linking instances
- from two database tables
The example attaches a :conference property to papers.
- The values of the property are generated by the map:Conference
- class map, not shown here. It may use a d2rq:uriPattern,
- d2rq:uriColumn or blank nodes to identify the conferences.
- The appropriate instance is found using the d2rq:join
- on the foreign key Papers.Conference.
-
-
Example: Joining a table to itself using d2rq:alias
Here, a topic may have a parent topic whose ID is found in
- the Topics.ParentID column. This foreign key refers
- back to the Topics.ID column. The table has to be
- joined to itself. A d2rq:alias is declared, and
- the join is established between the original table and the
- aliased table. This pattern is typical for hierarchical
- or graph-style relationships.
-
-
Example: Adding a constant property-value pair to each instance of a class map
-
-
Sometimes it is desirable to add a property with a constant value
- to every resource that is created by a class map. To achieve this,
- use a d2rq:PropertyBridge that uses d2rq:constantValue:
The example attaches a :conference property to papers.
+ The values of the property are generated by the map:Conference
+ class map, not shown here. It may use a d2rq:uriPattern,
+ d2rq:uriColumn or blank nodes to identify the conferences.
+ The appropriate instance is found using the d2rq:join
+ on the foreign key Papers.Conference.
+
+
Example: Joining a table to itself using d2rq:alias
Here, a topic may have a parent topic whose ID is found in
+ the Topics.ParentID column. This foreign key refers
+ back to the Topics.ID column. The table has to be
+ joined to itself. A d2rq:alias is declared, and
+ the join is established between the original table and the
+ aliased table. This pattern is typical for hierarchical
+ or graph-style relationships.
+
+
Example: Adding a constant property-value pair to each instance of a class map
+
+
Sometimes it is desirable to add a property with a constant value
+ to every resource that is created by a class map. To achieve this,
+ use a d2rq:PropertyBridge that uses d2rq:constantValue:
This adds an rdfs:seeAlso statement with a fixed URL object
+ to every instance of the map:PersonsClassMap class map.
+
7.4 Translation Tables
-
A d2rq:TranslationTable is an additional layer between the
- database and the RDF world. It translates back and forth between values taken
- from the database and RDF URIs or literals. A translation table can be attached
- to a class map or a property bridge using the d2rq:translateWith property. TranslationTables
- can be used only for mappings that are unique in both directions (1:1).
Links to a CSV file containing translations. Each line of
- the file is a translation and contains two strings separated by a comma.
- The first one is the DB value, the second the RDF value.
-
-
-
d2rq:javaClass
-
The qualified name of a Java class that performs the mapping.
- The class must implement the Translator
- interface. Custom Translators might be useful for encoding and decoding
- values, but are limited to 1:1 translations. Further datails can be found
- in the D2RQ javadocs.
-
-
-
+
A d2rq:TranslationTable is an additional layer between the
+ database and the RDF world. It translates back and forth between values taken
+ from the database and RDF URIs or literals. A translation table can be attached
+ to a class map or a property bridge using the d2rq:translateWith property. TranslationTables
+ can be used only for mappings that are unique in both directions (1:1).
Links to a CSV file containing translations. Each line of
+ the file is a translation and contains two strings separated by a comma.
+ The first one is the DB value, the second the RDF value.
+
+
+
d2rq:javaClass
+
The qualified name of a Java class that performs the mapping.
+ The class must implement the Translator
+ interface. Custom Translators might be useful for encoding and decoding
+ values, but are limited to 1:1 translations. Further datails can be found
+ in the D2RQ javadocs.
+
+
+
7.4.1 Translation
-
A d2rq:Translation is a single entry in a
- d2rq:TranslationTable.
-
-
Properties
-
-
-
d2rq:databaseValue
-
A value that might appear in a database column
- or might be generated by a d2rq:pattern.
-
-
-
d2rq:rdfValue
-
A translation of that value to be used in RDF constructs.
-
-
-
Example: Translating color codes
-
A typical application are database columns containing type codes or similar
- enumerated values. A translation table can be used to turn them into RDF resources.
- In this example, the column ShinyObject.Color contains a color code: "R" for
- red, "G" for green etc. These codes must be translated into RDF resources: :red,
- :green etc.
-
:red a :Color;
-:green a :Color;
-# ... more colors omitted ...
-:blue a :Color;
-
-map:ColorBridge a d2rq:PropertyBridge;
- d2rq:belongsToClassMap map:ShinyObjectMap;
- d2rq:property :color;
- d2rq:uriColumn "ShinyObject.Color";
- d2rq:translateWith map:ColorTable.
-
-map:ColorTable a d2rq:TranslationTable;
- d2rq:translation [ d2rq:databaseValue "R"; d2rq:rdfValue :red; ];
- d2rq:translation [ d2rq:databaseValue "G"; d2rq:rdfValue :green; ];
- # ... more translations omitted ...
- d2rq:translation [ d2rq:databaseValue "B"; d2rq:rdfValue :blue; ].
-
The d2rq:translateWith statement tells D2RQ to look up database values in the
- map:ColorTable. There, a translation must be given for each possible value.
- If the database contains values which are not in the translation table, D2RQ
- will not generate a :color statement for that :ShinyObject instance.
-
Note that the type of the resulting RDF node is determined by the bridge
- and not by the node type of the rdfValues. map:ColorBridge uses d2rq:uriColumn.
- Thus, the translation will create URI nodes. If it used d2rq:column,
- then literals would be created.
-
+
A d2rq:Translation is a single entry in a
+ d2rq:TranslationTable.
+
+
Properties
+
+
+
d2rq:databaseValue
+
A value that might appear in a database column
+ or might be generated by a d2rq:pattern.
+
+
+
d2rq:rdfValue
+
A translation of that value to be used in RDF constructs.
+
+
+
Example: Translating color codes
+
A typical application are database columns containing type codes or similar
+ enumerated values. A translation table can be used to turn them into RDF resources.
+ In this example, the column ShinyObject.Color contains a color code: "R" for
+ red, "G" for green etc. These codes must be translated into RDF resources: :red,
+ :green etc.
+
:red a :Color;
+:green a :Color;
+# ... more colors omitted ...
+:blue a :Color;
+
+map:ColorBridge a d2rq:PropertyBridge;
+ d2rq:belongsToClassMap map:ShinyObjectMap;
+ d2rq:property :color;
+ d2rq:uriColumn "ShinyObject.Color";
+ d2rq:translateWith map:ColorTable.
+
+map:ColorTable a d2rq:TranslationTable;
+ d2rq:translation [ d2rq:databaseValue "R"; d2rq:rdfValue :red; ];
+ d2rq:translation [ d2rq:databaseValue "G"; d2rq:rdfValue :green; ];
+ # ... more translations omitted ...
+ d2rq:translation [ d2rq:databaseValue "B"; d2rq:rdfValue :blue; ].
+
The d2rq:translateWith statement tells D2RQ to look up database values in the
+ map:ColorTable. There, a translation must be given for each possible value.
+ If the database contains values which are not in the translation table, D2RQ
+ will not generate a :color statement for that :ShinyObject instance.
+
Note that the type of the resulting RDF node is determined by the bridge
+ and not by the node type of the rdfValues. map:ColorBridge uses d2rq:uriColumn.
+ Thus, the translation will create URI nodes. If it used d2rq:column,
+ then literals would be created.
+
7.5 Configuration
A d2rq:Configuration controls global behaviour of D2RQ. It is generally not required if the defaults are satisfactory.
Properties
@@ -1466,66 +1466,66 @@
7.5 Configuration
map:Configuration a d2rq:Configuration;
d2rq:useAllOptimizations true.
-
7.6 Conditional Mappings
-
Sometimes only certain information from a database should be accessible, because
- parts of the database might be confidential or outdated. Using d2rq:condition
- you can specify conditions, which must be satisfied by all accessible data.
-
-
You can use d2rq:condition on class map and property bridge level. The d2rq:condition
- value is added as an additional SQL WHERE clause to all queries generated using
- the class map or property bridge. If the condition evaluates to FALSE for a
- SQL result set row, then no triples will be generated from that row.
Only those papers with a value of 1 in the Papers.Publish column will be accessible.
- All other papers are ignored.
-
Example: Filtering zero-length strings
-
Usually, the special value NULL is used in a database to indicate that some
- field has no value, or that the value is unknown. Some databases, however, are
- using a zero-length string ("") instead. D2RQ doesn't generate RDF statements
- from NULL values, but it doesn't recognize zero-length strings and will generate
- statements like :Person123 :firstName "". if the person's first name
- is a zero-length string. In oder to suppress these statements, a d2rq:condition
- can be added to the property bridge:
Imagine a table Rel_Paper_Topic that associates rows from a Papers table
- with rows from a Topics table. The Rel_Paper_Topic table contains a
- PaperID column to reference the papers, a TopicID to reference the topics,
- and a RelationshipType column which contains 1 if the topic is a primary
- topic of the paper, and 2 if it is a secondary topic.
-
For primary topic relationships, the :primaryTopic property shall be used,
- and for others the :secondaryTopic property.
-
We can build a map for this scenario by creating two property bridges.
- One for :primaryTopic, one for :secondaryTopic. We add a d2rq:condition
- to both bridges to suppress those statements where the RelationshipType
- column doesn't have the correct value.
Sometimes only certain information from a database should be accessible, because
+ parts of the database might be confidential or outdated. Using d2rq:condition
+ you can specify conditions, which must be satisfied by all accessible data.
+
+
You can use d2rq:condition on class map and property bridge level. The d2rq:condition
+ value is added as an additional SQL WHERE clause to all queries generated using
+ the class map or property bridge. If the condition evaluates to FALSE for a
+ SQL result set row, then no triples will be generated from that row.
Only those papers with a value of 1 in the Papers.Publish column will be accessible.
+ All other papers are ignored.
+
Example: Filtering zero-length strings
+
Usually, the special value NULL is used in a database to indicate that some
+ field has no value, or that the value is unknown. Some databases, however, are
+ using a zero-length string ("") instead. D2RQ doesn't generate RDF statements
+ from NULL values, but it doesn't recognize zero-length strings and will generate
+ statements like :Person123 :firstName "". if the person's first name
+ is a zero-length string. In oder to suppress these statements, a d2rq:condition
+ can be added to the property bridge:
Imagine a table Rel_Paper_Topic that associates rows from a Papers table
+ with rows from a Topics table. The Rel_Paper_Topic table contains a
+ PaperID column to reference the papers, a TopicID to reference the topics,
+ and a RelationshipType column which contains 1 if the topic is a primary
+ topic of the paper, and 2 if it is a secondary topic.
+
For primary topic relationships, the :primaryTopic property shall be used,
+ and for others the :secondaryTopic property.
+
We can build a map for this scenario by creating two property bridges.
+ One for :primaryTopic, one for :secondaryTopic. We add a d2rq:condition
+ to both bridges to suppress those statements where the RelationshipType
+ column doesn't have the correct value.
In the spirit of Linked Data, vocabulary data should be dereferencable by clients. D2RQ infers types of class and property resources as rdfs:Class and rdf:Property and allows the user to provide labels
@@ -1593,105 +1593,105 @@
7.7.3 Controlling vocabulary serving
map:Configuration a d2rq:Configuration;
d2rq:serveVocabulary false.
-
7.8 Performance Optimization using Hint Properties
-
This section covers hint properties that can be added to property bridges in
- order to speed up queries: d2rq:valueMaxLength, d2rq:valueRegex and d2rq:valueContains.
The d2rq:valueMaxLength property can be used to tell D2RQ that the length of
- Persons.FirstName values is limited to 15 characters. Using this information,
- D2RQ doesn't have to look in the database anymore to figure out, that a given
- FirstName which is longer than 15 characters isn't fitting.
-
Here, the d2rq:valueRegex property is used to provide a regular
- expression for the Papers.Year column. The statement asserts that all values
- match the regular expression (or are NULL). The expression ^[0-9]{4}$
- matches every four-digit number. If you don't want to use the full regular expression
- machinery, you can use d2rq:valueContains to assert that all values generated
- by the property bridge contain a certain phrase.
-
-
You are geting the largest performance gain by providing hints for property
- bridges which are using d2rq:column. You should define hints on columns of large
- tables and on columns that are not indexed by the database. These are the cases
- where a well-placed optimization hint can result in an order-of-magnitude improvement
- for some queries. Don't bother to provide hints for property bridges based on
- d2rq:pattern. These can be optimized very well without hints. In general,
- the biggest payoff is expected for hints on large tables. If you have a few
- very large tables with non-indexed columns in your database, that's where you
- should focus your efforts.
-
-
Please keep in mind that hint properties are not intended for filtering of
- unwanted database values. They are only performance hints. Values that do not
- fulfill the criteria will still appear in query results if find patterns like
- (URI, ANY, ANY) are used. In oder to filter values, use d2rq:condition
- or a translation table with a custom Java class
- that returns null for unwanted database values.
-
-
7.9 Deprecated Language Constructs
-
-
This section lists several language constructs from older versions
-of the D2RQ mapping language that have been replaced by better
-alternatives and should no longer be used.
-
-
7.9.1 d2rq:DatatypePropertyBridge and d2rq:ObjectPropertyBridge
-
-
Older versions of the language used two different classes to
-distinguish between property bridges that produce literals, and
-bridges that produce resources.
-
-
In the current version, both cases
-are handled by the d2rq:PropertyBridge class. The distinction
-is made by using an appropriate property on the bridge declaration:
-d2rq:column and d2rq:pattern for literals,
-d2rq:uriColumn, d2rq:uriPattern and
-d2rq:bNodeIdColumns for resources.
-
-
7.9.2 d2rq:additionalProperty
-
Up until D2RQ 0.5.1, the d2rq:AdditionalProperty construct could be used
- to add a constant property-value pairs to all instances of a class map. An example
- is shown below:
-
map:PersonsClassMap a d2rq:ClassMap;
- d2rq:class :Person;
- d2rq:additionalProperty map:SeeAlsoStatement.
-
-map:SeeAlsoStatement a d2rq:AdditionalProperty;
- d2rq:propertyName rdfs:seeAlso;
- d2rq:propertyValue <http://annotation.semanticweb.org/iswc2003/>.
-
This adds an rdfs:seeAlso statement with a fixed URL object to every instance of
- the persons class map. In recent versions of the mapping language, the same is
- achieved by adding a property bridge to the class map, and giving it a
- d2rq:constantValue property with the fixed URL as the object, as
- shown in this example.
-
-
d2rq:AdditionalProperty constructs are still used with d2rq:additionalClassDefinitionProperty
-and d2rq:additionalPropertyDefinitionProperty (see section 7.4.1).
-
-
-
-
8. Change Log
-
-
2009-08-10: Updated for D2RQ v0.7 and D2R Server v0.7.
-
2009-02-19: Updated for D2RQ v0.6 and D2R Server v0.6.
-
2007-10-25: Updated for D2RQ v0.5.1 and D2R Server v0.4.
-
2006-10-26: Updated for D2RQ v0.5.
-
2005-10-21: Updated for D2RQ v0.4.
-
2005-04-29: Updated for D2RQ v0.3.
-
2004-08-03: Updated reflecting changes in D2RQ v0.2.
7.8 Performance Optimization using Hint Properties
+
This section covers hint properties that can be added to property bridges in
+ order to speed up queries: d2rq:valueMaxLength, d2rq:valueRegex and d2rq:valueContains.
The d2rq:valueMaxLength property can be used to tell D2RQ that the length of
+ Persons.FirstName values is limited to 15 characters. Using this information,
+ D2RQ doesn't have to look in the database anymore to figure out, that a given
+ FirstName which is longer than 15 characters isn't fitting.
+
Here, the d2rq:valueRegex property is used to provide a regular
+ expression for the Papers.Year column. The statement asserts that all values
+ match the regular expression (or are NULL). The expression ^[0-9]{4}$
+ matches every four-digit number. If you don't want to use the full regular expression
+ machinery, you can use d2rq:valueContains to assert that all values generated
+ by the property bridge contain a certain phrase.
+
+
You are geting the largest performance gain by providing hints for property
+ bridges which are using d2rq:column. You should define hints on columns of large
+ tables and on columns that are not indexed by the database. These are the cases
+ where a well-placed optimization hint can result in an order-of-magnitude improvement
+ for some queries. Don't bother to provide hints for property bridges based on
+ d2rq:pattern. These can be optimized very well without hints. In general,
+ the biggest payoff is expected for hints on large tables. If you have a few
+ very large tables with non-indexed columns in your database, that's where you
+ should focus your efforts.
+
+
Please keep in mind that hint properties are not intended for filtering of
+ unwanted database values. They are only performance hints. Values that do not
+ fulfill the criteria will still appear in query results if find patterns like
+ (URI, ANY, ANY) are used. In oder to filter values, use d2rq:condition
+ or a translation table with a custom Java class
+ that returns null for unwanted database values.
+
+
7.9 Deprecated Language Constructs
+
+
This section lists several language constructs from older versions
+of the D2RQ mapping language that have been replaced by better
+alternatives and should no longer be used.
+
+
7.9.1 d2rq:DatatypePropertyBridge and d2rq:ObjectPropertyBridge
+
+
Older versions of the language used two different classes to
+distinguish between property bridges that produce literals, and
+bridges that produce resources.
+
+
In the current version, both cases
+are handled by the d2rq:PropertyBridge class. The distinction
+is made by using an appropriate property on the bridge declaration:
+d2rq:column and d2rq:pattern for literals,
+d2rq:uriColumn, d2rq:uriPattern and
+d2rq:bNodeIdColumns for resources.
+
+
7.9.2 d2rq:additionalProperty
+
Up until D2RQ 0.5.1, the d2rq:AdditionalProperty construct could be used
+ to add a constant property-value pairs to all instances of a class map. An example
+ is shown below:
+
map:PersonsClassMap a d2rq:ClassMap;
+ d2rq:class :Person;
+ d2rq:additionalProperty map:SeeAlsoStatement.
+
+map:SeeAlsoStatement a d2rq:AdditionalProperty;
+ d2rq:propertyName rdfs:seeAlso;
+ d2rq:propertyValue <http://annotation.semanticweb.org/iswc2003/>.
+
This adds an rdfs:seeAlso statement with a fixed URL object to every instance of
+ the persons class map. In recent versions of the mapping language, the same is
+ achieved by adding a property bridge to the class map, and giving it a
+ d2rq:constantValue property with the fixed URL as the object, as
+ shown in this example.
+
+
d2rq:AdditionalProperty constructs are still used with d2rq:additionalClassDefinitionProperty
+and d2rq:additionalPropertyDefinitionProperty (see section 7.4.1).
+
+
+
+
8. Change Log
+
+
2009-08-10: Updated for D2RQ v0.7 and D2R Server v0.7.
+
2009-02-19: Updated for D2RQ v0.6 and D2R Server v0.6.
+
2007-10-25: Updated for D2RQ v0.5.1 and D2R Server v0.4.
+
2006-10-26: Updated for D2RQ v0.5.
+
2005-10-21: Updated for D2RQ v0.4.
+
2005-04-29: Updated for D2RQ v0.3.
+
2004-08-03: Updated reflecting changes in D2RQ v0.2.
D2R Server is a tool for publishing the content of relational databases on the Semantic Web. This document details how three different kinds of Web agents can access this data through simple HTTP-based interfaces: RDF browsers, traditional HTML browsers, and SPARQL query clients.
The Semantic Web is a global information space consisting of inter-linked data about resources. There are two access paradigms to the Semantic Web: browsing and searching. Using a Semantic Web browser like Tabulator (slides), a surfer can follow links from data about one resource to data about other resources. The second access paradigm is searching. Using the SPARQL query language and protocol, a client application can query data-sources for information about resources.
-
D2R Server is a tool for publishing the content of relational databases on the Semantic Web. Database content is mapped to RDF by a customizable mapping which specifies how resources are identified and which properties are used to describe resources. Based on this mapping, D2R Server allows a RDF representation of the database to be browsed and searched. The server provides two interfaces: The dereferencing interface allows instance and vocabulary URIs to be dereferenced over the HTTP protocol. The interface supports content-negotiation and serves RDF and XHTML representations of resources. The generated representations are richly interlinked on RDF and XHTML level in order to enable browsers and crawlers to navigate database content. The SPARQL interface enables applications to query the database using the SPARQL query language over the SPARQL protocol.
-
-
-
-
The server takes requests from the Web and rewrites them via a D2RQ mapping into SQL queries against a relational database. This on-the-fly translation allows clients to access the content of large databases without having to replicate them into RDF.
-
-
2. Access with an RDF Browser
-
-
RDF browsers, like Tabulator or Disco, retrieve data from the Semantic Web by dereferencing instance and vocabulary URIs and by following rdfs:seeAlso links within the data. The screenshot below shows an example database being browsed with the Tabulator RDF browser.
D2R Server allows database-generated URIs to be dereferenced. The HTTP request below, sent to the server http://www3.wiwiss.fu-berlin.de:2020, requests an RDF representation of the resource http://www3.wiwiss.fu-berlin.de:2020/resource/persons/6. Note that the request asks for content type application/rdf+xml.
-
-
GET /resource/persons/6 HTTP/1.0
-Accept: application/rdf+xml
-
-
According to the httpRange-14 TAG finding, only information resources (i.e. documents) can
- have representations served on the Web over HTTP. When
- URIs that identify other kinds of resources, such as a person,
- are dereferenced, then the HTTP response must be a 303
- redirect to a second URI. At that location, a document describing
- the real-world resource (i.e. person) is served. D2R Server implements this
- behavior and will answer the request above with an HTTP response like this:
-
-
HTTP/1.1 303 See Other
-Location: http://www3.wiwiss.fu-berlin.de:2020/data/persons/6
-Connection: close
-
-
The client has to perform a second HTTP GET request on the
- Location URI. D2R Server will respond now with an RDF/XML document
- containing an RDF/XML description of the person:
The description is generated on the fly from the content of the database.
- Note that the response contains URIs of related resources such as papers
- and topics. Descriptions of these can be retrieved in the same way.
- Beside of triples having resource/persons/6 as subject (out-arcs), the representation also contains triples having resource/persons/6 as object (in-arcs). Within our example, this enables RDF browsers and crawlers to follow the link from Andy to his paper resource/papers/4.
-
Future versions of D2R Server should also provide rdf:type and rdfs:label statements
- for each referenced resource. This would leave a breadcrumb trail to help browsers decide
- which links to follow.
-
-
-
2.2 Dereferencing External URIs
-
-
The database may also contain information about resources whose URIs are outside the server's namespace. When the server generates output that mentions such a resource, it adds an rdfs:seeAlso statement to the resource pointing at an RDF/XML document that contains all information from within the database about the external resource. By dereferencing the external URI and by following the rdfs:seeAlso link, an RDF browser can retrieve both authoritative as well as non-authoritative information about the resource.
-
2.3 Referring to Database Content from other Web Documents
-
You can use D2R Server's database-generated URIs to refer to database content from other Web documents. For instance, you could use the URI http://www3.wiwiss.fu-berlin.de:2020/resource/persons/6 in a foaf:knows statement within your FOAF profile to refer to Andy Seaborne. By dereferencing the URI, an RDF browser can navigate from your FOAF profile to information about Andy in the database.
D2R Server also offers a traditional HTML interface to the data. Each resource has an XHTML representation. Web browsers retrieve these representations by dereferencing the resource URI with an HTTP request that asks for HTML (content type text/html) or XHTML (application/xhtml+xml).
-
-
GET /resource/persons/4 HTTP/1.0
-Accept: text/html
-
-
Like in the application/rdf+xml case, D2R Server will redirect to a document describing the resource, but this time an XHTML page:
-
-
-
-
-
The representation contains navigation links (Home | All Persons) that allow the complete content of the database to be browsed.
-
The <head> section of the HTML page contains a <link rel="alternate" /> tag pointing to the resource's RDF representation. This allows tools like Piggy Bank to switch between the HTML and RDF views. The RDF icon in the corner links to the same RDF document.
-
All pages are rendered from Velocity templates to allow customization. Future version of D2R Server might employ Fresnel lenses to improve resource display.
D2R Server is a tool for publishing the content of relational databases on the Semantic Web. This document details how three different kinds of Web agents can access this data through simple HTTP-based interfaces: RDF browsers, traditional HTML browsers, and SPARQL query clients.
The Semantic Web is a global information space consisting of inter-linked data about resources. There are two access paradigms to the Semantic Web: browsing and searching. Using a Semantic Web browser like Tabulator (slides), a surfer can follow links from data about one resource to data about other resources. The second access paradigm is searching. Using the SPARQL query language and protocol, a client application can query data-sources for information about resources.
+
D2R Server is a tool for publishing the content of relational databases on the Semantic Web. Database content is mapped to RDF by a customizable mapping which specifies how resources are identified and which properties are used to describe resources. Based on this mapping, D2R Server allows a RDF representation of the database to be browsed and searched. The server provides two interfaces: The dereferencing interface allows instance and vocabulary URIs to be dereferenced over the HTTP protocol. The interface supports content-negotiation and serves RDF and XHTML representations of resources. The generated representations are richly interlinked on RDF and XHTML level in order to enable browsers and crawlers to navigate database content. The SPARQL interface enables applications to query the database using the SPARQL query language over the SPARQL protocol.
+
+
+
+
The server takes requests from the Web and rewrites them via a D2RQ mapping into SQL queries against a relational database. This on-the-fly translation allows clients to access the content of large databases without having to replicate them into RDF.
+
+
2. Access with an RDF Browser
+
+
RDF browsers, like Tabulator or Disco, retrieve data from the Semantic Web by dereferencing instance and vocabulary URIs and by following rdfs:seeAlso links within the data. The screenshot below shows an example database being browsed with the Tabulator RDF browser.
D2R Server allows database-generated URIs to be dereferenced. The HTTP request below, sent to the server http://www3.wiwiss.fu-berlin.de:2020, requests an RDF representation of the resource http://www3.wiwiss.fu-berlin.de:2020/resource/persons/6. Note that the request asks for content type application/rdf+xml.
+
+
GET /resource/persons/6 HTTP/1.0
+Accept: application/rdf+xml
+
+
According to the httpRange-14 TAG finding, only information resources (i.e. documents) can
+ have representations served on the Web over HTTP. When
+ URIs that identify other kinds of resources, such as a person,
+ are dereferenced, then the HTTP response must be a 303
+ redirect to a second URI. At that location, a document describing
+ the real-world resource (i.e. person) is served. D2R Server implements this
+ behavior and will answer the request above with an HTTP response like this:
+
+
HTTP/1.1 303 See Other
+Location: http://www3.wiwiss.fu-berlin.de:2020/data/persons/6
+Connection: close
+
+
The client has to perform a second HTTP GET request on the
+ Location URI. D2R Server will respond now with an RDF/XML document
+ containing an RDF/XML description of the person:
The description is generated on the fly from the content of the database.
+ Note that the response contains URIs of related resources such as papers
+ and topics. Descriptions of these can be retrieved in the same way.
+ Beside of triples having resource/persons/6 as subject (out-arcs), the representation also contains triples having resource/persons/6 as object (in-arcs). Within our example, this enables RDF browsers and crawlers to follow the link from Andy to his paper resource/papers/4.
+
Future versions of D2R Server should also provide rdf:type and rdfs:label statements
+ for each referenced resource. This would leave a breadcrumb trail to help browsers decide
+ which links to follow.
+
+
+
2.2 Dereferencing External URIs
+
+
The database may also contain information about resources whose URIs are outside the server's namespace. When the server generates output that mentions such a resource, it adds an rdfs:seeAlso statement to the resource pointing at an RDF/XML document that contains all information from within the database about the external resource. By dereferencing the external URI and by following the rdfs:seeAlso link, an RDF browser can retrieve both authoritative as well as non-authoritative information about the resource.
+
2.3 Referring to Database Content from other Web Documents
+
You can use D2R Server's database-generated URIs to refer to database content from other Web documents. For instance, you could use the URI http://www3.wiwiss.fu-berlin.de:2020/resource/persons/6 in a foaf:knows statement within your FOAF profile to refer to Andy Seaborne. By dereferencing the URI, an RDF browser can navigate from your FOAF profile to information about Andy in the database.
D2R Server also offers a traditional HTML interface to the data. Each resource has an XHTML representation. Web browsers retrieve these representations by dereferencing the resource URI with an HTTP request that asks for HTML (content type text/html) or XHTML (application/xhtml+xml).
+
+
GET /resource/persons/4 HTTP/1.0
+Accept: text/html
+
+
Like in the application/rdf+xml case, D2R Server will redirect to a document describing the resource, but this time an XHTML page:
+
+
+
+
+
The representation contains navigation links (Home | All Persons) that allow the complete content of the database to be browsed.
+
The <head> section of the HTML page contains a <link rel="alternate" /> tag pointing to the resource's RDF representation. This allows tools like Piggy Bank to switch between the HTML and RDF views. The RDF icon in the corner links to the same RDF document.
+
All pages are rendered from Velocity templates to allow customization. Future version of D2R Server might employ Fresnel lenses to improve resource display.
<sparql
+ xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema#"
+ xmlns="http://www.w3.org/2005/sparql-results#" >
+ <head>
+ <variable name="title"/>
+ <variable name="authorName"/>
+ </head>
+ <results ordered="true" distinct="false">
+ <result>
+ <binding name="title">
+ <literal>Automatic Generation of Java/SQL based Inference Engines from RDF Schema and RuleML</literal>
+ </binding>
+ <binding name="authorName">
+ <literal>Andreas Eberhart</literal>
+ </binding>
+ </result>
+ <result>
+ <binding name="title">
+ <literal>Trusting Information Sources One Citizen at a Time</literal>
+ </binding>
+ <binding name="authorName">
+ <literal>Varun Ratnakar</literal>
+ </binding>
+ </result>
+ <result>
+ <binding name="title">
+ <literal>Trusting Information Sources One Citizen at a Time</literal>
+ </binding>
+ <binding name="authorName">
+ <literal>Yolanda Gil</literal>
+ </binding>
+ </result>
+ </results>
+</sparql>
+
+
+
+
+
diff --git a/dump-rdf.bat b/dump-rdf.bat
index 61401edb..c4051791 100644
--- a/dump-rdf.bat
+++ b/dump-rdf.bat
@@ -1,15 +1,15 @@
-@echo off
-set D2R_ROOT=%~p0
-set CP="%D2RQ_ROOT%build"
-call :findjars "%D2RQ_ROOT%lib"
-set LOGCONFIG=file:%D2RQ_ROOT%etc/log4j.properties
-java -cp %CP% -Xmx256M "-Dlog4j.configuration=%LOGCONFIG%" d2rq.dump_rdf %1 %2 %3 %4 %5 %6 %7 %8 %9
-exit /B
-
-:findjars
-for %%j in (%1\*.jar) do call :addjar "%%j"
-for /D %%d in (%1\*) do call :findjars "%%d"
-exit /B
-
-:addjar
-set CP=%CP%;%1
+@echo off
+set D2R_ROOT=%~p0
+set CP="%D2RQ_ROOT%build"
+call :findjars "%D2RQ_ROOT%lib"
+set LOGCONFIG=file:%D2RQ_ROOT%etc/log4j.properties
+java -cp %CP% -Xmx256M "-Dlog4j.configuration=%LOGCONFIG%" d2rq.dump_rdf %1 %2 %3 %4 %5 %6 %7 %8 %9
+exit /B
+
+:findjars
+for %%j in (%1\*.jar) do call :addjar "%%j"
+for /D %%d in (%1\*) do call :findjars "%%d"
+exit /B
+
+:addjar
+set CP=%CP%;%1
diff --git a/etc/log4j-file.properties b/etc/log4j-file.properties
index 81ec08c7..8f96e608 100644
--- a/etc/log4j-file.properties
+++ b/etc/log4j-file.properties
@@ -1,15 +1,15 @@
-# Logging to file
-log4j.rootLogger=INFO, fileout
-log4j.appender.fileout=org.apache.log4j.FileAppender
-log4j.appender.fileout.File=d2rq.log
-log4j.appender.fileout.layout=org.apache.log4j.PatternLayout
-log4j.appender.fileout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} - %-25c{1} :: %m%n
-
-# Jena logging
-log4j.logger.com.hp.hpl.jena=WARN
-
-# Joseki server logging
-log4j.logger.org.joseki=INFO
-
-# Velocity log events are sent by Velocity to this logger
-log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
+# Logging to file
+log4j.rootLogger=INFO, fileout
+log4j.appender.fileout=org.apache.log4j.FileAppender
+log4j.appender.fileout.File=d2rq.log
+log4j.appender.fileout.layout=org.apache.log4j.PatternLayout
+log4j.appender.fileout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} - %-25c{1} :: %m%n
+
+# Jena logging
+log4j.logger.com.hp.hpl.jena=WARN
+
+# Joseki server logging
+log4j.logger.org.joseki=INFO
+
+# Velocity log events are sent by Velocity to this logger
+log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
diff --git a/etc/log4j-server.properties b/etc/log4j-server.properties
index 1f29fde4..bb3792c4 100644
--- a/etc/log4j-server.properties
+++ b/etc/log4j-server.properties
@@ -1,17 +1,17 @@
-# Logging to standard out
-log4j.rootLogger=INFO, stdout
-log4j.appender.stdout=org.apache.log4j.ConsoleAppender
-log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{HH:mm:ss} %-5p %-20c{1} :: %m%n
-
-# Jena logging
-log4j.logger.com.hp.hpl.jena=WARN
-
-# Joseki server logging
-log4j.logger.org.joseki=INFO
-
-# Velocity log events are sent by Velocity to this logger
-log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
-
-# Show SELECT statements executed by D2RQ
-log4j.logger.de.fuberlin.wiwiss.d2rq.sql.QueryExecutionIterator=ALL
+# Logging to standard out
+log4j.rootLogger=INFO, stdout
+log4j.appender.stdout=org.apache.log4j.ConsoleAppender
+log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
+log4j.appender.stdout.layout.ConversionPattern=%d{HH:mm:ss} %-5p %-20c{1} :: %m%n
+
+# Jena logging
+log4j.logger.com.hp.hpl.jena=WARN
+
+# Joseki server logging
+log4j.logger.org.joseki=INFO
+
+# Velocity log events are sent by Velocity to this logger
+log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
+
+# Show SELECT statements executed by D2RQ
+log4j.logger.de.fuberlin.wiwiss.d2rq.sql.QueryExecutionIterator=ALL
diff --git a/etc/log4j.properties b/etc/log4j.properties
index 32385da7..798bec8e 100644
--- a/etc/log4j.properties
+++ b/etc/log4j.properties
@@ -1,14 +1,14 @@
-# Logging to standard out
-log4j.rootLogger=INFO, stdout
-log4j.appender.stdout=org.apache.log4j.ConsoleAppender
-log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{HH:mm:ss} %-5p %-20c{1} :: %m%n
-
-# Jena logging
-log4j.logger.com.hp.hpl.jena=WARN
-
-# Joseki server logging
-log4j.logger.org.joseki=INFO
-
-# Velocity log events are sent by Velocity to this logger
-log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
+# Logging to standard out
+log4j.rootLogger=INFO, stdout
+log4j.appender.stdout=org.apache.log4j.ConsoleAppender
+log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
+log4j.appender.stdout.layout.ConversionPattern=%d{HH:mm:ss} %-5p %-20c{1} :: %m%n
+
+# Jena logging
+log4j.logger.com.hp.hpl.jena=WARN
+
+# Joseki server logging
+log4j.logger.org.joseki=INFO
+
+# Velocity log events are sent by Velocity to this logger
+log4j.logger.de.fuberlin.wiwiss.d2rq.server.velocity=WARN
diff --git a/generate-mapping.bat b/generate-mapping.bat
index 4010e8aa..5ee208d0 100644
--- a/generate-mapping.bat
+++ b/generate-mapping.bat
@@ -1,15 +1,15 @@
-@echo off
-set D2RQ_ROOT=%~p0
-set CP="%D2RQ_ROOT%build"
-call :findjars "%D2RQ_ROOT%lib"
-set LOGCONFIG=file:%D2RQ_ROOT%etc/log4j.properties
-java -cp %CP% -Xmx256M "-Dlog4j.configuration=%LOGCONFIG%" d2rq.generate_mapping %1 %2 %3 %4 %5 %6 %7 %8 %9
-exit /B
-
-:findjars
-for %%j in (%1\*.jar) do call :addjar "%%j"
-for /D %%d in (%1\*) do call :findjars "%%d"
-exit /B
-
-:addjar
-set CP=%CP%;%1
+@echo off
+set D2RQ_ROOT=%~p0
+set CP="%D2RQ_ROOT%build"
+call :findjars "%D2RQ_ROOT%lib"
+set LOGCONFIG=file:%D2RQ_ROOT%etc/log4j.properties
+java -cp %CP% -Xmx256M "-Dlog4j.configuration=%LOGCONFIG%" d2rq.generate_mapping %1 %2 %3 %4 %5 %6 %7 %8 %9
+exit /B
+
+:findjars
+for %%j in (%1\*.jar) do call :addjar "%%j"
+for /D %%d in (%1\*) do call :findjars "%%d"
+exit /B
+
+:addjar
+set CP=%CP%;%1
diff --git a/src/de/fuberlin/wiwiss/d2rq/engine/QueryEngineD2RQ.java b/src/de/fuberlin/wiwiss/d2rq/engine/QueryEngineD2RQ.java
index e43dfff8..982a2482 100644
--- a/src/de/fuberlin/wiwiss/d2rq/engine/QueryEngineD2RQ.java
+++ b/src/de/fuberlin/wiwiss/d2rq/engine/QueryEngineD2RQ.java
@@ -1,93 +1,93 @@
-package de.fuberlin.wiwiss.d2rq.engine;
-
-import com.hp.hpl.jena.query.Query;
-import com.hp.hpl.jena.sparql.algebra.AlgebraGenerator;
-import com.hp.hpl.jena.sparql.algebra.Op;
-import com.hp.hpl.jena.sparql.core.DatasetGraph;
-import com.hp.hpl.jena.sparql.engine.ExecutionContext;
-import com.hp.hpl.jena.sparql.engine.Plan;
-import com.hp.hpl.jena.sparql.engine.QueryEngineBase;
-import com.hp.hpl.jena.sparql.engine.QueryEngineFactory;
-import com.hp.hpl.jena.sparql.engine.QueryEngineRegistry;
-import com.hp.hpl.jena.sparql.engine.QueryIterator;
-import com.hp.hpl.jena.sparql.engine.binding.Binding;
-import com.hp.hpl.jena.sparql.engine.binding.BindingRoot;
-import com.hp.hpl.jena.sparql.engine.iterator.QueryIterRoot;
-import com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorCheck;
-import com.hp.hpl.jena.sparql.engine.main.QC;
-import com.hp.hpl.jena.sparql.engine.main.QueryEngineMain;
-import com.hp.hpl.jena.sparql.util.Context;
-
-import de.fuberlin.wiwiss.d2rq.GraphD2RQ;
-import de.fuberlin.wiwiss.d2rq.optimizer.D2RQTreeOptimizer;
-
-/**
- * TODO: @@@ QueryEngineD2RQ and the whole package is work in progress
- *
- * @author Richard Cyganiak (richard@cyganiak.de)
- * @version $Id: QueryEngineD2RQ.java,v 1.5 2009/02/09 12:21:30 fatorange Exp $
- */
-public class QueryEngineD2RQ extends QueryEngineMain {
- private GraphD2RQ graph;
-
- public QueryEngineD2RQ(GraphD2RQ graph, Query query) {
- this(graph, query, null);
- }
-
- public QueryEngineD2RQ(GraphD2RQ graph, Query query, Context context) {
- super(query, new D2RQDatasetGraph(graph), BindingRoot.create(), context);
- this.graph = graph;
- }
-
- public QueryEngineD2RQ(GraphD2RQ graph, Op op, Context context) {
- super(op, new D2RQDatasetGraph(graph), BindingRoot.create(), context);
- this.graph = graph;
- }
-
- protected Op modifyOp(Op op) {
- Op optimizedOp;
-
- optimizedOp = D2RQTreeOptimizer.optimize(op, graph);
- return optimizedOp;
- }
-
- public QueryIterator eval(Op op, DatasetGraph dataset, Binding input, Context context) {
-// ExecutionContext execCxt = new ExecutionContext(context, dataset.getDefaultGraph(), dataset, getFactory()) ;
-// QueryIterator qIter1 = QueryIterRoot.create(input, execCxt) ;
-// QueryIterator qIter = QC.compile(op, qIter1, execCxt);
-// return QueryIteratorCheck.check(qIter, execCxt);
- return super.eval(op, dataset, input, context);
- }
-
- // Factory stuff
- private static QueryEngineFactory factory = new QueryEngineFactoryD2RQ();
- public static QueryEngineFactory getFactory() {
- return factory;
- }
- public static void register() {
- QueryEngineRegistry.addFactory(factory);
- }
- public static void unregister() {
- QueryEngineRegistry.removeFactory(factory);
- }
- private static class QueryEngineFactoryD2RQ implements QueryEngineFactory {
- public boolean accept(Query query, DatasetGraph dataset, Context context) {
- return dataset instanceof D2RQDatasetGraph
- || dataset.getDefaultGraph() instanceof GraphD2RQ;
- }
- public Plan create(Query query, DatasetGraph dataset,
- Binding inputBinding, Context context) {
- return new QueryEngineD2RQ((GraphD2RQ) dataset.getDefaultGraph(),
- query, context).getPlan();
- }
- public boolean accept(Op op, DatasetGraph dataset, Context context) {
- return dataset instanceof D2RQDatasetGraph
- || dataset.getDefaultGraph() instanceof GraphD2RQ;
- }
- public Plan create(Op op, DatasetGraph dataset,
- Binding inputBinding, Context context) {
- return new QueryEngineD2RQ((GraphD2RQ) dataset.getDefaultGraph(),
- op, context).getPlan();
- }
- }
+package de.fuberlin.wiwiss.d2rq.engine;
+
+import com.hp.hpl.jena.query.Query;
+import com.hp.hpl.jena.sparql.algebra.AlgebraGenerator;
+import com.hp.hpl.jena.sparql.algebra.Op;
+import com.hp.hpl.jena.sparql.core.DatasetGraph;
+import com.hp.hpl.jena.sparql.engine.ExecutionContext;
+import com.hp.hpl.jena.sparql.engine.Plan;
+import com.hp.hpl.jena.sparql.engine.QueryEngineBase;
+import com.hp.hpl.jena.sparql.engine.QueryEngineFactory;
+import com.hp.hpl.jena.sparql.engine.QueryEngineRegistry;
+import com.hp.hpl.jena.sparql.engine.QueryIterator;
+import com.hp.hpl.jena.sparql.engine.binding.Binding;
+import com.hp.hpl.jena.sparql.engine.binding.BindingRoot;
+import com.hp.hpl.jena.sparql.engine.iterator.QueryIterRoot;
+import com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorCheck;
+import com.hp.hpl.jena.sparql.engine.main.QC;
+import com.hp.hpl.jena.sparql.engine.main.QueryEngineMain;
+import com.hp.hpl.jena.sparql.util.Context;
+
+import de.fuberlin.wiwiss.d2rq.GraphD2RQ;
+import de.fuberlin.wiwiss.d2rq.optimizer.D2RQTreeOptimizer;
+
+/**
+ * TODO: @@@ QueryEngineD2RQ and the whole package is work in progress
+ *
+ * @author Richard Cyganiak (richard@cyganiak.de)
+ * @version $Id: QueryEngineD2RQ.java,v 1.5 2009/02/09 12:21:30 fatorange Exp $
+ */
+public class QueryEngineD2RQ extends QueryEngineMain {
+ private GraphD2RQ graph;
+
+ public QueryEngineD2RQ(GraphD2RQ graph, Query query) {
+ this(graph, query, null);
+ }
+
+ public QueryEngineD2RQ(GraphD2RQ graph, Query query, Context context) {
+ super(query, new D2RQDatasetGraph(graph), BindingRoot.create(), context);
+ this.graph = graph;
+ }
+
+ public QueryEngineD2RQ(GraphD2RQ graph, Op op, Context context) {
+ super(op, new D2RQDatasetGraph(graph), BindingRoot.create(), context);
+ this.graph = graph;
+ }
+
+ protected Op modifyOp(Op op) {
+ Op optimizedOp;
+
+ optimizedOp = D2RQTreeOptimizer.optimize(op, graph);
+ return optimizedOp;
+ }
+
+ public QueryIterator eval(Op op, DatasetGraph dataset, Binding input, Context context) {
+// ExecutionContext execCxt = new ExecutionContext(context, dataset.getDefaultGraph(), dataset, getFactory()) ;
+// QueryIterator qIter1 = QueryIterRoot.create(input, execCxt) ;
+// QueryIterator qIter = QC.compile(op, qIter1, execCxt);
+// return QueryIteratorCheck.check(qIter, execCxt);
+ return super.eval(op, dataset, input, context);
+ }
+
+ // Factory stuff
+ private static QueryEngineFactory factory = new QueryEngineFactoryD2RQ();
+ public static QueryEngineFactory getFactory() {
+ return factory;
+ }
+ public static void register() {
+ QueryEngineRegistry.addFactory(factory);
+ }
+ public static void unregister() {
+ QueryEngineRegistry.removeFactory(factory);
+ }
+ private static class QueryEngineFactoryD2RQ implements QueryEngineFactory {
+ public boolean accept(Query query, DatasetGraph dataset, Context context) {
+ return dataset instanceof D2RQDatasetGraph
+ || dataset.getDefaultGraph() instanceof GraphD2RQ;
+ }
+ public Plan create(Query query, DatasetGraph dataset,
+ Binding inputBinding, Context context) {
+ return new QueryEngineD2RQ((GraphD2RQ) dataset.getDefaultGraph(),
+ query, context).getPlan();
+ }
+ public boolean accept(Op op, DatasetGraph dataset, Context context) {
+ return dataset instanceof D2RQDatasetGraph
+ || dataset.getDefaultGraph() instanceof GraphD2RQ;
+ }
+ public Plan create(Op op, DatasetGraph dataset,
+ Binding inputBinding, Context context) {
+ return new QueryEngineD2RQ((GraphD2RQ) dataset.getDefaultGraph(),
+ op, context).getPlan();
+ }
+ }
}
\ No newline at end of file
diff --git a/src/de/fuberlin/wiwiss/d2rq/optimizer/D2RQTreeOptimizer.java b/src/de/fuberlin/wiwiss/d2rq/optimizer/D2RQTreeOptimizer.java
index 53c9dfaa..e05c42c1 100644
--- a/src/de/fuberlin/wiwiss/d2rq/optimizer/D2RQTreeOptimizer.java
+++ b/src/de/fuberlin/wiwiss/d2rq/optimizer/D2RQTreeOptimizer.java
@@ -1,981 +1,981 @@
-package de.fuberlin.wiwiss.d2rq.optimizer;
-
-import java.util.ArrayList;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Set;
-import java.util.Stack;
-import com.hp.hpl.jena.sparql.algebra.Op;
-import com.hp.hpl.jena.sparql.algebra.OpVisitor;
-import com.hp.hpl.jena.sparql.algebra.Transform;
-import com.hp.hpl.jena.sparql.algebra.Transformer;
-import com.hp.hpl.jena.sparql.algebra.op.Op0;
-import com.hp.hpl.jena.sparql.algebra.op.Op1;
-import com.hp.hpl.jena.sparql.algebra.op.Op2;
-import com.hp.hpl.jena.sparql.algebra.op.OpAssign;
-import com.hp.hpl.jena.sparql.algebra.op.OpBGP;
-import com.hp.hpl.jena.sparql.algebra.op.OpConditional;
-import com.hp.hpl.jena.sparql.algebra.op.OpDatasetNames;
-import com.hp.hpl.jena.sparql.algebra.op.OpDiff;
-import com.hp.hpl.jena.sparql.algebra.op.OpDistinct;
-import com.hp.hpl.jena.sparql.algebra.op.OpExt;
-import com.hp.hpl.jena.sparql.algebra.op.OpFilter;
-import com.hp.hpl.jena.sparql.algebra.op.OpGraph;
-import com.hp.hpl.jena.sparql.algebra.op.OpGroupAgg;
-import com.hp.hpl.jena.sparql.algebra.op.OpJoin;
-import com.hp.hpl.jena.sparql.algebra.op.OpLabel;
-import com.hp.hpl.jena.sparql.algebra.op.OpLeftJoin;
-import com.hp.hpl.jena.sparql.algebra.op.OpList;
-import com.hp.hpl.jena.sparql.algebra.op.OpN;
-import com.hp.hpl.jena.sparql.algebra.op.OpNull;
-import com.hp.hpl.jena.sparql.algebra.op.OpOrder;
-import com.hp.hpl.jena.sparql.algebra.op.OpPath;
-import com.hp.hpl.jena.sparql.algebra.op.OpProcedure;
-import com.hp.hpl.jena.sparql.algebra.op.OpProject;
-import com.hp.hpl.jena.sparql.algebra.op.OpPropFunc;
-import com.hp.hpl.jena.sparql.algebra.op.OpQuadPattern;
-import com.hp.hpl.jena.sparql.algebra.op.OpReduced;
-import com.hp.hpl.jena.sparql.algebra.op.OpSequence;
-import com.hp.hpl.jena.sparql.algebra.op.OpService;
-import com.hp.hpl.jena.sparql.algebra.op.OpSlice;
-import com.hp.hpl.jena.sparql.algebra.op.OpTable;
-import com.hp.hpl.jena.sparql.algebra.op.OpTriple;
-import com.hp.hpl.jena.sparql.algebra.op.OpUnion;
-import com.hp.hpl.jena.sparql.expr.Expr;
-import com.hp.hpl.jena.sparql.expr.ExprList;
-import com.hp.hpl.jena.sparql.util.ALog;
-import de.fuberlin.wiwiss.d2rq.GraphD2RQ;
-import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformAddFilters;
-import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformPrepareOpTreeForOptimizing;
-import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformApplyD2RQOpimizingRules;
-import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformD2RQ;
-
-/**
- * Class for optimizing an op-tree especially for D2RQ.
- *
- * @author Herwig Leimer
- *
- */
-public class D2RQTreeOptimizer
-{
- /**
- * Constructor
- */
- private D2RQTreeOptimizer()
- {
- }
-
- /**
- * Method for optimizing an operator-tree.
- * In the first step for every operator in the tree, its threated object-vars
- * ale calculated, and put into a labelOp. This means the transformed-object-tree
- * looks like (originalop means the original operator): originalop -> OpLabel -> originalop -> OpLabel ....
- * This information is used for applying the rules for optimizing.
- * In the second step, all the rules of optimizing (see above) will be applied. This is done to
- * move the Filter-Conditions as far as possible down in the tree. In the optimal way, the filterconditions
- * is the parent of an opBGP
- * @param op
- * @return Optimized op
- */
- public static Op optimize(Op op, GraphD2RQ graph)
- {
- Op transformedOp;
-
- // add OpLabels with the threated object-vars for every operator
- transformedOp = Transformer.transform(new TransformPrepareOpTreeForOptimizing(), op);
- // now apply the rules for optimizing the operator-tree, replaces the OpBGPs through
- // OpD2RQs and removes unneeded Ops like labels and filters with no conditions
- transformedOp = optimizeAndGenerate(transformedOp, graph);
-
- return transformedOp;
- }
-
-
- /**
- * Method for optimizing and transforming the operator-tree.
- * @param transform - Transformer for changing the operator-tree
- * @param op - root-node of the operator-tree
- * @return Op - the transformed operator-tree
- */
- private static Op optimizeAndGenerate(Op op, GraphD2RQ graph)
- {
- if ( op == null )
- {
- ALog.warn(D2RQTreeOptimizer.class, "Attempt to transform a null Op - ignored") ;
- return op ;
- }
-
-
- // visitor that applies the rules for moving down the filterconditions
- D2RQOpTreeVisitor d2RQOpTreeVisitor = new D2RQOpTreeVisitor(graph) ;
- // start visiting
- op.visit(d2RQOpTreeVisitor) ;
- Op r = d2RQOpTreeVisitor.result() ;
- return r ;
- }
-
- /**
- * Visitor for traversing the operator-tree, moving down the filterconditions
- * as far as possible, and for applying the transformer to change all OpBGPs to OpD2RQs
- * The optimizing will be done during the top-down-stepping, the transformation during
- * the bottom-up-stepping
- *
- * @author Herwig Leimer
- *
- */
- private static final class D2RQOpTreeVisitor implements OpVisitor
- {
- private Transform optimizingTransform, d2rqTransform, addFilterTransform;
- private Stack stack;
- private List filterExpr;
-
- /**
- * Constructor
- * @param transform - Transformer to be applied on the operator-tree
- */
- public D2RQOpTreeVisitor(GraphD2RQ graph)
- {
- this.filterExpr = new ArrayList();
- this.optimizingTransform = new TransformApplyD2RQOpimizingRules();
- this.d2rqTransform = new TransformD2RQ(graph);
- this.addFilterTransform = new TransformAddFilters();
- this.stack = new Stack();
- }
-
- /**
- * Returns the changed operator-tree
- * @return Op - root-node of the operator-tree
- */
- public Op result()
- {
- if ( stack.size() != 1 )
- ALog.warn(this, "Stack is not aligned") ;
- return (Op)this.stack.pop() ;
- }
-
- /**
- * When visiting an OpFilter, all its filterconditions are collected during the top-down-stepping.
- * During the bottom-up-stepping all filterconditions which were moved down, are removed
- */
- public void visit(OpFilter opFilter)
- {
- Op newOp, subOp = null ;
- ExprList exprList;
-
- // get the filterconditions
- exprList = opFilter.getExprs();
-
- // add all the filterconditions to the condition-list
- this.filterExpr.addAll(exprList.getList());
-
- // step down
- if (opFilter.getSubOp() != null )
- {
- opFilter.getSubOp().visit(this);
- subOp = (Op)this.stack.pop();
- }
-
- // remove available filterexpressions
- opFilter.getExprs().getList().removeAll(this.filterExpr);
-
- // removes the filter if it has no expressions
- newOp = opFilter.apply(d2rqTransform, subOp);
-
- this.stack.push(newOp);
- }
-
- /**
- * When visiting an OpUnion also 3 conditions for moving down the filterconditions are
- * checked. Only when a condition is satisfied, the filtercondition can be moved
- * down to the next operator in the tree. Otherwise the condition will stay here
- * and during the bottum-up-stepping an OpFilter containing these remained
- * filterconditions will be inserted in the operator-tree.
- * Conditions for moving down a filtercondition:
- * (M1, M2 are graphpatterns, F is a filtercondition)
- * 1) Filter(Union(M1, M2), F)) will become Union(Filter(M1, F), M2) when the
- * filterexpression is only referenced to M1
- * 2) Filter(Union(M1, M2), F)) will become Union(M1, Filter(M2, F)) when the
- * filterexpression is only referenced to M2
- * 3) Filter(Union(M1, M2), F)) will become Join(Union(M1, F), Union(M2, F)) when the
- * filterexpression is referenced to M1 and M2
- */
- public void visit(OpUnion opUnion)
- {
- checkMoveDownFilterExprAndVisitOpUnion(opUnion);
- }
-
- /**
- * When visiting an OpJoin 3 conditions for moving down the filterconditions are
- * checked. Only when a condition is satisfied, the filtercondition can be moved
- * down to the next operator in the tree. Otherwise the condition will stay here
- * and during the bottum-up-stepping an OpFilter containing these remained
- * filterconditions will be inserted in the operator-tree.
- * Conditions for moving down a filtercondition:
- * (M1, M2 are graphpatterns, F is a filtercondition)
- * 1) Filter(Join(M1, M2), F)) will become Join(Filter(M1, F), M2) when the
- * filterexpression is only referenced to M1
- * 2) Filter(Join(M1, M2), F)) will become Join(M1, Filter(M2, F)) when the
- * filterexpression is only referenced to M2
- * 3) Filter(Join(M1, M2), F)) will become Join(Filter(M1, F), Filter(M2, F)) when the
- * filterexpression is referenced to M1 and M2
- */
- public void visit(OpJoin opJoin)
- {
- checkMoveDownFilterExprAndVisitOpJoin(opJoin);
- }
-
-
- /**
- * When there are some filterexpressions which belong to an OpBGP, the OpBGP will be converted
- * to an OpFilteredBGP. A OpFilteredBGP is nearly the same like an OpBGP but it has
- * a link to its parent, which is an OpFilter with the coresponding filter-conditions,
- * because in the transforming-process of the OpBGPs to OpD2RQs a link to the above OpFilter
- * is needed.
- */
- public void visit(OpBGP op)
- {
- Op newOp;
-
- newOp = op;
-
-
- // are there some filterexpression which belong to this BGP ?
- if (!this.filterExpr.isEmpty())
- {
- // apply the optimizing rules
- // create a filteredOpBGP and a filter
- newOp = op.apply(optimizingTransform);
- // add the current valid filterconditions
- ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
- }
-
- if (newOp instanceof OpFilter)
- {
- // OpFilteredBGP must be transformed
- newOp = ((OpFilter)newOp).getSubOp();
- }
-
- // apply the tranformation from an OpBGP/OpFilteredBGP to an OpD2RQ
- newOp = ((OpBGP)newOp).apply(d2rqTransform);
-
- this.stack.push(newOp);
- }
-
- /**
- * When visiting an OpDiff 3 conditions for moving down the filterconditions are
- * checked. Only when a condition is satisfied, the filtercondition can be moved
- * down to the next operator in the tree. Otherwise the condition will stay here
- * and during the bottum-up-stepping an OpFilter containing these remained
- * filterconditions will be inserted in the operator-tree.
- * Conditions for moving down a filtercondition:
- * (M1, M2 are graphpatterns, F is a filtercondition)
- * 1) Filter(Diff(M1, M2), F)) will become Diff(Filter(M1, F), M2) when the
- * filterexpression is only referenced to M1
- * 2) Filter(Diff(M1, M2), F)) will become Diff(M1, Filter(M2, F)) when the
- * filterexpression is only referenced to M2
- * 3) Filter(Diff(M1, M2), F)) will become Diff(Filter(M1, F), Filter(M2, F)) when the
- * filterexpression is referenced to M1 and M2
- */
- public void visit(OpDiff opDiff)
- {
- // TODO: Regel nochmal ueberdenken !!!
- checkMoveDownFilterExprAndVisitOpDiff(opDiff);
- }
-
- public void visit(OpConditional opCondition) {
- notMoveDownFilterExprAndVisitOp2(opCondition); //TODO moving down / improvements possible!! Check this
- }
-
- public void visit(OpProcedure opProc)
- {
- notMoveDownFilterExprAndVisitOp1(opProc);
- }
-
- public void visit(OpPropFunc opPropFunc)
- {
- notMoveDownFilterExprAndVisitOp1(opPropFunc);
- }
-
- public void visit(OpTable opTable)
- {
- notMoveDownFilterExprAndVisitOp0(opTable);
- }
-
- public void visit(OpQuadPattern quadPattern)
- {
- notMoveDownFilterExprAndVisitOp0(quadPattern);
- }
-
- public void visit(OpPath opPath)
- {
- notMoveDownFilterExprAndVisitOp0(opPath);
- }
-
- public void visit(OpTriple opTriple)
- {
- notMoveDownFilterExprAndVisitOp0(opTriple);
- }
-
- public void visit(OpDatasetNames dsNames)
- {
- notMoveDownFilterExprAndVisitOp0(dsNames);
- }
-
- public void visit(OpSequence opSequence)
- {
- notMoveDownFilterExprAndVisitOpN(opSequence);
- }
-
- /**
- * When visiting an OpJoin 2 conditions for moving down the filterconditions are
- * checked. Only when a condition is satisfied, the filtercondition can be moved
- * down to the next operator in the tree. Otherwise the condition will stay here
- * and during the bottum-up-stepping an OpFilter containing these remained
- * filterconditions will be inserted in the operator-tree.
- * Conditions for moving down a filtercondition:
- * (M1, M2 are graphpatterns, F is a filtercondition)
- * 1) Filter(LeftJoin(M1, M2), F)) will become LeftJoin(Filter(M1, F), M2) when the
- * filterexpression is only referenced to M1
- * 2) Filter(LeftJoin(M1, M2), F)) will become LeftJoin(Filter(M1, F), Filter(M2, F)) when the
- * filterexpression is referenced to M1 and M2
- */
- public void visit(OpLeftJoin opLeftJoin)
- {
- checkMoveDownFilterExprAndVisitOpLeftJoin(opLeftJoin);
- }
-
- public void visit(OpGraph opGraph)
- {
- notMoveDownFilterExprAndVisitOp1(opGraph);
- }
-
- public void visit(OpService opService)
- {
- notMoveDownFilterExprAndVisitOp1(opService);
- }
-
- public void visit(OpExt opExt)
- {
- Op newOp;
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!this.filterExpr.isEmpty())
- {
- // create the filter
- newOp = addFilterTransform.transform(opExt);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
- }else
- {
- // nothing must be done
- newOp = opExt;
- }
-
- this.stack.push(newOp);
- }
-
- public void visit(OpNull opNull)
- {
- notMoveDownFilterExprAndVisitOp0(opNull);
- }
-
- public void visit(OpLabel opLabel)
- {
- Op newOp, subOp = null ;
-
- // check for subtree
- if (opLabel.getSubOp() != null )
- {
- opLabel.getSubOp().visit(this);
- subOp = (Op)this.stack.pop();
- }
-
- // remove the label
- newOp = opLabel.apply(d2rqTransform, subOp);
- this.stack.push(newOp);
-
- }
-
- public void visit(OpList opList)
- {
- notMoveDownFilterExprAndVisitOp1(opList);
- }
-
- public void visit(OpOrder opOrder)
- {
- notMoveDownFilterExprAndVisitOp1(opOrder);
- }
-
- public void visit(OpProject opProject)
- {
- notMoveDownFilterExprAndVisitOp1(opProject);
- }
-
- public void visit(OpDistinct opDistinct)
- {
- notMoveDownFilterExprAndVisitOp1(opDistinct);
- }
-
- public void visit(OpReduced opReduced)
- {
- notMoveDownFilterExprAndVisitOp1(opReduced);
- }
-
- public void visit(OpAssign opAssign)
- {
- notMoveDownFilterExprAndVisitOp1(opAssign);
- }
-
- public void visit(OpSlice opSlice)
- {
- notMoveDownFilterExprAndVisitOp1(opSlice);
- }
-
- public void visit(OpGroupAgg opGroupAgg)
- {
- notMoveDownFilterExprAndVisitOp1(opGroupAgg);
- }
-
- /**
- * Calculates the set of valid filterexpressions as a subset from the whole set of
- * possibilities.
- * @param possiblities - the whole set
- * @param opLabel - contains the threated object-vars, which are used to determine
- * the valid filterexpressions
- * @return List - the subset from the set of possiblities
- */
- private List calcValidFilterExpr(List possiblities, OpLabel opLabel)
- {
- Set threatedObjectVars;
- Expr expr;
- Set mentionedVarsInExpr;
- List result;
-
- // get the set of the threated object-vars
- threatedObjectVars = (Set) opLabel.getObject();
-
- result = new ArrayList();
-
- // now check every var of every filterexpression in the set of all
- // filterexpression-possibilities to check if the filterexpression is valid
- for (Iterator iterator = possiblities.iterator(); iterator.hasNext();)
- {
- expr = (Expr)iterator.next();
-
- // get all in the expression mentioned object-vars
- mentionedVarsInExpr = expr.getVarsMentioned();
-
- // check if the alle mentioned-vars of the expression are a subset
- // of the threated-Object-Vars
- // if yes, the expression is valid, otherwise not
- if (threatedObjectVars.containsAll(mentionedVarsInExpr))
- {
- result.add(expr);
- }
- }
-
- return result;
- }
-
-
- /**
- * Checks first if a filterexpression can be moved down.
- * And after visits the operator
- */
- private void checkMoveDownFilterExprAndVisitOpUnion(OpUnion opUnion)
- {
- Op left = null ;
- Op right = null ;
- Op newOp;
- OpLabel opLeftLabel, opRightLabel;
- List filterExprBeforeOpUnion, filterExprAfterOpUnion, notMoveableFilterExpr;
-
- // contains the filterexpressions that are valid before this op2
- filterExprBeforeOpUnion = new ArrayList(this.filterExpr);
- // contains the filterexpressions that are valid after this op2
- // this is needed because during the bottom-up-stepping all filterexpressions
- // which could not be transformed down, must be inserted by means of an OpFilter
- // above this op2
- filterExprAfterOpUnion = new ArrayList();
-
- // check left subtree
- if ((left = opUnion.getLeft()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // left subtree
- if (left instanceof OpLabel)
- {
- opLeftLabel = (OpLabel)left;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnion, opLeftLabel);
- filterExprAfterOpUnion.addAll(this.filterExpr);
- }
- // step down
- opUnion.getLeft().visit(this);
- left = (Op)this.stack.pop();
- }
-
- // check the right subtree
- if ((right = opUnion.getRight()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // right subtree
- if (right instanceof OpLabel)
- {
- opRightLabel = (OpLabel)right;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnion, opRightLabel);
- filterExprAfterOpUnion.addAll(this.filterExpr);
- }
- // step down
- opUnion.getRight().visit(this);
- right = (Op)this.stack.pop();
- }
-
- // note: filterExprAfterOpUnion contains now all filterexpressions which could
- // be moved down
- // now calculate all filterexpressions which were not moveable
- notMoveableFilterExpr = new ArrayList(filterExprBeforeOpUnion);
- notMoveableFilterExpr.removeAll(filterExprAfterOpUnion);
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = opUnion.apply(addFilterTransform, left, right);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- newOp = opUnion.apply(d2rqTransform, left, right);
- }
-
- // restore filterexpressions
- this.filterExpr = filterExprBeforeOpUnion;
-
- this.stack.push(newOp);
- }
-
- /**
- * Checks first if a filterexpression can be moved down.
- * And after visits the operator
- */
- private void checkMoveDownFilterExprAndVisitOpJoin(OpJoin opJoin)
- {
- Op left = null ;
- Op right = null ;
- Op newOp;
- OpLabel opLeftLabel, opRightLabel;
- List filterExprBeforeOpJoin, filterExprAfterOpJoin, notMoveableFilterExpr;
-
- // contains the filterexpressions that are valid before this op2
- filterExprBeforeOpJoin = new ArrayList(this.filterExpr);
- // contains the filterexpressions that are valid after this op2
- // this is needed because during the bottom-up-stepping all filterexpressions
- // which could not be transformed down, must be inserted by means of an OpFilter
- // above this op2
- filterExprAfterOpJoin = new ArrayList();
-
- // check left subtree
- if ((left = opJoin.getLeft()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // left subtree
- if (left instanceof OpLabel)
- {
- opLeftLabel = (OpLabel)left;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpJoin, opLeftLabel);
- filterExprAfterOpJoin.addAll(this.filterExpr);
- }
- // step down
- opJoin.getLeft().visit(this);
- left = (Op)this.stack.pop();
- }
-
- // check the right subtree
- if ((right = opJoin.getRight()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // right subtree
- if (right instanceof OpLabel)
- {
- opRightLabel = (OpLabel)right;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpJoin, opRightLabel);
- filterExprAfterOpJoin.addAll(this.filterExpr);
- }
- // step down
- opJoin.getRight().visit(this);
- right = (Op)this.stack.pop();
- }
-
- // note: filterExprAfterOpUnion contains now all filterexpressions which could
- // be moved down
- // now calculate all filterexpressions which were not moveable
- notMoveableFilterExpr = new ArrayList(filterExprBeforeOpJoin);
- notMoveableFilterExpr.removeAll(filterExprAfterOpJoin);
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = opJoin.apply(addFilterTransform, left, right);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- // nothing must be done
- newOp = opJoin.apply(d2rqTransform, left, right);
- }
-
- // restore filterexpressions
- this.filterExpr = filterExprBeforeOpJoin;
-
- this.stack.push(newOp);
- }
-
- /**
- * Checks first if a filterexpression can be moved down.
- * And after visits the operator
- */
- private void checkMoveDownFilterExprAndVisitOpLeftJoin(OpLeftJoin opLeftJoin)
- {
- Op left = null ;
- Op right = null ;
- Op newOp;
- OpLabel opLeftLabel, opRightLabel;
- List filterExprBeforeOpLeftJoin, filterExprAfterOpLeftJoin, notMoveableFilterExpr, filterExprRightSide, validFilterExprRightSide;
- Expr expr;
-
- // contains the filterexpressions that are valid before this op2
- filterExprBeforeOpLeftJoin = new ArrayList(this.filterExpr);
- // contains the filterexpressions that are valid after this op2
- // this is needed because during the bottom-up-stepping all filterexpressions
- // which could not be transformed down, must be inserted by means of an OpFilter
- // above this op2
- filterExprAfterOpLeftJoin = new ArrayList();
-
-
- // check left subtree
- if ((left = opLeftJoin.getLeft()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // left subtree
- if (left instanceof OpLabel)
- {
- opLeftLabel = (OpLabel)left;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpLeftJoin, opLeftLabel);
- filterExprAfterOpLeftJoin.addAll(this.filterExpr);
- }
- // step down
- opLeftJoin.getLeft().visit(this);
- left = (Op)this.stack.pop();
- }
-
- // check the right subtree
- if ((right = opLeftJoin.getRight()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // right subtree
- if (right instanceof OpLabel)
- {
- opRightLabel = (OpLabel)right;
-
- filterExprRightSide = calcValidFilterExpr(filterExprBeforeOpLeftJoin, opRightLabel);
- validFilterExprRightSide = new ArrayList();
-
- // now check for expr that are vaild for left-side and right-side
- // only when an expr is vaid for left-side, it can be vaild for right-side
- for(Iterator iterator = filterExprRightSide.iterator(); iterator.hasNext();)
- {
- expr = (Expr)iterator.next();
-
- if (this.filterExpr.contains(expr))
- {
- // valid
- validFilterExprRightSide.add(expr);
- }
- }
-
- this.filterExpr = validFilterExprRightSide;
- filterExprAfterOpLeftJoin.addAll(this.filterExpr);
- }
- // step down
- opLeftJoin.getRight().visit(this);
- right = (Op)this.stack.pop();
- }
-
-
-
- // note: filterExprAfterOpUnion contains now all filterexpressions which could
- // be moved down
- // now calculate all filterexpressions which were not moveable
- notMoveableFilterExpr = new ArrayList(filterExprBeforeOpLeftJoin);
- notMoveableFilterExpr.removeAll(filterExprAfterOpLeftJoin);
-
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter for an opleftjoin
- newOp = opLeftJoin.apply(addFilterTransform, left, right);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- // nothing must be done
- newOp = opLeftJoin.apply(d2rqTransform, left, right);
- }
-
- // restore filterexpressions
- this.filterExpr = filterExprBeforeOpLeftJoin;
-
- this.stack.push(newOp);
- }
-
-
- /**
- * Checks first if a filterexpression can be moved down.
- * And after visits the operator
- */
- private void checkMoveDownFilterExprAndVisitOpDiff(Op2 opDiff)
- {
- Op left = null ;
- Op right = null ;
- Op newOp;
- OpLabel opLeftLabel, opRightLabel;
- List filterExprBeforeOpUnionOpJoin, filterExprAfterOpUnionOpJoin, notMoveableFilterExpr;
-
- // contains the filterexpressions that are valid before this op2
- filterExprBeforeOpUnionOpJoin = new ArrayList(this.filterExpr);
- // contains the filterexpressions that are valid after this op2
- // this is needed because during the bottom-up-stepping all filterexpressions
- // which could not be transformed down, must be inserted by means of an OpFilter
- // above this op2
- filterExprAfterOpUnionOpJoin = new ArrayList();
-
- // check left subtree
- if ((left = opDiff.getLeft()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // left subtree
- if (left instanceof OpLabel)
- {
- opLeftLabel = (OpLabel)left;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnionOpJoin, opLeftLabel);
- filterExprAfterOpUnionOpJoin.addAll(this.filterExpr);
- }
- // step down
- opDiff.getLeft().visit(this);
- left = (Op)this.stack.pop();
- }
-
- // check the right subtree
- if ((right = opDiff.getRight()) != null )
- {
- // calculate the set of filterexpressions that are also valid for the
- // right subtree
- if (right instanceof OpLabel)
- {
- opRightLabel = (OpLabel)right;
- this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnionOpJoin, opRightLabel);
- filterExprAfterOpUnionOpJoin.addAll(this.filterExpr);
- }
- // step down
- opDiff.getRight().visit(this);
- right = (Op)this.stack.pop();
- }
-
- // note: filterExprAfterOpUnion contains now all filterexpressions which could
- // be moved down
- // now calculate all filterexpressions which were not moveable
- notMoveableFilterExpr = new ArrayList(filterExprBeforeOpUnionOpJoin);
- notMoveableFilterExpr.removeAll(filterExprAfterOpUnionOpJoin);
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = opDiff.apply(addFilterTransform, left, right);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- // nothing must be done
- newOp = opDiff.apply(d2rqTransform, left, right);
- }
-
- // restore filterexpressions
- this.filterExpr = filterExprBeforeOpUnionOpJoin;
-
- this.stack.push(newOp);
- }
-
-
- /**
- * Handels and visits all op0 for which no filterexpression can be moved down.
- * @param op0 - an Op0
- */
- private void notMoveDownFilterExprAndVisitOp0(Op0 op0)
- {
- Op newOp;
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!this.filterExpr.isEmpty())
- {
- // create the filter
- newOp = op0.apply(addFilterTransform);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
- }else
- {
- // nothing must be done
- newOp = op0.apply(d2rqTransform);
- }
-
- this.stack.push(newOp);
- }
-
- /**
- * Handels and visits all op1 for which no filterexpression can be moved down.
- * @param op1 - an Op1
- */
- private void notMoveDownFilterExprAndVisitOp1(Op1 op1)
- {
- List notMoveableFilterExpr;
- Op newOp, subOp = null ;
-
- // all available FilterExpr could not be moved down
- notMoveableFilterExpr = new ArrayList(this.filterExpr);
- // clear the filterExpr
- filterExpr.clear();
-
- // check for subtree
- if (op1.getSubOp() != null )
- {
- op1.getSubOp().visit(this);
- subOp = (Op)this.stack.pop();
- }
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = op1.apply(addFilterTransform, subOp);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- newOp = op1.apply(d2rqTransform, subOp);
- }
-
- // restore filterexpressions
- this.filterExpr = notMoveableFilterExpr;
- this.stack.push(newOp);
- }
-
- /**
- * Handels and visits all op2 for which no filterexpression can be moved down.
- * @param op2 - an Op2
- */
- private void notMoveDownFilterExprAndVisitOp2(Op2 op2)
- {
- List notMoveableFilterExpr;
- Op left = null ;
- Op right = null ;
- Op newOp;
-
- // all available FilterExpr could not be moved down
- notMoveableFilterExpr = new ArrayList(this.filterExpr);
- // clear the filterExpr
- filterExpr.clear();
-
- // check for left subtree
- if (op2.getLeft() != null )
- {
- // step down
- op2.getLeft().visit(this);
- left = (Op)this.stack.pop();
- }
- if (op2.getRight() != null )
- {
- // step down
- op2.getRight().visit(this);
- right = (Op)this.stack.pop();
- }
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = op2.apply(addFilterTransform, left, right);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- // nothing must be done
- newOp = op2.apply(d2rqTransform, left, right);
- }
-
- // restore filterexpressions
- this.filterExpr = notMoveableFilterExpr;
- this.stack.push(newOp);
- }
-
- /**
- * Handels and visits all opN for which no filterexpression can be moved down.
- * @param opN - an OpN
- */
- private void notMoveDownFilterExprAndVisitOpN(OpN opN)
- {
- List list;
- List notMoveableFilterExpr;
- Op subOp, op, newOp;
-
- // all available FilterExpr could not be moved down
- notMoveableFilterExpr = new ArrayList(this.filterExpr);
- // clear the filterExpr
- filterExpr.clear();
-
- list = new ArrayList(opN.size()) ;
-
- // check for subtrees
- for (Iterator iterator = opN.iterator(); iterator.hasNext();)
- {
- subOp = (Op)iterator.next();
- subOp.visit(this);
-
- op = (Op)this.stack.pop();
- if (op != null)
- {
- list.add(op);
- }
- }
-
- // if there are some filterexpressions which could not be moved down,
- // an opFilter must be inserted that contains this filterexpressions
- if (!notMoveableFilterExpr.isEmpty())
- {
- // create the filter
- newOp = opN.apply(addFilterTransform, list);
- // add the conditions
- ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
- }else
- {
- // nothing must be done
- newOp = opN.apply(d2rqTransform, list);
- }
-
- // restore filterexpressions
- this.filterExpr = notMoveableFilterExpr;
- this.stack.push(newOp);
- }
-
- }
-
-}
+package de.fuberlin.wiwiss.d2rq.optimizer;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.Stack;
+import com.hp.hpl.jena.sparql.algebra.Op;
+import com.hp.hpl.jena.sparql.algebra.OpVisitor;
+import com.hp.hpl.jena.sparql.algebra.Transform;
+import com.hp.hpl.jena.sparql.algebra.Transformer;
+import com.hp.hpl.jena.sparql.algebra.op.Op0;
+import com.hp.hpl.jena.sparql.algebra.op.Op1;
+import com.hp.hpl.jena.sparql.algebra.op.Op2;
+import com.hp.hpl.jena.sparql.algebra.op.OpAssign;
+import com.hp.hpl.jena.sparql.algebra.op.OpBGP;
+import com.hp.hpl.jena.sparql.algebra.op.OpConditional;
+import com.hp.hpl.jena.sparql.algebra.op.OpDatasetNames;
+import com.hp.hpl.jena.sparql.algebra.op.OpDiff;
+import com.hp.hpl.jena.sparql.algebra.op.OpDistinct;
+import com.hp.hpl.jena.sparql.algebra.op.OpExt;
+import com.hp.hpl.jena.sparql.algebra.op.OpFilter;
+import com.hp.hpl.jena.sparql.algebra.op.OpGraph;
+import com.hp.hpl.jena.sparql.algebra.op.OpGroupAgg;
+import com.hp.hpl.jena.sparql.algebra.op.OpJoin;
+import com.hp.hpl.jena.sparql.algebra.op.OpLabel;
+import com.hp.hpl.jena.sparql.algebra.op.OpLeftJoin;
+import com.hp.hpl.jena.sparql.algebra.op.OpList;
+import com.hp.hpl.jena.sparql.algebra.op.OpN;
+import com.hp.hpl.jena.sparql.algebra.op.OpNull;
+import com.hp.hpl.jena.sparql.algebra.op.OpOrder;
+import com.hp.hpl.jena.sparql.algebra.op.OpPath;
+import com.hp.hpl.jena.sparql.algebra.op.OpProcedure;
+import com.hp.hpl.jena.sparql.algebra.op.OpProject;
+import com.hp.hpl.jena.sparql.algebra.op.OpPropFunc;
+import com.hp.hpl.jena.sparql.algebra.op.OpQuadPattern;
+import com.hp.hpl.jena.sparql.algebra.op.OpReduced;
+import com.hp.hpl.jena.sparql.algebra.op.OpSequence;
+import com.hp.hpl.jena.sparql.algebra.op.OpService;
+import com.hp.hpl.jena.sparql.algebra.op.OpSlice;
+import com.hp.hpl.jena.sparql.algebra.op.OpTable;
+import com.hp.hpl.jena.sparql.algebra.op.OpTriple;
+import com.hp.hpl.jena.sparql.algebra.op.OpUnion;
+import com.hp.hpl.jena.sparql.expr.Expr;
+import com.hp.hpl.jena.sparql.expr.ExprList;
+import com.hp.hpl.jena.sparql.util.ALog;
+import de.fuberlin.wiwiss.d2rq.GraphD2RQ;
+import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformAddFilters;
+import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformPrepareOpTreeForOptimizing;
+import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformApplyD2RQOpimizingRules;
+import de.fuberlin.wiwiss.d2rq.optimizer.transformer.TransformD2RQ;
+
+/**
+ * Class for optimizing an op-tree especially for D2RQ.
+ *
+ * @author Herwig Leimer
+ *
+ */
+public class D2RQTreeOptimizer
+{
+ /**
+ * Constructor
+ */
+ private D2RQTreeOptimizer()
+ {
+ }
+
+ /**
+ * Method for optimizing an operator-tree.
+ * In the first step for every operator in the tree, its threated object-vars
+ * ale calculated, and put into a labelOp. This means the transformed-object-tree
+ * looks like (originalop means the original operator): originalop -> OpLabel -> originalop -> OpLabel ....
+ * This information is used for applying the rules for optimizing.
+ * In the second step, all the rules of optimizing (see above) will be applied. This is done to
+ * move the Filter-Conditions as far as possible down in the tree. In the optimal way, the filterconditions
+ * is the parent of an opBGP
+ * @param op
+ * @return Optimized op
+ */
+ public static Op optimize(Op op, GraphD2RQ graph)
+ {
+ Op transformedOp;
+
+ // add OpLabels with the threated object-vars for every operator
+ transformedOp = Transformer.transform(new TransformPrepareOpTreeForOptimizing(), op);
+ // now apply the rules for optimizing the operator-tree, replaces the OpBGPs through
+ // OpD2RQs and removes unneeded Ops like labels and filters with no conditions
+ transformedOp = optimizeAndGenerate(transformedOp, graph);
+
+ return transformedOp;
+ }
+
+
+ /**
+ * Method for optimizing and transforming the operator-tree.
+ * @param transform - Transformer for changing the operator-tree
+ * @param op - root-node of the operator-tree
+ * @return Op - the transformed operator-tree
+ */
+ private static Op optimizeAndGenerate(Op op, GraphD2RQ graph)
+ {
+ if ( op == null )
+ {
+ ALog.warn(D2RQTreeOptimizer.class, "Attempt to transform a null Op - ignored") ;
+ return op ;
+ }
+
+
+ // visitor that applies the rules for moving down the filterconditions
+ D2RQOpTreeVisitor d2RQOpTreeVisitor = new D2RQOpTreeVisitor(graph) ;
+ // start visiting
+ op.visit(d2RQOpTreeVisitor) ;
+ Op r = d2RQOpTreeVisitor.result() ;
+ return r ;
+ }
+
+ /**
+ * Visitor for traversing the operator-tree, moving down the filterconditions
+ * as far as possible, and for applying the transformer to change all OpBGPs to OpD2RQs
+ * The optimizing will be done during the top-down-stepping, the transformation during
+ * the bottom-up-stepping
+ *
+ * @author Herwig Leimer
+ *
+ */
+ private static final class D2RQOpTreeVisitor implements OpVisitor
+ {
+ private Transform optimizingTransform, d2rqTransform, addFilterTransform;
+ private Stack stack;
+ private List filterExpr;
+
+ /**
+ * Constructor
+ * @param transform - Transformer to be applied on the operator-tree
+ */
+ public D2RQOpTreeVisitor(GraphD2RQ graph)
+ {
+ this.filterExpr = new ArrayList();
+ this.optimizingTransform = new TransformApplyD2RQOpimizingRules();
+ this.d2rqTransform = new TransformD2RQ(graph);
+ this.addFilterTransform = new TransformAddFilters();
+ this.stack = new Stack();
+ }
+
+ /**
+ * Returns the changed operator-tree
+ * @return Op - root-node of the operator-tree
+ */
+ public Op result()
+ {
+ if ( stack.size() != 1 )
+ ALog.warn(this, "Stack is not aligned") ;
+ return (Op)this.stack.pop() ;
+ }
+
+ /**
+ * When visiting an OpFilter, all its filterconditions are collected during the top-down-stepping.
+ * During the bottom-up-stepping all filterconditions which were moved down, are removed
+ */
+ public void visit(OpFilter opFilter)
+ {
+ Op newOp, subOp = null ;
+ ExprList exprList;
+
+ // get the filterconditions
+ exprList = opFilter.getExprs();
+
+ // add all the filterconditions to the condition-list
+ this.filterExpr.addAll(exprList.getList());
+
+ // step down
+ if (opFilter.getSubOp() != null )
+ {
+ opFilter.getSubOp().visit(this);
+ subOp = (Op)this.stack.pop();
+ }
+
+ // remove available filterexpressions
+ opFilter.getExprs().getList().removeAll(this.filterExpr);
+
+ // removes the filter if it has no expressions
+ newOp = opFilter.apply(d2rqTransform, subOp);
+
+ this.stack.push(newOp);
+ }
+
+ /**
+ * When visiting an OpUnion also 3 conditions for moving down the filterconditions are
+ * checked. Only when a condition is satisfied, the filtercondition can be moved
+ * down to the next operator in the tree. Otherwise the condition will stay here
+ * and during the bottum-up-stepping an OpFilter containing these remained
+ * filterconditions will be inserted in the operator-tree.
+ * Conditions for moving down a filtercondition:
+ * (M1, M2 are graphpatterns, F is a filtercondition)
+ * 1) Filter(Union(M1, M2), F)) will become Union(Filter(M1, F), M2) when the
+ * filterexpression is only referenced to M1
+ * 2) Filter(Union(M1, M2), F)) will become Union(M1, Filter(M2, F)) when the
+ * filterexpression is only referenced to M2
+ * 3) Filter(Union(M1, M2), F)) will become Join(Union(M1, F), Union(M2, F)) when the
+ * filterexpression is referenced to M1 and M2
+ */
+ public void visit(OpUnion opUnion)
+ {
+ checkMoveDownFilterExprAndVisitOpUnion(opUnion);
+ }
+
+ /**
+ * When visiting an OpJoin 3 conditions for moving down the filterconditions are
+ * checked. Only when a condition is satisfied, the filtercondition can be moved
+ * down to the next operator in the tree. Otherwise the condition will stay here
+ * and during the bottum-up-stepping an OpFilter containing these remained
+ * filterconditions will be inserted in the operator-tree.
+ * Conditions for moving down a filtercondition:
+ * (M1, M2 are graphpatterns, F is a filtercondition)
+ * 1) Filter(Join(M1, M2), F)) will become Join(Filter(M1, F), M2) when the
+ * filterexpression is only referenced to M1
+ * 2) Filter(Join(M1, M2), F)) will become Join(M1, Filter(M2, F)) when the
+ * filterexpression is only referenced to M2
+ * 3) Filter(Join(M1, M2), F)) will become Join(Filter(M1, F), Filter(M2, F)) when the
+ * filterexpression is referenced to M1 and M2
+ */
+ public void visit(OpJoin opJoin)
+ {
+ checkMoveDownFilterExprAndVisitOpJoin(opJoin);
+ }
+
+
+ /**
+ * When there are some filterexpressions which belong to an OpBGP, the OpBGP will be converted
+ * to an OpFilteredBGP. A OpFilteredBGP is nearly the same like an OpBGP but it has
+ * a link to its parent, which is an OpFilter with the coresponding filter-conditions,
+ * because in the transforming-process of the OpBGPs to OpD2RQs a link to the above OpFilter
+ * is needed.
+ */
+ public void visit(OpBGP op)
+ {
+ Op newOp;
+
+ newOp = op;
+
+
+ // are there some filterexpression which belong to this BGP ?
+ if (!this.filterExpr.isEmpty())
+ {
+ // apply the optimizing rules
+ // create a filteredOpBGP and a filter
+ newOp = op.apply(optimizingTransform);
+ // add the current valid filterconditions
+ ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
+ }
+
+ if (newOp instanceof OpFilter)
+ {
+ // OpFilteredBGP must be transformed
+ newOp = ((OpFilter)newOp).getSubOp();
+ }
+
+ // apply the tranformation from an OpBGP/OpFilteredBGP to an OpD2RQ
+ newOp = ((OpBGP)newOp).apply(d2rqTransform);
+
+ this.stack.push(newOp);
+ }
+
+ /**
+ * When visiting an OpDiff 3 conditions for moving down the filterconditions are
+ * checked. Only when a condition is satisfied, the filtercondition can be moved
+ * down to the next operator in the tree. Otherwise the condition will stay here
+ * and during the bottum-up-stepping an OpFilter containing these remained
+ * filterconditions will be inserted in the operator-tree.
+ * Conditions for moving down a filtercondition:
+ * (M1, M2 are graphpatterns, F is a filtercondition)
+ * 1) Filter(Diff(M1, M2), F)) will become Diff(Filter(M1, F), M2) when the
+ * filterexpression is only referenced to M1
+ * 2) Filter(Diff(M1, M2), F)) will become Diff(M1, Filter(M2, F)) when the
+ * filterexpression is only referenced to M2
+ * 3) Filter(Diff(M1, M2), F)) will become Diff(Filter(M1, F), Filter(M2, F)) when the
+ * filterexpression is referenced to M1 and M2
+ */
+ public void visit(OpDiff opDiff)
+ {
+ // TODO: Regel nochmal ueberdenken !!!
+ checkMoveDownFilterExprAndVisitOpDiff(opDiff);
+ }
+
+ public void visit(OpConditional opCondition) {
+ notMoveDownFilterExprAndVisitOp2(opCondition); //TODO moving down / improvements possible!! Check this
+ }
+
+ public void visit(OpProcedure opProc)
+ {
+ notMoveDownFilterExprAndVisitOp1(opProc);
+ }
+
+ public void visit(OpPropFunc opPropFunc)
+ {
+ notMoveDownFilterExprAndVisitOp1(opPropFunc);
+ }
+
+ public void visit(OpTable opTable)
+ {
+ notMoveDownFilterExprAndVisitOp0(opTable);
+ }
+
+ public void visit(OpQuadPattern quadPattern)
+ {
+ notMoveDownFilterExprAndVisitOp0(quadPattern);
+ }
+
+ public void visit(OpPath opPath)
+ {
+ notMoveDownFilterExprAndVisitOp0(opPath);
+ }
+
+ public void visit(OpTriple opTriple)
+ {
+ notMoveDownFilterExprAndVisitOp0(opTriple);
+ }
+
+ public void visit(OpDatasetNames dsNames)
+ {
+ notMoveDownFilterExprAndVisitOp0(dsNames);
+ }
+
+ public void visit(OpSequence opSequence)
+ {
+ notMoveDownFilterExprAndVisitOpN(opSequence);
+ }
+
+ /**
+ * When visiting an OpJoin 2 conditions for moving down the filterconditions are
+ * checked. Only when a condition is satisfied, the filtercondition can be moved
+ * down to the next operator in the tree. Otherwise the condition will stay here
+ * and during the bottum-up-stepping an OpFilter containing these remained
+ * filterconditions will be inserted in the operator-tree.
+ * Conditions for moving down a filtercondition:
+ * (M1, M2 are graphpatterns, F is a filtercondition)
+ * 1) Filter(LeftJoin(M1, M2), F)) will become LeftJoin(Filter(M1, F), M2) when the
+ * filterexpression is only referenced to M1
+ * 2) Filter(LeftJoin(M1, M2), F)) will become LeftJoin(Filter(M1, F), Filter(M2, F)) when the
+ * filterexpression is referenced to M1 and M2
+ */
+ public void visit(OpLeftJoin opLeftJoin)
+ {
+ checkMoveDownFilterExprAndVisitOpLeftJoin(opLeftJoin);
+ }
+
+ public void visit(OpGraph opGraph)
+ {
+ notMoveDownFilterExprAndVisitOp1(opGraph);
+ }
+
+ public void visit(OpService opService)
+ {
+ notMoveDownFilterExprAndVisitOp1(opService);
+ }
+
+ public void visit(OpExt opExt)
+ {
+ Op newOp;
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!this.filterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = addFilterTransform.transform(opExt);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = opExt;
+ }
+
+ this.stack.push(newOp);
+ }
+
+ public void visit(OpNull opNull)
+ {
+ notMoveDownFilterExprAndVisitOp0(opNull);
+ }
+
+ public void visit(OpLabel opLabel)
+ {
+ Op newOp, subOp = null ;
+
+ // check for subtree
+ if (opLabel.getSubOp() != null )
+ {
+ opLabel.getSubOp().visit(this);
+ subOp = (Op)this.stack.pop();
+ }
+
+ // remove the label
+ newOp = opLabel.apply(d2rqTransform, subOp);
+ this.stack.push(newOp);
+
+ }
+
+ public void visit(OpList opList)
+ {
+ notMoveDownFilterExprAndVisitOp1(opList);
+ }
+
+ public void visit(OpOrder opOrder)
+ {
+ notMoveDownFilterExprAndVisitOp1(opOrder);
+ }
+
+ public void visit(OpProject opProject)
+ {
+ notMoveDownFilterExprAndVisitOp1(opProject);
+ }
+
+ public void visit(OpDistinct opDistinct)
+ {
+ notMoveDownFilterExprAndVisitOp1(opDistinct);
+ }
+
+ public void visit(OpReduced opReduced)
+ {
+ notMoveDownFilterExprAndVisitOp1(opReduced);
+ }
+
+ public void visit(OpAssign opAssign)
+ {
+ notMoveDownFilterExprAndVisitOp1(opAssign);
+ }
+
+ public void visit(OpSlice opSlice)
+ {
+ notMoveDownFilterExprAndVisitOp1(opSlice);
+ }
+
+ public void visit(OpGroupAgg opGroupAgg)
+ {
+ notMoveDownFilterExprAndVisitOp1(opGroupAgg);
+ }
+
+ /**
+ * Calculates the set of valid filterexpressions as a subset from the whole set of
+ * possibilities.
+ * @param possiblities - the whole set
+ * @param opLabel - contains the threated object-vars, which are used to determine
+ * the valid filterexpressions
+ * @return List - the subset from the set of possiblities
+ */
+ private List calcValidFilterExpr(List possiblities, OpLabel opLabel)
+ {
+ Set threatedObjectVars;
+ Expr expr;
+ Set mentionedVarsInExpr;
+ List result;
+
+ // get the set of the threated object-vars
+ threatedObjectVars = (Set) opLabel.getObject();
+
+ result = new ArrayList();
+
+ // now check every var of every filterexpression in the set of all
+ // filterexpression-possibilities to check if the filterexpression is valid
+ for (Iterator iterator = possiblities.iterator(); iterator.hasNext();)
+ {
+ expr = (Expr)iterator.next();
+
+ // get all in the expression mentioned object-vars
+ mentionedVarsInExpr = expr.getVarsMentioned();
+
+ // check if the alle mentioned-vars of the expression are a subset
+ // of the threated-Object-Vars
+ // if yes, the expression is valid, otherwise not
+ if (threatedObjectVars.containsAll(mentionedVarsInExpr))
+ {
+ result.add(expr);
+ }
+ }
+
+ return result;
+ }
+
+
+ /**
+ * Checks first if a filterexpression can be moved down.
+ * And after visits the operator
+ */
+ private void checkMoveDownFilterExprAndVisitOpUnion(OpUnion opUnion)
+ {
+ Op left = null ;
+ Op right = null ;
+ Op newOp;
+ OpLabel opLeftLabel, opRightLabel;
+ List filterExprBeforeOpUnion, filterExprAfterOpUnion, notMoveableFilterExpr;
+
+ // contains the filterexpressions that are valid before this op2
+ filterExprBeforeOpUnion = new ArrayList(this.filterExpr);
+ // contains the filterexpressions that are valid after this op2
+ // this is needed because during the bottom-up-stepping all filterexpressions
+ // which could not be transformed down, must be inserted by means of an OpFilter
+ // above this op2
+ filterExprAfterOpUnion = new ArrayList();
+
+ // check left subtree
+ if ((left = opUnion.getLeft()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // left subtree
+ if (left instanceof OpLabel)
+ {
+ opLeftLabel = (OpLabel)left;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnion, opLeftLabel);
+ filterExprAfterOpUnion.addAll(this.filterExpr);
+ }
+ // step down
+ opUnion.getLeft().visit(this);
+ left = (Op)this.stack.pop();
+ }
+
+ // check the right subtree
+ if ((right = opUnion.getRight()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // right subtree
+ if (right instanceof OpLabel)
+ {
+ opRightLabel = (OpLabel)right;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnion, opRightLabel);
+ filterExprAfterOpUnion.addAll(this.filterExpr);
+ }
+ // step down
+ opUnion.getRight().visit(this);
+ right = (Op)this.stack.pop();
+ }
+
+ // note: filterExprAfterOpUnion contains now all filterexpressions which could
+ // be moved down
+ // now calculate all filterexpressions which were not moveable
+ notMoveableFilterExpr = new ArrayList(filterExprBeforeOpUnion);
+ notMoveableFilterExpr.removeAll(filterExprAfterOpUnion);
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = opUnion.apply(addFilterTransform, left, right);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ newOp = opUnion.apply(d2rqTransform, left, right);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = filterExprBeforeOpUnion;
+
+ this.stack.push(newOp);
+ }
+
+ /**
+ * Checks first if a filterexpression can be moved down.
+ * And after visits the operator
+ */
+ private void checkMoveDownFilterExprAndVisitOpJoin(OpJoin opJoin)
+ {
+ Op left = null ;
+ Op right = null ;
+ Op newOp;
+ OpLabel opLeftLabel, opRightLabel;
+ List filterExprBeforeOpJoin, filterExprAfterOpJoin, notMoveableFilterExpr;
+
+ // contains the filterexpressions that are valid before this op2
+ filterExprBeforeOpJoin = new ArrayList(this.filterExpr);
+ // contains the filterexpressions that are valid after this op2
+ // this is needed because during the bottom-up-stepping all filterexpressions
+ // which could not be transformed down, must be inserted by means of an OpFilter
+ // above this op2
+ filterExprAfterOpJoin = new ArrayList();
+
+ // check left subtree
+ if ((left = opJoin.getLeft()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // left subtree
+ if (left instanceof OpLabel)
+ {
+ opLeftLabel = (OpLabel)left;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpJoin, opLeftLabel);
+ filterExprAfterOpJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opJoin.getLeft().visit(this);
+ left = (Op)this.stack.pop();
+ }
+
+ // check the right subtree
+ if ((right = opJoin.getRight()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // right subtree
+ if (right instanceof OpLabel)
+ {
+ opRightLabel = (OpLabel)right;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpJoin, opRightLabel);
+ filterExprAfterOpJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opJoin.getRight().visit(this);
+ right = (Op)this.stack.pop();
+ }
+
+ // note: filterExprAfterOpUnion contains now all filterexpressions which could
+ // be moved down
+ // now calculate all filterexpressions which were not moveable
+ notMoveableFilterExpr = new ArrayList(filterExprBeforeOpJoin);
+ notMoveableFilterExpr.removeAll(filterExprAfterOpJoin);
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = opJoin.apply(addFilterTransform, left, right);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = opJoin.apply(d2rqTransform, left, right);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = filterExprBeforeOpJoin;
+
+ this.stack.push(newOp);
+ }
+
+ /**
+ * Checks first if a filterexpression can be moved down.
+ * And after visits the operator
+ */
+ private void checkMoveDownFilterExprAndVisitOpLeftJoin(OpLeftJoin opLeftJoin)
+ {
+ Op left = null ;
+ Op right = null ;
+ Op newOp;
+ OpLabel opLeftLabel, opRightLabel;
+ List filterExprBeforeOpLeftJoin, filterExprAfterOpLeftJoin, notMoveableFilterExpr, filterExprRightSide, validFilterExprRightSide;
+ Expr expr;
+
+ // contains the filterexpressions that are valid before this op2
+ filterExprBeforeOpLeftJoin = new ArrayList(this.filterExpr);
+ // contains the filterexpressions that are valid after this op2
+ // this is needed because during the bottom-up-stepping all filterexpressions
+ // which could not be transformed down, must be inserted by means of an OpFilter
+ // above this op2
+ filterExprAfterOpLeftJoin = new ArrayList();
+
+
+ // check left subtree
+ if ((left = opLeftJoin.getLeft()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // left subtree
+ if (left instanceof OpLabel)
+ {
+ opLeftLabel = (OpLabel)left;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpLeftJoin, opLeftLabel);
+ filterExprAfterOpLeftJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opLeftJoin.getLeft().visit(this);
+ left = (Op)this.stack.pop();
+ }
+
+ // check the right subtree
+ if ((right = opLeftJoin.getRight()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // right subtree
+ if (right instanceof OpLabel)
+ {
+ opRightLabel = (OpLabel)right;
+
+ filterExprRightSide = calcValidFilterExpr(filterExprBeforeOpLeftJoin, opRightLabel);
+ validFilterExprRightSide = new ArrayList();
+
+ // now check for expr that are vaild for left-side and right-side
+ // only when an expr is vaid for left-side, it can be vaild for right-side
+ for(Iterator iterator = filterExprRightSide.iterator(); iterator.hasNext();)
+ {
+ expr = (Expr)iterator.next();
+
+ if (this.filterExpr.contains(expr))
+ {
+ // valid
+ validFilterExprRightSide.add(expr);
+ }
+ }
+
+ this.filterExpr = validFilterExprRightSide;
+ filterExprAfterOpLeftJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opLeftJoin.getRight().visit(this);
+ right = (Op)this.stack.pop();
+ }
+
+
+
+ // note: filterExprAfterOpUnion contains now all filterexpressions which could
+ // be moved down
+ // now calculate all filterexpressions which were not moveable
+ notMoveableFilterExpr = new ArrayList(filterExprBeforeOpLeftJoin);
+ notMoveableFilterExpr.removeAll(filterExprAfterOpLeftJoin);
+
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter for an opleftjoin
+ newOp = opLeftJoin.apply(addFilterTransform, left, right);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = opLeftJoin.apply(d2rqTransform, left, right);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = filterExprBeforeOpLeftJoin;
+
+ this.stack.push(newOp);
+ }
+
+
+ /**
+ * Checks first if a filterexpression can be moved down.
+ * And after visits the operator
+ */
+ private void checkMoveDownFilterExprAndVisitOpDiff(Op2 opDiff)
+ {
+ Op left = null ;
+ Op right = null ;
+ Op newOp;
+ OpLabel opLeftLabel, opRightLabel;
+ List filterExprBeforeOpUnionOpJoin, filterExprAfterOpUnionOpJoin, notMoveableFilterExpr;
+
+ // contains the filterexpressions that are valid before this op2
+ filterExprBeforeOpUnionOpJoin = new ArrayList(this.filterExpr);
+ // contains the filterexpressions that are valid after this op2
+ // this is needed because during the bottom-up-stepping all filterexpressions
+ // which could not be transformed down, must be inserted by means of an OpFilter
+ // above this op2
+ filterExprAfterOpUnionOpJoin = new ArrayList();
+
+ // check left subtree
+ if ((left = opDiff.getLeft()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // left subtree
+ if (left instanceof OpLabel)
+ {
+ opLeftLabel = (OpLabel)left;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnionOpJoin, opLeftLabel);
+ filterExprAfterOpUnionOpJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opDiff.getLeft().visit(this);
+ left = (Op)this.stack.pop();
+ }
+
+ // check the right subtree
+ if ((right = opDiff.getRight()) != null )
+ {
+ // calculate the set of filterexpressions that are also valid for the
+ // right subtree
+ if (right instanceof OpLabel)
+ {
+ opRightLabel = (OpLabel)right;
+ this.filterExpr = calcValidFilterExpr(filterExprBeforeOpUnionOpJoin, opRightLabel);
+ filterExprAfterOpUnionOpJoin.addAll(this.filterExpr);
+ }
+ // step down
+ opDiff.getRight().visit(this);
+ right = (Op)this.stack.pop();
+ }
+
+ // note: filterExprAfterOpUnion contains now all filterexpressions which could
+ // be moved down
+ // now calculate all filterexpressions which were not moveable
+ notMoveableFilterExpr = new ArrayList(filterExprBeforeOpUnionOpJoin);
+ notMoveableFilterExpr.removeAll(filterExprAfterOpUnionOpJoin);
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = opDiff.apply(addFilterTransform, left, right);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = opDiff.apply(d2rqTransform, left, right);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = filterExprBeforeOpUnionOpJoin;
+
+ this.stack.push(newOp);
+ }
+
+
+ /**
+ * Handels and visits all op0 for which no filterexpression can be moved down.
+ * @param op0 - an Op0
+ */
+ private void notMoveDownFilterExprAndVisitOp0(Op0 op0)
+ {
+ Op newOp;
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!this.filterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = op0.apply(addFilterTransform);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(this.filterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = op0.apply(d2rqTransform);
+ }
+
+ this.stack.push(newOp);
+ }
+
+ /**
+ * Handels and visits all op1 for which no filterexpression can be moved down.
+ * @param op1 - an Op1
+ */
+ private void notMoveDownFilterExprAndVisitOp1(Op1 op1)
+ {
+ List notMoveableFilterExpr;
+ Op newOp, subOp = null ;
+
+ // all available FilterExpr could not be moved down
+ notMoveableFilterExpr = new ArrayList(this.filterExpr);
+ // clear the filterExpr
+ filterExpr.clear();
+
+ // check for subtree
+ if (op1.getSubOp() != null )
+ {
+ op1.getSubOp().visit(this);
+ subOp = (Op)this.stack.pop();
+ }
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = op1.apply(addFilterTransform, subOp);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ newOp = op1.apply(d2rqTransform, subOp);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = notMoveableFilterExpr;
+ this.stack.push(newOp);
+ }
+
+ /**
+ * Handels and visits all op2 for which no filterexpression can be moved down.
+ * @param op2 - an Op2
+ */
+ private void notMoveDownFilterExprAndVisitOp2(Op2 op2)
+ {
+ List notMoveableFilterExpr;
+ Op left = null ;
+ Op right = null ;
+ Op newOp;
+
+ // all available FilterExpr could not be moved down
+ notMoveableFilterExpr = new ArrayList(this.filterExpr);
+ // clear the filterExpr
+ filterExpr.clear();
+
+ // check for left subtree
+ if (op2.getLeft() != null )
+ {
+ // step down
+ op2.getLeft().visit(this);
+ left = (Op)this.stack.pop();
+ }
+ if (op2.getRight() != null )
+ {
+ // step down
+ op2.getRight().visit(this);
+ right = (Op)this.stack.pop();
+ }
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = op2.apply(addFilterTransform, left, right);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = op2.apply(d2rqTransform, left, right);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = notMoveableFilterExpr;
+ this.stack.push(newOp);
+ }
+
+ /**
+ * Handels and visits all opN for which no filterexpression can be moved down.
+ * @param opN - an OpN
+ */
+ private void notMoveDownFilterExprAndVisitOpN(OpN opN)
+ {
+ List list;
+ List notMoveableFilterExpr;
+ Op subOp, op, newOp;
+
+ // all available FilterExpr could not be moved down
+ notMoveableFilterExpr = new ArrayList(this.filterExpr);
+ // clear the filterExpr
+ filterExpr.clear();
+
+ list = new ArrayList(opN.size()) ;
+
+ // check for subtrees
+ for (Iterator iterator = opN.iterator(); iterator.hasNext();)
+ {
+ subOp = (Op)iterator.next();
+ subOp.visit(this);
+
+ op = (Op)this.stack.pop();
+ if (op != null)
+ {
+ list.add(op);
+ }
+ }
+
+ // if there are some filterexpressions which could not be moved down,
+ // an opFilter must be inserted that contains this filterexpressions
+ if (!notMoveableFilterExpr.isEmpty())
+ {
+ // create the filter
+ newOp = opN.apply(addFilterTransform, list);
+ // add the conditions
+ ((OpFilter)newOp).getExprs().getList().addAll(notMoveableFilterExpr);
+ }else
+ {
+ // nothing must be done
+ newOp = opN.apply(d2rqTransform, list);
+ }
+
+ // restore filterexpressions
+ this.filterExpr = notMoveableFilterExpr;
+ this.stack.push(newOp);
+ }
+
+ }
+
+}
diff --git a/src/de/fuberlin/wiwiss/d2rq/sql/ResultRowMap.java b/src/de/fuberlin/wiwiss/d2rq/sql/ResultRowMap.java
index 39616637..0ab0aa26 100644
--- a/src/de/fuberlin/wiwiss/d2rq/sql/ResultRowMap.java
+++ b/src/de/fuberlin/wiwiss/d2rq/sql/ResultRowMap.java
@@ -25,18 +25,18 @@ public class ResultRowMap implements ResultRow {
public static ResultRowMap fromResultSet(ResultSet resultSet, List projectionSpecs) throws SQLException {
Map result = new HashMap();
- ResultSetMetaData metaData = resultSet.getMetaData();
-
+ ResultSetMetaData metaData = resultSet.getMetaData();
+
for (int i = 0; i < projectionSpecs.size(); i++) {
/*
* Return string representations of the values using information from the type map
*
* TODO Generally use resultSet.getObject(i+1).toString() instead of resultSet.getString(i+1); maybe even map to Objects instead of Strings?
* This would convert at JDBC/Java level, which will likely differ from current data, so it's probably best to keep things as they are for now
- */
- if (metaData != null) {
- String classString = metaData.getColumnClassName(i + 1);
- /*
+ */
+ if (metaData != null) {
+ String classString = metaData.getColumnClassName(i + 1);
+ /*
* Specifically handle Oracle DATEs and TIMESTAMPs because the regular getString()
* returns them in non-standard fashion, e.g. "2008-3-22.0.0. 0. 0".
* This occurs independently of the NLS_DATE_FORMAT / NLS_TIMESTAMP_FORMAT in use.
@@ -59,20 +59,20 @@ public static ResultRowMap fromResultSet(ResultSet resultSet, List projectionSpe
}
}
/*
- * Let the JDBC driver convert boolean values for us as their representation differs greatly amongst DBs (e.g. PostgreSQL employs 't' and 'f', others use 0 and 1)
+ * Let the JDBC driver convert boolean values for us as their representation differs greatly amongst DBs (e.g. PostgreSQL employs 't' and 'f', others use 0 and 1)
*/
- else if (classString != null && classString.equals("java.lang.Boolean"))
+ else if (classString != null && classString.equals("java.lang.Boolean"))
result.put(projectionSpecs.get(i), Boolean.toString(resultSet.getBoolean(i + 1)));
- else {
- /*
- * Return native string representation of the object
- */
+ else {
+ /*
+ * Return native string representation of the object
+ */
result.put(projectionSpecs.get(i), resultSet.getString(i + 1));
}
- }
- else
- result.put(projectionSpecs.get(i), resultSet.getString(i + 1)); /* treat everything as String if no type map is available */
- }
+ }
+ else
+ result.put(projectionSpecs.get(i), resultSet.getString(i + 1)); /* treat everything as String if no type map is available */
+ }
return new ResultRowMap(result);
}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-1.rq
index 45f04e2e..b8e7049d 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-1.rq
@@ -1,8 +1,8 @@
-PREFIX :
-
-SELECT ?x
-{
- ?x ?p '''manner gatemen'''
-}
-
-
+PREFIX :
+
+SELECT ?x
+{
+ ?x ?p '''manner gatemen'''
+}
+
+
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-2.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-2.rq
index 5d10afd2..19a66441 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-2.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/basic/quotes-2.rq
@@ -1,8 +1,8 @@
-PREFIX :
-
-SELECT ?x
-{
- ?x ?p """coterie ahchoo"""
-}
-
-
+PREFIX :
+
+SELECT ?x
+{
+ ?x ?p """coterie ahchoo"""
+}
+
+
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/distinct/distinct-star-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/distinct/distinct-star-1.rq
index 17dcd615..e31c43f1 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/distinct/distinct-star-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/distinct/distinct-star-1.rq
@@ -1,15 +1,15 @@
-PREFIX :
-PREFIX xsd:
-
-SELECT DISTINCT *
-WHERE
-{
- {
- ?s :rating2 ?o
- }
- UNION
- {
- ?s :rating1 ?o
- }
-}
-
+PREFIX :
+PREFIX xsd:
+
+SELECT DISTINCT *
+WHERE
+{
+ {
+ ?s :rating2 ?o
+ }
+ UNION
+ {
+ ?s :rating1 ?o
+ }
+}
+
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-ge-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-ge-1.rq
index dd8a5bef..971f29d8 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-ge-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-ge-1.rq
@@ -1,8 +1,8 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating1 ?o .
- FILTER(?o >= 3) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating1 ?o .
+ FILTER(?o >= 3) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-le-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-le-1.rq
index 69b109a7..fb1cb0c6 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-le-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-le-1.rq
@@ -1,8 +1,8 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating2 ?o .
- FILTER(?o <= 7) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating2 ?o .
+ FILTER(?o <= 7) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-minus-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-minus-1.rq
index 5ff3bdef..c39969c3 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-minus-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-minus-1.rq
@@ -1,9 +1,9 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating1 ?o .
- ?s2 :rating1 ?o2 .
- FILTER(?o - ?o2 = 3) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating1 ?o .
+ ?s2 :rating1 ?o2 .
+ FILTER(?o - ?o2 = 3) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-mul-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-mul-1.rq
index 52e2910d..d4d8d875 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-mul-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-mul-1.rq
@@ -1,9 +1,9 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating1 ?o .
- ?s2 :rating1 ?o2 .
- FILTER(?o * ?o2 = 2) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating1 ?o .
+ ?s2 :rating1 ?o2 .
+ FILTER(?o * ?o2 = 2) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-plus-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-plus-1.rq
index 62c28731..5617dd8d 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-plus-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-plus-1.rq
@@ -1,9 +1,9 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating1 ?o .
- ?s2 :rating1 ?o2 .
- FILTER(?o + ?o2 = 5) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating1 ?o .
+ ?s2 :rating1 ?o2 .
+ FILTER(?o + ?o2 = 5) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unminus-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unminus-1.rq
index 4f5569a8..2406fb9f 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unminus-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unminus-1.rq
@@ -1,7 +1,7 @@
-PREFIX :
-
-SELECT ?s WHERE
-{
- ?s :rating1 ?o .
- FILTER(-?o = -3) .
-}
+PREFIX :
+
+SELECT ?s WHERE
+{
+ ?s :rating1 ?o .
+ FILTER(-?o = -3) .
+}
diff --git a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unplus-1.rq b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unplus-1.rq
index 4f77c3b0..73f2d9a7 100644
--- a/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unplus-1.rq
+++ b/test/de/fuberlin/wiwiss/d2rq/d2rq_sdb/queries/dawg/expr-ops/query-unplus-1.rq
@@ -1,8 +1,8 @@
-PREFIX :
-
-SELECT ?s
-WHERE
-{
- ?s :rating1 ?o .
- FILTER(?o = +3) .
-}
+PREFIX :
+
+SELECT ?s
+WHERE
+{
+ ?s :rating1 ?o .
+ FILTER(?o = +3) .
+}
diff --git a/webapp/WEB-INF/web.xml b/webapp/WEB-INF/web.xml
index eac31cd1..92601c0b 100644
--- a/webapp/WEB-INF/web.xml
+++ b/webapp/WEB-INF/web.xml
@@ -1,139 +1,139 @@
-
-
-
-
- D2R Server
-
-
-
- configFile
- config-example.n3
-
-
-
- de.fuberlin.wiwiss.d2rq.server.WebappInitListener
-
-
-
-
-
-
- SPARQL service processor
- org.joseki.http.Servlet
-
-
-
- ResourceServlet
- de.fuberlin.wiwiss.d2rq.server.ResourceServlet
- 1
-
-
-
- ClassMapServlet
- de.fuberlin.wiwiss.d2rq.server.ClassMapServlet
-
-
-
- PageServlet
- de.fuberlin.wiwiss.d2rq.server.PageServlet
-
- org.apache.velocity.properties
- /WEB-INF/velocity.properties
-
-
-
-
- ResourceDescriptionServlet
- de.fuberlin.wiwiss.d2rq.server.ResourceDescriptionServlet
-
-
-
- DirectoryServlet
- de.fuberlin.wiwiss.d2rq.server.DirectoryServlet
-
- org.apache.velocity.properties
- /WEB-INF/velocity.properties
-
-
-
-
- RootServlet
- de.fuberlin.wiwiss.d2rq.server.RootServlet
-
- org.apache.velocity.properties
- /WEB-INF/velocity.properties
-
-
-
-
- NamespaceServlet
- de.fuberlin.wiwiss.d2rq.server.NamespaceServlet
-
-
-
-
-
- SPARQL service processor
- /sparql
-
-
-
- ResourceServlet
- /resource/*
-
-
-
- ResourceServlet
- /vocab/resource/*
-
-
-
- ClassMapServlet
- /all/*
-
-
-
- ClassMapServlet
- /all
-
-
-
- PageServlet
- /page/*
-
-
-
- PageServlet
- /vocab/page/*
-
-
-
- ResourceDescriptionServlet
- /data/*
-
-
-
- ResourceDescriptionServlet
- /vocab/data/*
-
-
-
- DirectoryServlet
- /directory/*
-
-
-
- RootServlet
- /index.html
-
-
-
- NamespaceServlet
- /snorql/namespaces.js
-
-
-
- index.html
-
-
+
+
+
+
+ D2R Server
+
+
+
+ configFile
+ config-example.n3
+
+
+
+ de.fuberlin.wiwiss.d2rq.server.WebappInitListener
+
+
+
+
+
+
+ SPARQL service processor
+ org.joseki.http.Servlet
+
+
+
+ ResourceServlet
+ de.fuberlin.wiwiss.d2rq.server.ResourceServlet
+ 1
+
+
+
+ ClassMapServlet
+ de.fuberlin.wiwiss.d2rq.server.ClassMapServlet
+
+
+
+ PageServlet
+ de.fuberlin.wiwiss.d2rq.server.PageServlet
+
+ org.apache.velocity.properties
+ /WEB-INF/velocity.properties
+
+
+
+
+ ResourceDescriptionServlet
+ de.fuberlin.wiwiss.d2rq.server.ResourceDescriptionServlet
+
+
+
+ DirectoryServlet
+ de.fuberlin.wiwiss.d2rq.server.DirectoryServlet
+
+ org.apache.velocity.properties
+ /WEB-INF/velocity.properties
+
+
+
+
+ RootServlet
+ de.fuberlin.wiwiss.d2rq.server.RootServlet
+
+ org.apache.velocity.properties
+ /WEB-INF/velocity.properties
+
+
+
+
+ NamespaceServlet
+ de.fuberlin.wiwiss.d2rq.server.NamespaceServlet
+
+
+
+
+
+ SPARQL service processor
+ /sparql
+
+
+
+ ResourceServlet
+ /resource/*
+
+
+
+ ResourceServlet
+ /vocab/resource/*
+
+
+
+ ClassMapServlet
+ /all/*
+
+
+
+ ClassMapServlet
+ /all
+
+
+
+ PageServlet
+ /page/*
+
+
+
+ PageServlet
+ /vocab/page/*
+
+
+
+ ResourceDescriptionServlet
+ /data/*
+
+
+
+ ResourceDescriptionServlet
+ /vocab/data/*
+
+
+
+ DirectoryServlet
+ /directory/*
+
+
+
+ RootServlet
+ /index.html
+
+
+
+ NamespaceServlet
+ /snorql/namespaces.js
+
+
+
+ index.html
+
+
diff --git a/webapp/snorql/xml-to-html.xsl b/webapp/snorql/xml-to-html.xsl
index 3f9e83d7..13330278 100644
--- a/webapp/snorql/xml-to-html.xsl
+++ b/webapp/snorql/xml-to-html.xsl
@@ -1,183 +1,183 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
