SPARQL endpoint

[mathias-goebel]

may with the rdf indexer by ljo: https://github.com/ljo/exist-sparql

[ingoboerner]

Hab' mit @lehkost über Wikidata-SPARQL-Abfragen über das dracor-Datenset gesprochen; Endpoint über eXist wäre wirklich sehr cool, momentan geht's mit externem Endpoint:

Metadaten aus gerdracor mit Wikidata-Verknüpfung als RDF (nur mal als Vorschlag, vielleicht gibt's da bei den einzelnen Properties bessere Vorschläge..)

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix pnd: <http://d-nb.info/gnd/> .
# GND- Schema: https://d-nb.info/standards/elementset/gnd ?
@prefix dc: <http://purl.org/dc/elements/1.1/> .
# eigenes dracor-Schema, wenn bspw. Netzwerkmaße usw. auch als LOD? :
@prefix dracor: <https://dracor.org/lod/schema#> .

# Beispieldatensatz

<https://dracor.org/ger/alberti-brot> a dracor:play ;
  dracor:id "alberti-brot" ;
  dc:title "Brot! Ein soziales Schauspiel in fünf Akten" ;
  dc:creator pnd:116009926 ;
  # wikidata?
  dracor:printYear "1888"^^xsd:gYear ;
  # dracor:premiereYear gibt's bei diesem Datensatz nicht
  dracor:writtenYear "1888"^^xsd:gYear ;
  owl:sameAs <http://www.wikidata.org/entity/Q51370104> .

Hab' die Daten aus der gerdracor/wikidata-Liste angelegt, allerdings nur in dieser Form (@lehkost, wie ging das nochmal per API, damit ich an die Metadaten für alle Stücke komm'?):

<https://dracor.org/ger/alberti-im-suff> a dracor:play ;
 owl:sameAs <http://www.wikidata.org/entity/Q51370930> .

Minimalversion der Verknüpfung:

<https://dracor.org/ger/alberti-im-suff> owl:sameAs <http://www.wikidata.org/entity/Q51370930> .
Wikidata unterstützt Federated Queries allerdings nur für registrierte Endpoints; zwischenzeitlich braucht's also einen eigenen Endpoint, über den der Query läuft und der dann die Daten zusammenführt.

Hab' dazu die Daten testweise in einen Triple Store eingespielt (Jena-Fuseki als Docker Container). SPARQL Endpoint ist hier (experimentell):

(http://digitalhumanities.germ.univie.ac.at/jena/gerdracor/query)

Idealerweise würde natürlich die eXist-DB den SPARQL Endpoint bereitstellen.

Query-Interface (YASGUI):
Das ließe sich sehr einfach in dracor einbinden:

(https://gist.github.com/ingoboerner/5df5118814e6dc4322376da52c591dc7) und würde dann so aussehen:

(https://rawgit.com/ingoboerner/5df5118814e6dc4322376da52c591dc7/raw/86dd11da46ea38be748874c1aa8064189c104018/sparql.html)

Funktioniert allerdings nur, wenn unsichere Skripte im Browser zugelassen werden, weil der SPARQL-Endpoint nicht über https:// läuft.

Mit folgendem Query kann man dann die Aufführungsorte (P4647) und die jeweiligen Koordinaten aus wikidata ziehen:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX dracor: <https://dracor.org/lod/schema#>

SELECT ?play ?wikidata ?ort ?coords WHERE {
?play a dracor:play;
    owl:sameAs ?wikidata .
  
  SERVICE <https://query.wikidata.org/sparql> {
 		?wikidata wdt:P4647 ?ort .
   	OPTIONAL {?ort wdt:P625 ?coords}
 		}  
  
} 
LIMIT 1000

Kartendarstellung funktionier mit YASGUI:
Bsp hier: unsichere Skripts zulassen und Query ausführen

[mathias-goebel]

using easyrdf seems to generate no overhead on our side. see https://gist.github.com/mathias-goebel/aba533f715bd616f9be8903a69d41e28

[ingoboerner]

Dockercontainer: stain/jena-fuseki:latest

[ingoboerner]

Generate Basic RDF:
https://gist.github.com/ingoboerner/f3be4de00f1583c4a1d74a265ac9d0ed

[ingoboerner]

as API-Function: ingoboerner@a578401

[ingoboerner]

I skeched an ontology for dracor that would allow us to also provide the network-mesures as rdf:
http://www.visualdataweb.de/webvowl/#iri=https://raw.githubusercontent.com/dracor-org/dracor-schema/master/ontology/dracor-ontology.xml
@cmil, @mathias-goebel , @lehkost what would be the base-uri of the ontology. I'm using https://dracor.org/ontology at the moment, which is probably not the best option

[ingoboerner]

https://www.w3.org/TR/cooluris/

[cmil]

@ingoboerner The installation of the ontology has been added to the deployment and it's available at https://dracor.org/ontology.

[ingoboerner]

the sparql-endpoint in the api is implemented in a way, that it returns the results in xml, instead of json. maybe we could even leave it this way for now and change the configuration of the sparql query interface yasgui, that could be implemented in the frontend. it seems that yasgui can also handle the results in xml; it can be even changed in the settings tab here:

...and maybe it's possible to preconfigure it. See documentation here: http://yasqe.yasgui.org/doc/
acceptHeaderSelect: String|function (default: "application/sparql-results+json,*/*;q=0.9")
the accept-header on this would be application/sparql-results+json. I think, the results we get back from the xquery sparql module already conform to the format: https://www.w3.org/TR/rdf-sparql-XMLres/

[cmil]

While an api:sparql method has been introduced with PR #36 it does not quite work set. When POSTing a query to the /sparql endpoint, it throws the following error in exist.log:

QueryException: Lexical error at line 1, column 108.  Encountered: "/" (47), after : "UFJFRklYIG5zOiAgICA8aHR0cDovL25hbWVzcGFjZS5vcmcvb250b2xvZ3kvPgpTRUxFQ1QgP3MgP2FnZQpXSEVSRSB7ID9zIG5zOmFnZSA"

It seems that the posted query needs to be decoded before it can be used.

[ingoboerner]

I tried to change the function that generates the rdf in load.xql with commit 4f72dc9 (maybe take only the code that concerns this function and undo the rest? sorry);
RDF should look as follows:

<rdf:RDF xml:lang="en" 
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
    xmlns:owl="http://www.w3.org/2002/07/owl#"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:dracon="http://dracor.org/ontology#">
    <rdf:Description rdf:about="https://dracor.org/rus/andreyev-mysl">
        
        <rdfs:label xml:lang="en">Andreyev, Leonid: A Thought</rdfs:label>
        
        <rdfs:label xml:lang="ru">Андреев, Леонид Николаевич: Мысль</rdfs:label>
        
        <dc:creator xml:lang="en">Leonid Andreev</dc:creator>
        <dc:creator xml:lang="ru">Леонид Николаевич Андреев</dc:creator>
        
        <dc:title xml:lang="en">A Thought</dc:title>
        <dc:title xml:lang="ru">Мысль</dc:title>
        
        <!-- http://dracor.org/ontology#normalisedYear -->
        <!-- http://dracor.org/ontology#premiereYear -->
        <!-- http://dracor.org/ontology#printYear -->
        <!-- http://dracor.org/ontology#writtenYear -->
        
        
        <!-- Author as blank node -->
        <dracon:has_author>
            <rdf:Description>
                <rdfs:label xml:lang="en">Leonid Andreev</rdfs:label>
                <rdfs:label xml:lang="ru">Леонид Николаевич Андреев</rdfs:label>
                <owl:sameAs rdf:resource="http://www.wikidata.org/entity/Q310866"/>
            </rdf:Description>
        </dracon:has_author>
        
       <!-- network-measures -->
        
        <!-- http://dracor.org/ontology#averageClustering -->
        <!-- http://dracor.org/ontology#averageDegree -->
        <!-- http://dracor.org/ontology#averagePathLength -->
        <!-- http://dracor.org/ontology#density -->
        <!-- http://dracor.org/ontology#diameter -->
        <!-- http://dracor.org/ontology#maxDegree -->
        <!-- http://dracor.org/ontology#maxDegreeIds -->
        <!-- http://dracor.org/ontology#numOfActs -->
        <!-- http://dracor.org/ontology#numOfSegments -->
        <!-- http://dracor.org/ontology#numOfSpeakers -->
        
        
        <dracon:in_corpus rdf:resource="https://dracor.org/rus"/>
        
        <owl:sameAs rdf:resource="http://www.wikidata.org/entity/Q59355429"/>
        
    </rdf:Description>
</rdf:RDF>

I generate a blank node for the author because within dracor it does not have its own dracor-id; I tested example-rdf in my local installation of Jena and it worked.

[ingoboerner]

@cmil one more thing, is there a way to access the metrics when I generate the RDF from within the load-module? Is it stats:get-network-metrics($url as xs:string) in stats.xqm?

[cmil]

As of 0.35.1 we have a working exist-sparql integration, so I consider this done. For the record:

$ http -pb https://dracor.org/api/info
{
    "existdb": "4.5.0",
    "name": "DraCor API",
    "status": "alpha",
    "version": "0.35.1"
}

$ echo "SELECT * WHERE {?s ?p ?o} LIMIT 10" | http -v https://dracor.org/api/sparql Content-Type:application/sparql-query
POST /api/sparql HTTP/1.1
Accept: application/json, */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 35
Content-Type: application/sparql-query
Host: dracor.org
User-Agent: HTTPie/1.0.0

SELECT * WHERE {?s ?p ?o} LIMIT 10

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Authorization, Origin, X-Requested-With, Content-Type, Accept, DNT
Access-Control-Allow-Methods: GET, POST, OPTIONS, HEAD
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/sparql-results+xml; charset=UTF-8
Date: Thu, 10 Jan 2019 18:15:04 GMT
Server: nginx/1.10.3 (Ubuntu)
Transfer-Encoding: chunked

<sparql xmlns="http://www.w3.org/2005/sparql-results#">
    <head>
        <variable name="s"/>
        <variable name="p"/>
        <variable name="o"/>
    </head>
    <results>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/hebbel-gyges-und-sein-ring</uri>
            </binding>
            <binding name="p">
                <uri>http://dracor.org/ontology#in_corpus</uri>
            </binding>
            <binding name="o">
                <uri>https://dracor.org/ger</uri>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/hebbel-gyges-und-sein-ring</uri>
            </binding>
            <binding name="p">
                <uri>http://dracor.org/ontology#has_author</uri>
            </binding>
            <binding name="o">
                <bnode>-3e34d9c9:16838f8fd65:-7fff</bnode>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/hebbel-gyges-und-sein-ring</uri>
            </binding>
            <binding name="p">
                <uri>http://purl.org/dc/elements/1.1/creator</uri>
            </binding>
            <binding name="o">
                <literal>Hebbel, Friedrich</literal>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/hebbel-gyges-und-sein-ring</uri>
            </binding>
            <binding name="p">
                <uri>http://www.w3.org/2002/07/owl#sameAs</uri>
            </binding>
            <binding name="o">
                <uri>http://www.wikidata.org/entity/Q56439734</uri>
            </binding>
        </result>
        <result>
            <binding name="s">
                <bnode>-3e34d9c9:16838f8fd65:-7fff</bnode>
            </binding>
            <binding name="p">
                <uri>http://www.w3.org/2002/07/owl#sameAs</uri>
            </binding>
            <binding name="o">
                <uri>http://d-nb.info/gnd/118547445</uri>
            </binding>
        </result>
        <result>
            <binding name="s">
                <bnode>-3e34d9c9:16838f8fd65:-7fff</bnode>
            </binding>
            <binding name="p">
                <uri>http://www.w3.org/2000/01/rdf-schema#label</uri>
            </binding>
            <binding name="o">
                <literal>Hebbel, Friedrich</literal>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/quistorp-der-hypochondrist</uri>
            </binding>
            <binding name="p">
                <uri>http://dracor.org/ontology#in_corpus</uri>
            </binding>
            <binding name="o">
                <uri>https://dracor.org/ger</uri>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/quistorp-der-hypochondrist</uri>
            </binding>
            <binding name="p">
                <uri>http://dracor.org/ontology#has_author</uri>
            </binding>
            <binding name="o">
                <bnode>-3e34d9c9:16838f8fd65:-7ffe</bnode>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/quistorp-der-hypochondrist</uri>
            </binding>
            <binding name="p">
                <uri>http://purl.org/dc/elements/1.1/creator</uri>
            </binding>
            <binding name="o">
                <literal>Quistorp, Theodor Johann</literal>
            </binding>
        </result>
        <result>
            <binding name="s">
                <uri>https://dracor.org/ger/quistorp-der-hypochondrist</uri>
            </binding>
            <binding name="p">
                <uri>http://www.w3.org/2002/07/owl#sameAs</uri>
            </binding>
            <binding name="o">
                <uri>http://www.wikidata.org/entity/Q51406156</uri>
            </binding>
        </result>
    </results>
</sparql>

Feel free to open new issues to deal with remaining problems.