-
Notifications
You must be signed in to change notification settings - Fork 147
Description
TL;DR - LDH bootstrap fails against QLever: LDH posts TriG dataset to a:quadStore, but QLever only accepts SPARQL Update or graph-scoped GSP (400 “application/trig not supported”)
What I’m trying to do
Run LinkedDataHub (LDH) via the Docker Compose example, but use QLever as the backing store instead of the bundled Fuseki instances.
I configured the root admin and end-user services in config/system.trig to point at QLever (running in Kubernetes, port-forwarded to my laptop). From Docker, it’s reachable at:
http://host.docker.internal:7001
Note
I'm using k8s but that's just because I've already indexed a bunch of data into the graph database on k8s so it was just more convenient to portforward from k8s. The same general result should occur when just running Qlever locally on Docker.
My end goal is to use LDH as a CMS to edit the data indexed on Qlever and to host it on K8s in a small VPS.
What happened
LDH reaches the QLever URL, then fails during bootstrap when it tries to load the default datasets:
- LDH logs: “Loading end-user dataset into the triplestore...”
- then
curl: (22) ... 400 - container exits with code 22
Startup log excerpt
### Quad store URL of the root end-user service: [http://host.docker.internal:7001/](http://host.docker.internal:7001/) or [http://localhost:7001/](http://localhost:7001/) (due to port forwarding)
### Quad store URL of the root admin service: [http://host.docker.internal:7001/](http://host.docker.internal:7001/) or [http://localhost:7001/](http://localhost:7001/) (due to port forwarding)
### Waiting for [http://host.docker.internal:7001/](http://host.docker.internal:7001/)...
### URL [http://host.docker.internal:7001/](http://host.docker.internal:7001/) responded
### Loading end-user dataset into the triplestore...
curl: (22) The requested URL returned error: 400
linkeddatahub-1 exited with code 22
Sanity checks: what works with QLever
1) SPARQL query works
From inside the LDH container, SPARQL query works:
curl -s \
-H 'Content-Type: application/sparql-query' \
-H 'Accept: application/sparql-results+json' \
--data 'SELECT * WHERE { ?s ?p ?o } LIMIT 1' \
'http://host.docker.internal:7001'2) Graph Store Protocol (single graph) write works with token + ?graph=
export QLEVER_TOKEN="…"
printf '<urn:s> <urn:p> "o" .\n' > /tmp/one.ttl
curl -i -X PUT \
-H "Content-Type: text/turtle" \
-H "Authorization: Bearer $QLEVER_TOKEN" \
--data-binary @/tmp/one.ttl \
"http://host.docker.internal:7001?graph=urn:test"This returns HTTP/1.1 200 OK.
3) SPARQL Update write works with token
export QLEVER_TOKEN="…"
curl -i -X POST \
-H "Content-Type: application/sparql-update" \
-H "Authorization: Bearer $QLEVER_TOKEN" \
--data 'INSERT DATA { GRAPH <urn:test> { <urn:s> <urn:p> "o" } }' \
"http://host.docker.internal:7001"This returns HTTP/1.1 200 OK.
Repro: why bootstrap fails (QLever rejects dataset-level TriG POST)
LDH bootstrap appears to be attempting a dataset-level upload (TriG). Reproducing directly:
curl -i -X POST \
-H "Content-Type: application/trig" \
--data-binary @./platform/datasets/end-user.trig \
"http://host.docker.internal:7001/"Response from QLever:
HTTP/1.1 400 Bad Request
Content-Type: text/plain
POST request with content type "application/trig" not supported
(must be Query/Update with content type
"application/x-www-form-urlencoded",
"application/sparql-query" or "application/sparql-update"
or a valid graph store protocol POST request)
So:
- QLever accepts SPARQL Query/Update at this endpoint
- QLever accepts Graph Store Protocol when graph-scoped (e.g.,
?graph=...) - QLever rejects dataset-level TriG POST (
Content-Type: application/trig) at/
My system.trig change (key parts)
I changed both services to point to QLever (admin and end-user):
<urn:linkeddatahub:services/admin>
{
<urn:linkeddatahub:services/admin> a sd:Service ;
dct:title "LinkedDataHub admin service (QLever)" ;
sd:supportedLanguage sd:SPARQL11Query, sd:SPARQL11Update ;
sd:endpoint <http://host.docker.internal:7001/> ;
a:graphStore <http://host.docker.internal:7001/> ;
a:quadStore <http://host.docker.internal:7001/> ;
lapp:backendProxy <http://host.docker.internal:7001/> .
}
<urn:linkeddatahub:services/end-user>
{
<urn:linkeddatahub:services/end-user> a sd:Service ;
dct:title "LinkedDataHub service (QLever)" ;
sd:supportedLanguage sd:SPARQL11Query, sd:SPARQL11Update ;
sd:endpoint <http://host.docker.internal:7001/> ;
a:graphStore <http://host.docker.internal:7001/> ;
a:quadStore <http://host.docker.internal:7001/> ;
lapp:backendProxy <http://host.docker.internal:7001/> .
}
Tip
You can repro Qlever using https://github.com/okikio/sparql-client/tree/main/infra/qlever, holds both docker and k8s kustomize yaml files for replicating the exact behaviours I'm hitting.
Expected behavior
One of these outcomes (any would be fine, but right now the expectation isn’t clear):
- LDH boots successfully against a backend that supports SPARQL Query/Update + Graph Store Protocol (GSP), even if it does NOT support dataset-level TriG uploads
OR
- LDH docs clearly state that
a:quadStorerequires dataset-level RDF Dataset CRUD (e.g., POST/PUT of TriG / N-Quads), and list compatible backends / required endpoints
OR
-
LDH can be configured to perform bootstrap imports via:
- SPARQL Update (
INSERT DATA { GRAPH ... }), or - per-graph GSP requests (
PUT/POSTwith?graph=...)
instead of dataset-level TriGPOST/PUTtoa:quadStore
- SPARQL Update (
Hypothesis: a:quadStore semantics mismatch (dataset CRUD vs graph CRUD)
From AtomGraph Core API docs, QuadStore is described as a Graph Store Protocol interface extended to support quads, with dataset-level methods like post(Dataset) / put(Dataset).
LDH bootstrap likely uses a:quadStore as a dataset-level endpoint (POST/PUT an RDF Dataset in TriG) to load default datasets.
However, QLever’s HTTP interface (as confirmed by the 400 response above) does not accept POST application/trig as a dataset-level operation at /. It only accepts:
- SPARQL Query/Update content types, or
- graph-scoped GSP requests (e.g., with
?graph=...)
Additionally, QLever write operations require an access token (Bearer header), which LDH bootstrap isn’t providing and from what I understand there is no currently supported approach that would allow including Bearer Tokens as part of the Header of a Update request.
A couple questions
-
What exact URLs/paths should be used for:
- SPARQL endpoint (
sd:endpoint) - Graph Store endpoint (
a:graphStore) - Quad store endpoint (
a:quadStore)
for a non-Fuseki backend?
- SPARQL endpoint (
-
Does LDH require
a:quadStoreto support dataset-level CRUD (POST/PUT an RDF Dataset payload like TriG)?- If yes: is there a list of supported quad-store implementations, or an adapter pattern?
-
Can LDH bootstrap/import be configured to use:
- SPARQL Update for initial dataset loading, or
- per-graph Graph Store Protocol calls
instead of dataset-level TriG upload?
-
Is there a supported way to pass backend auth credentials for bootstrap imports (e.g., Bearer token) so stores like QLever can be used safely?
Possible next steps...
Any of the following could potentially make QLever workable:
- A documented configuration option that makes LDH import via SPARQL Update (
INSERT DATA { GRAPH ... }) rather than dataset-levelPOST application/trig - A mode that splits TriG into named graphs and imports them via graph-scoped GSP (
PUT/POST ?graph=...) - A pluggable “quad store adapter” layer (LDH speaks QuadStore; adapter translates to backend-specific write APIs), if one exists, and maybe some direction on the best approach to creating one if possible (I'm still fairly new to the RDF/SPARQL scene).
- Clear docs stating:
a:quadStorerequires dataset-level TriG/N-Quads CRUD and QLever is not compatible without an adapter
Environment
- LinkedDataHub via Docker Compose example setup
- QLever in Kubernetes, port-forwarded to
localhost:7001 - Docker compose which uses
extra_hosts: host.docker.internal:host-gatewayunder thelinkeddatahubservice - Host platform: M1 Macbook (some platform mismatch warnings in Docker, but the core issue is a protocol mismatch: QLever returns 400 for TriG dataset POST)