dcat-ap-no-validator-service

A shacl based validator backend service for validating catalogs against dcat-ap-no specification

Input

The validator need the following input graphs:

A data graph

The RDF graph containing the data to be validated

A shapes graph

The RDF graph containing the SHACL shapes to validate with

An ontology graph (optional)

The RDF graph containing extra ontological information. The validator will try to import ontologies referenced in owl:imports statements

Input graphs should be supplied in of the following ways:

• a file containing your graph, or
• a url pointing to a resource on the internet containing your graph, or

Config

The input may also contain a configuration record containing two options:

• expand (boolean: true/false): if set to true, the validator will try to fetch remote triples referenced to in the data-graph.
• includeExpandedTriples (boolean: true/false): if set to true, the validator will include the remote triples and the ontologies in the response.

Ref the openAPI specification. An example config record:

{
  "expand": "true",
  "includeExpandedTriples": "true"
}

Response

The response will be a RDF graph consisting of

• the input data graph, and
• the report as a graph according to a SHACL validation report.

If includeExpandedTriples is True, the response will aslo include:

• the additional triples added to the data graph, and
• the ontology graph

Usage by curl examples

Validate file

% curl -i \
 -H "Accept: text/turtle" \
 -H "Content-Type: multipart/form-data" \
 -F "data-graph-file=@tests/files/valid_catalog.ttl;type=text/turtle" \
 -F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
 -F "ontology-graph-file=@tests/files/ontologies.ttl" \
 -X POST http://localhost:8000/validator

Validate endpoint(url)

% curl -i \
 -H "Accept: text/turtle" \
 -H "Content-Type: multipart/form-data" \
 -F "data-graph-url=https://example.com/mygraph" \
 -F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
 -F "ontology-graph-file=@tests/files/ontologies.ttl" \
 -X POST http://localhost:8000/validator

With config parameters

% curl -i \
 -H "Accept: text/turtle" \
 -H "Content-Type: multipart/form-data" \
 -F "data-graph-file=@tests/files/valid_catalog.ttl;type=text/turtle" \
 -F "shapes-graph-file=@tests/files/mock_dcat-ap-no-shacl_shapes_2.00.ttl" \
 -F "ontology-graph-file=@tests/files/ontologies.ttl" \
 -F "config=@tests/files/config.json;type=application/json" \
-X POST http://localhost:8000/validator

List all available shacl shapes

% curl -i \
 -H "Accept: application/json" \
 -X GET http://localhost:8000/shapes

Get details about given shacl shapes

% curl -i \
 -H "Accept: application/json" \
 -X GET http://localhost:8000/shapes/1

Develop and run locally

Requirements

• pyenv (recommended)
• poetry
• nox
• nox-poetry

Install software

% git clone https://github.com/Informasjonsforvaltning/dcat-ap-no-validator-service.git
% cd dcat-ap-no-validator-service
% pyenv install 3.10.6
% pyenv local 3.10.6
% python get-poetry.py
% pipx install nox
% pipx inject nox nox-poetry
% poetry install

Running the API locally

Start the server locally:

% poetry run adev runserver dcat_ap_no_validator_service

Running the API in a wsgi-server (gunicorn)

% poetry run gunicorn dcat_ap_no_validator_service:create_app --bind localhost:8000 --worker-class aiohttp.GunicornWebWorker

Running the wsgi-server in Docker

To build and run the api in a Docker container:

% docker build -t digdir/dcat-ap-no-validator-service:latest .
% docker run --env-file .env -p 8000:8080 -d digdir/dcat-ap-no-validator-service:latest

The easier way would be with docker-compose:

docker-compose up --build

Running tests

We use pytest for contract testing.

To run linters, checkers and tests:

% nox

To run tests with logging, do:

% nox -s integration_tests -- --log-cli-level=DEBUG

Environment variables

REDIS_HOST

Hostname where the remote redis is reachable on default port (6379). Default: localhost

REDIS_PASSWORD

Password to the remote redis is reachable. Default: 6379

LOGGING_LEVEL

One of the supported levels found here. Default: INFO

CONFIG

One of

• dev: will not use cache backend
• test: will not use cache backend
• production: will require and use a redis backend, cf docker-compose.yml Default: production

An example .env file for local development without use of redis cache:

LOGGING_LEVEL=DEBUG
CONFIG=dev