DCAT support: scope and approach

[sorenwacker]

DCAT (W3C Data Catalog Vocabulary) is the catalog/discovery layer for datasets and data services. It is complementary to the domain content standards metaseed already supports (MIAPPE, ISA, DiSSCo, Darwin Core): DCAT describes that a dataset exists and where to get it, while those standards describe what is inside it.

Opening this to scope what DCAT support in metaseed should mean before committing to an approach.

Core model (fits metaseed's root-entity + nested-entity hierarchy)

• dcat:Catalog -> dcat:Dataset / dcat:DataService
• dcat:Dataset -> one or more dcat:Distribution (a concrete form: CSV, API, ...)
• dcat:Distribution -> dcat:accessURL, dcat:downloadURL, dcat:mediaType, dcterms:format
• dcat:CatalogRecord, with dcat:Resource as the abstract superclass

DCAT reuses external vocabularies (Dublin Core Terms, FOAF, vCard, SKOS, PROV) rather than defining its own, which maps well to metaseed's ontology_term / ontology-prefix mechanism. Current version is DCAT 3 (W3C Recommendation, 2024); DCAT 2 (2020) is still widely deployed. Application profiles (DCAT-AP for the EU, plus Geo/Stat/Health variants, and DCAT-US) are where it is used in practice.

Questions for discussion

1. Goal: model catalogs in metaseed, or make existing metaseed datasets discoverable via DCAT (export a CatalogRecord), or both?
2. Plain DCAT 3 first, or go straight to a profile like DCAT-AP that encodes real cardinalities and controlled vocabularies?
3. RDF/JSON-LD export: DCAT is triple-oriented, so faithful interchange needs Turtle/RDF-XML/JSON-LD, a different target than the current YAML/JSON. In scope for a first iteration, or a follow-up?

Tracking issue with the concrete scope options: #24

[sorenwacker]

Research notes: how other tools implement DCAT, and a concrete path for metaseed

Findings from a multi-source review (W3C specs, SEMIC/EU, government profiles, RDFLib/Pydantic libraries, and prior-art repos), with adversarial verification. Primary sources linked inline; honest gaps flagged at the end.

1. Target version: DCAT 3 / DCAT-AP 3.0

DCAT 3 is a W3C Recommendation (2024) and is additive over DCAT 2 — a DCAT 2-shaped export is already DCAT 3 conformant unless it uses the new features (distribution checksums via dcat:Checksum, resource versioning, and dcat:DatasetSeries). So we lose nothing by targeting DCAT 3 from the start.

• DCAT 3: https://www.w3.org/TR/vocab-dcat-3/ · DCAT 2: https://www.w3.org/TR/vocab-dcat-2/ · implementation report: https://w3c.github.io/dxwg/dcat3-implementation-report/

The class set an internal model must map onto is small and stable: Catalog, Dataset, Distribution, DataService, CatalogRecord, plus DCAT 3's DatasetSeries. dcat:Dataset and dcat:DataService are subclasses of abstract dcat:Resource.

2. Suggested mapping for metaseed

metaseed concept	DCAT
a profile collection / the catalog itself	dcat:Catalog
a dataset built under a profile (the root_entity instance)	dcat:Dataset
each serialized output (the YAML/JSON export)	dcat:Distribution (+ dcat:Checksum, dcat:mediaType, dcat:downloadURL)
any served API endpoint	dcat:DataService
versions of the same dataset over time	dcat:DatasetSeries (DCAT 3)

This reuses metaseed's existing root-entity + nested-entity structure; the DCAT layer wraps a dataset rather than replacing the domain profile (MIAPPE/ISA/Darwin Core stays the content model).

3. Python implementation path (non-RDF Pydantic model -> RDF)

The practical route, confirmed against upstream docs:

1. Build an RDFLib Graph from the Pydantic model.
2. Graph.serialize(format="json-ld") (or "turtle", "xml") — same API across formats.
3. Pass a custom @context as a prefix->IRI dict (dct -> http://purl.org/dc/terms/, dcat -> http://www.w3.org/ns/dcat#, ...).

• RDFLib JSON-LD: https://rdflib.readthedocs.io/en/stable/apidocs/examples.jsonld_serialization/
• Optional declarative layer: pydantic-rdf maps Pydantic models to RDF via rdf_type / _rdf_namespace / WithPredicate annotations: https://github.com/Omegaice/pydantic-rdf

This fits metaseed's ontology_term / ontology-prefix mechanism — those prefixes become the JSON-LD context.

4. Profiles and validation (SHACL)

DCAT-AP is the EU "master profile" of DCAT 3; national/domain profiles layer constraints on it and are validated with SHACL shapes shipped as Turtle files:

• DCAT-AP 3.0: https://semiceu.github.io/DCAT-AP/releases/3.0.0/ · SHACL shapes repo: https://github.com/SEMICeu/dcat-ap_shacl
• National/domain profiles as the layering pattern we'd follow: DCAT-US v3.0 (https://resources.data.gov/resources/dcat-us3/), DCAT-AP.de (https://github.com/GovDataOfficial/DCAT-AP.de-SHACL-Validation), DCAT-AP-ES (https://datosgobes.github.io/DCAT-AP-ES/en/validation/), GeoDCAT-AP (ISO 19115 mapping: https://github.com/SEMICeu/GeoDCAT-AP)
• SEMIC's reuse-first discipline (reuse DCAT-AP -> tighten -> reuse other vocabs -> mint new terms only as last resort; ship HTML spec + SHACL + dereferenceable URIs): https://interoperable-europe.ec.europa.eu/collection/semic-support-centre/dcat-ap-reuse-guidelines-building-profiles-right-way

Implication: our existing profile.yaml validation_rules could compile toward SHACL, and we can validate exports against the published DCAT-AP shapes rather than inventing our own.

5. How comparable projects solve it + related discussions

• CKAN / ckanext-dcat — the closest prior art: a pluggable "profiles" mechanism maps CKAN's internal dataset dict to DCAT and serializes RDF/Turtle/JSON-LD. Worth reading its mapping + profile-authoring model directly:
  • mapping: https://docs.ckan.org/projects/ckanext-dcat/en/latest/mapping/ · writing profiles: https://docs.ckan.org/projects/ckanext-dcat/en/v2.3.0/writing-profiles/
  • related issue/discussion: Support for DCAT 3 ckan/ckanext-dcat#257 · Support for DCAT 3 ckan/ckan#7855
• Frictionless Data -> DCAT (directly analogous: schema-driven, non-RDF source -> DCAT): discussion Mapping data packages to DCAT2 frictionlessdata/frictionlessdata.io#551 · worked mapping https://github.com/OSUKED/Frictionless-Data-to-DCAT
• Dataverse DCAT export request/design: Support for Importing and Exporting DCAT Metadata IQSS/dataverse#1592

6. Recommendation

• Target DCAT 3 / DCAT-AP 3.0; emit JSON-LD + Turtle via RDFLib.
• Implement as an export adapter (dataset -> dcat:Dataset, profile -> dcat:Catalog, outputs -> dcat:Distribution + checksum), mirroring ckanext-dcat's profile/serializer split.
• Reuse DCAT-AP mandatory properties + controlled vocabularies; validate against the published SHACL shapes.
• Audit which DCAT-AP mandatory fields (publisher, title, description, license, issued/modified, contactPoint, theme) aren't yet captured by current spec fields — that gap list is the real first task.

Honest gaps (verify before relying on these)

• The concrete internal-model->DCAT mappings of CKAN, Dataverse, and InvenioRDM were not independently verified from their own source in this pass — the links above are starting points, not confirmed mappings.
• Open question worth its own look: is there an emerging life-science / biodiversity DCAT-AP profile (HealthDCAT-AP, or a Darwin Core / GBIF DCAT mapping) we should reuse rather than mint terms for MIAPPE/ISA/Darwin Core? (Related: ckanext-dcat Dataset Series work, w3c/dxwg.)
• DCAT-AP SHACL shapes are migrating 2.0 -> 3.0; pin the exact shape version before validating.

---

_{Compiled from an automated multi-source research pass with adversarial verification. Primary sources linked inline.}

[sorenwacker]

Follow-up: F-UJI as the validation + motivation angle for DCAT

F-UJI (FAIRsFAIR Research Data Object Assessment Service; PANGAEA / Devaraju & Huber, EU Horizon 2020 FAIRsFAIR, later extended under FAIR-IMPACT) is a web service that programmatically scores a dataset's FAIRness against the FAIRsFAIR Data Object Assessment Metrics. It matters here because it is a concrete, third-party consumer of DCAT metadata — which reframes "add DCAT export" from a catalog feature into something with a measurable payoff.

Why it's relevant to this discussion

• F-UJI's tests are built around generic cross-domain metadata standards — Dublin Core, DCAT, DataCite, schema.org — and it ships dedicated parsers for them in XML / RDF / JSON.
• For RDF metadata it uses RDFLib + SPARQL queries to retrieve Dublin Core and DCAT(-2) metadata — the same RDFLib path recommended above for our exporter. So a DCAT export that F-UJI can read is the same artifact we were already planning to produce.
• It assesses from aggregated metadata: embedded in the landing/data page (e.g. JSON-LD), retrieved via PID content negotiation (DataCite), signposting / typed links, and other services. So to actually score, DCAT has to be discoverable — embedded in a landing page as JSON-LD and/or served via content negotiation — not just written to a standalone file.

What this implies for the metaseed DCAT feature

1. It gives us an objective acceptance test. Once metaseed emits DCAT(-AP) JSON-LD, we can run F-UJI against a published dataset and read the per-metric FsF results — an external, automated measure of whether our export improves Findability/Accessibility/Interoperability/Reusability, not just SHACL conformance.
2. Harvestability is a requirement, not an afterthought. F-UJI needs the DCAT to be reachable (embedded JSON-LD in HTML, content negotiation, or signposting links). If the UI/REST layer exposes a dataset landing page, embedding the DCAT JSON-LD there is what makes the score move.
3. Target DCAT-2 core for the harvester. F-UJI's SPARQL harvesting targets DCAT-2 properties. DCAT 3 is additive, so a DCAT-2-compatible core (the mandatory DCAT-AP properties) is what F-UJI will actually pick up — reinforces the "DCAT-2 core, DCAT-3 extras optional" plan above.

Links

• Tool: https://www.f-uji.net/ · assess: https://www.f-uji.net/?action=test
• Source: https://github.com/pangaea-data-publisher/fuji (and the FAIR-IMPACT software-assessment fork https://github.com/FAIR-IMPACT/fuji4software)
• FsF metrics: https://doi.org/10.5281/zenodo.3775793
• Methodology paper (automated FAIR measurement): https://pmc.ncbi.nlm.nih.gov/articles/PMC8600246/
• Related, directly on point: "An Extensive Methodology and Framework for Quality Assessment of DCAT-AP Datasets" — https://ouci.dntb.gov.ua/en/works/9GwdKBP9/

Suggested addition to the scope

Add a fourth direction to the scope options: "DCAT export validated by F-UJI" — produce harvestable DCAT(-AP) JSON-LD, then use F-UJI (self-hostable via its Docker image / API) as an automated FAIRness regression check in CI alongside SHACL validation.

---

_{Sourced from f-uji.net, the FAIRsFAIR/FAIR-IMPACT repos, and the FsF metrics; F-UJI's DCAT-2/RDFLib/SPARQL harvesting confirmed from its documentation and published methodology.}

[sorenwacker]

Outcome (260628): DCAT is implemented

This discussion's scope is now built and on main:

• Version/profile: target is plain DCAT 3 vocabulary, serialized via rdflib; DCAT-AP-style mapping is declared per profile.
• Mapping is spec-driven (the key decision): each profile's root entity annotates its fields with a dcat: property (dct:title, dcat:contactPoint, …) in profile.yaml (spec_version 0.5). No hardcoded registry — any profile (incl. Spec-Builder ones) self-describes its mapping. MIAPPE/ISA/ENA are annotated.
• Record-rooted profiles (Darwin Core/DiSSCo) supply explicit dataset-level CatalogMetadata via an editor on the in-app DCAT panel.
• Output: clean JSON-LD (minimal @context) + Turtle; the rdflib dependency is behind the metaseed[dcat] extra so core stays lean.
• Viewer: a /dcat page + /api/dcat panel with copy/download.
• Hardened: valid RDF for any identifier/URL, graceful route errors.

PRs: #53 (model+serializer+viewer), #54 (spec-driven mapping), #55/#56 (hardening), #58 (catalog-metadata editor). Tracking epic: #24.

Remaining (open): SHACL/DCAT-AP conformance (#29), harvestable exposure for portals/F-UJI (#30, handled separately), and the F-UJI regression check (#31). The F-UJI angle from this thread is captured in #31.

[sorenwacker]

Status (260628): export-adapter pattern extended — first step, untested

DCAT remains the reference exporter (done, hardened). It now has siblings: a first cut of export adapters for ENA (submission XML), PRIDE (submission.px), MetaboLights (ISA-Tab + MAF), and BrAPI (JSON), each with a matching importer (bidirectional bridge — umbrella #75).

Honest caveat: unlike DCAT, these are first-step and untested — smoke-tested against a single live record each, not comprehensively tested and not validated against the real submission systems. The pattern and the seam are in place; correctness at scale is not yet proven.