Skip to content

Week 09 Report

Jump to bottom
Vishmayraj Zala edited this page Aug 19, 2026 · 1 revision

Weekly Report - Week 9 (July 25 - July 31, 2026)

I performed the following tasks during week 9:

PR #204 opened

  • Opened feat(connector): network-as-subcatalog architecture, STAC compliance fixes, DCAT mapping migration, bundling the network architecture from week 6, the STAC compliance fixes and DCAT mapping migration from week 8, and the testing-infrastructure fix from week 7 into a single reviewable PR. Depends on #195.

dcat_transformer.py -> network-scoped case and per-Dataset detail

  • Implemented build_dcat_catalog_with_networks(), returning one independent graph per scope (root, orphan, per-Network) so each scope serializes and caches on its own rather than through a merged quad store.
  • Made the root graph under NETWORK=1 deliberately structural-only (Catalog + DataService + Agents + dct:hasPart), with orphan and each Network carrying their own Dataset/DatasetSeries content.
  • Filled out dcat:Dataset nodes with license, access rights, and contact point (vcard), and added the three dcat:Distributions per Dataset mirroring STAC's Item Assets (JSON observations, CSV export, Datastream entity metadata).
  • Implemented dcat:DatasetSeries extent (bbox/temporal) computed per-scope from its own successfully-built Datasets, with a fallback to the Thing's own Location geometry -- same fallback chain as STAC's Collection extent, kept independent per scope.
  • Decided to keep DatasetSeries unlinked via dcat:dataset (DCAT-AP has no clean "Catalog contains this Series" predicate, per the open gap at SEMICeu/DCAT-AP#289), preserving one-hop reachability either direction via dcat:inSeries/dct:isPartOf instead.

License & language resolution (config.py)

  • Implemented resolve_license_uri(): normalizes a license value (URI passthrough, SPDX-id lookup, or best-effort spdx.org guess with a warning) for every dct:license triple across catalog, dataset, and distribution nodes.
  • Implemented resolve_language_uri(): maps a BCP-47 tag to its EU Publications Office NAL URI for dct:language, kept distinct from the lang= tag used on title/description/keyword literals.

Master-switch system

  • Added STAC_TRANSFORMER / DCAT_TRANSFORMER env-var flags, parsed once at import time via a dedicated _env_flag() helper (explicit "1" opt-in, not truthy-string coercion), so the scheduler and api.py read the same parsed value instead of two independently-parsed copies drifting apart.

Dual-format caching (cache.py)

  • Each DCAT scope is now cached twice, as Turtle and as JSON-LD, under sibling dcat:graph:{scope} / dcat:graph:{scope}:jsonld Redis keys, one Graph.serialize() call per format at write time.
  • write_dcat_catalog() / write_dcat_catalog_with_networks() purge stale dcat:* keys before writing, same SCAN-then-delete pattern as the STAC cache writer.

Details can be found in:

  • PR: #204
  • DCAT transformer work: api/app/v1/connector/dcat_transformer.py, config.py, cache.py

What do I plan to do next week?

  • Build out the DCAT API layer (/connector/dcat/root, /orphan, /{network_id} routes, Turtle + JSON-LD).
  • Run the generated graphs through pyshacl against the official DCAT-AP 3.0 SHACL shapes and fix whatever it surfaces.

Am I blocked on anything?

  • No blockages.

Clone this wiki locally