Skip to content

Weeks 3 Report

Jump to bottom
Vishmayraj Zala edited this page Jun 13, 2026 · 1 revision

Weekly Report - Week 3 (June 8 - June 14, 2026)

I performed the following tasks during week 3:

DCAT-AP 3.0 transformation layer design and reference documentation

  • Authored docs/STA-DCAT-AP-Transformation-Layer-Reference.md, a complete field-by-field mapping specification covering the full STA to DCAT-AP 3.0 transformation: Datastream to dcat:Dataset, Thing to dcat:DatasetSeries, the deployment-level dcat:Catalog, and dcat:Distribution construction for JSON, CSV, and MQTT access points.
  • Documented the mandatory field gap strategy: DCAT-AP 3.0 requires dct:title, dct:description, and dct:publisher on the root catalog, none of which have a source in the SensorThings API schema. These are read from environment variables. For Dataset-level gaps (license, access rights, contact point, accrual periodicity), the connector reads from Datastream.properties using a documented camelCase key convention, allowing operators to annotate individual Datastreams without schema changes.
  • Covered JSON-LD and Turtle serialization strategy, language tagging for RDF literals, blank node vs URI handling for the publisher agent, and dct:spatial construction from Location geometry.

Architecture proposal prepared for mentor review

  • Prepared a full written proposal for the revised Postgres-direct architecture to present to mentors at the end of this week, including the harvest JOIN query, the four statement-level NOTIFY triggers for istsos_schema.sql, the Redis key structure, the integration point inside api/app/connector/, and the lifespan wiring in main.py.
  • The proposal addresses the benchmark findings from weeks 1-2 and incorporates Daniele's earlier advice on using Postgres and Redis directly.

Details can be found in:

  • DCAT-AP Mapping Reference: docs/STA-DCAT-AP-Transformation-Layer-Reference.md
  • Architecture proposal: prepared as agenda for mentor meeting, documented in docs/Harvesting-Layer-Reference.md (updated to reflect the proposed integrated design)

What do I plan to do next week?

  • Confirm the architecture direction with mentors and begin harvester.py implementation in the confirmed shape (asyncpg JOIN query + LISTEN/NOTIFY listener).
  • Start cache.py alongside harvester since the two are tightly coupled.
  • Open a PR to the main istSOS4 repo with the four NOTIFY triggers for istsos_schema.sql once the integration approach is confirmed by mentors.

Am I blocked on anything?

  • No Blockages
  • Ready to transition into Week 4 implementation immediately following the mentor meeting and architecture sign-off.

Clone this wiki locally