Skip to content

Data Model Provenance

mavrovde edited this page Nov 1, 2026 · 1 revision

Data Model: Provenance

Part of the Data Model. A logical model for the API's read side, not a copy of any physical schema; storage names may differ. Conventions, the record envelope and retention classes: Data Model section 5. API side: Data Licensing and Attribution, Common Envelope (meta.attribution, meta.notes, meta.freshness) and Service Endpoints (/v1/sources).

Provenance is the shared kernel: every record of every other domain names a source, and the source names its publisher and licence, its legal notes, and its freshness. Localised texts and their origin are also provenance and live here as text variants.

A source in this model is what the API calls a source family: a group of data with one publisher relation and one licence treatment. A source is named by a stable, generic id (roads.federal, charging.availability), never by a system name (LIC-010).

1. Diagram

erDiagram
  publisher ||--o{ source : "publishes"
  licence ||--o{ source : "governs"
  source ||--o{ source_note : "requires"
  source ||--|| source_freshness : "has"
  source ||--o{ text_variant : "originates"
  publisher {
    string publisher_id PK
    string display_name
    string website
  }
  licence {
    string licence_code PK
    string licence_name
    string licence_url
    string licence_category
    bool share_alike
    bool attribution_required
  }
  source {
    string source_ref PK
    string domain
    string publisher_id FK
    string licence_code FK
    string attribution_text
    string changes_statement
    string terms_url
    string share_alike_offer_url
    bool acceptance_required
    int freshness_target_s
    int stale_after_s
    date register_version
  }
  source_note {
    string source_ref FK
    string note_kind
    string lang
    string body
  }
  source_freshness {
    string source_ref FK
    instant data_as_of
    string freshness_version
  }
  text_variant {
    string owner_entity
    string owner_id
    string field
    string lang
    string body
    string text_origin
    string source_ref FK
  }
Loading

2. Entities

publisher

An organisation that publishes data, named as its licence asks. Retention REF.

Field Type Req Unit / format Constraints and values Description API field
publisher_id id yes Identity none
display_name text yes The name the licence requires meta.attribution[].publisher, /v1/sources publisher
website text no URL Publisher's site none in v1

licence

A licence or set of terms. Retention REF.

Field Type Req Unit / format Constraints and values Description API field
licence_code key yes SPDX identifier where one exists, else a documented id unique Identity licenceId
licence_name text yes Full name licence
licence_url text yes URL Canonical text licenceUrl
licence_category enum<licence_category> yes attribution, zero, share_alike, restricted_purpose, acceptance, official Category of [[Data Licensing and Attribution Spec-Data-Licensing-and-Attribution]] section 1
share_alike bool yes Share-alike obligation shareAlike
attribution_required bool yes Whether attribution is a condition gate behaviour

source

A source family: the unit of attribution, of freshness and of the licence gate. Retention REF. This entity is the register /v1/sources and the licence gate both read (FR-SVC-002, LIC-008).

Field Type Req Unit / format Constraints and values Description API field
source_ref key yes <domain>.<name>, lower snake case unique, stable Identity family, sourceFamily
domain enum<domain> yes places, roads, energy, parking, transit, accessibility, weather, rules Owning domain /v1/sources domain
publisher_id id → publisher yes Publisher publisher
licence_code key → licence yes Licence licence, licenceId, licenceUrl
attribution_text text yes verbatim The exact notice the licence requires meta.attribution[].text, attributionText
changes_statement text no The modification statement, where the licence asks for one changesMade
terms_url text no URL Terms to follow or accept termsUrl
share_alike_offer_url text no URL required when the licence is share-alike Where the derived database is offered shareAlikeOffer, shareAlikeExport
acceptance_required bool yes Whether the key holder must accept terms first termsAcceptanceRequired
use_classes list<enum<use_class>> yes subset of consumer, non_commercial, commercial; empty until the licence review is signed off (LIC-009) Use classes the licence permits useClasses, licence gate
freshness_target_s int yes s > 0 Design target for the age of the newest data freshnessTargetS
stale_after_s int yes s ≥ freshness_target_s Age beyond which the data is stale meta.freshness[].staleAfterS
review_signed_on date no When the licence review was signed off gate enables the source
register_version date yes Version of the register entry meta.registerVersion

source_note

A condition note or purpose note a licence requires to accompany its data, verbatim and in its original language. Retention REF.

Field Type Req Unit / format Constraints and values Description API field
source_ref key → source yes Source meta.notes.*[].family
note_kind enum<note_kind> yes condition, purpose Kind of note meta.notes.condition[] or meta.notes.purpose[]
lang text yes BCP 47 Language of the note, never translated in place (LIC-003) meta.notes.*[].lang
body text yes verbatim The note meta.notes.*[].text, conditionNotes, purposeNotes

source_freshness

The freshness of a source: the instant of its newest data and a version that changes whenever that data changes. One row per source, replaced in place. Retention CUR.

Field Type Req Unit / format Constraints and values Description API field
source_ref key → source yes unique Source meta.freshness[].family
data_as_of instant no UTC null before the first data Instant of the newest data meta.freshness[].dataAsOf, meta.asOf, ageS, stale
freshness_version text yes opaque changes on every data change Version for entity tags ETag (FR-BRIEF-007, NFR-FRESH-004)

text_variant

A text in a language other than the original, or the original itself where the owning field holds a normalised form. Retention follows the owning record.

Field Type Req Unit / format Constraints and values Description API field
owner_entity text yes entity name of this model Entity of the owning record none
owner_id id yes Owning record none
field text yes field name of this model an ltext field Which field it translates none
lang text yes BCP 47 unique with the three above Language textLanguage
body text yes plain text The text the translated field
text_origin enum<text_origin> yes original, curated, machine How the text came to be language_fallback warning, documentation
source_ref key → source yes Source of the original originalLanguage context

3. Keys and relationships

Relationship Cardinality Rule
publisher → source one to many
licence → source one to many A source has exactly one licence (DM-011)
source → source_note one to many
source → source_freshness one to one
Every record of every domain → source many to one Through the record envelope

4. Row-level provenance

The record envelope (Data Model section 5.10) puts source_ref, origin_key, published_at, recorded_at, changed_at and withdrawn_at on every record. The API uses only source_ref (for sourceFamily, attribution and the licence gate) and changed_at/published_at (for updatedAt); the rest stays on the storage side (DM-018).

The licence gate filters by source_ref before reading and checks every returned record against the permitted set again (Architecture Overview, request lifecycle).

5. Temporal behaviour

  • Publishers, licences, sources and notes are reference data (REF); the register is versioned by register_version.
  • source_freshness is current state (CUR).
  • A source whose review is not signed off has an empty use_classes list and is never served.

6. Access paths

Path Used by
All sources with publisher, licence and notes /v1/sources, the licence gate (loaded once, refreshed on register change)
Freshness of the sources an operation reads meta.freshness[], ETag, 304 answers
Text variants of a set of records in one language localised answers

7. Entity to API

Entity Operations Resource members
source, publisher, licence, source_note every data operation; GET /v1/sources, GET /v1/sources/{family} meta.attribution[], meta.notes, source register entries, Link: rel="license"
source_freshness every data operation meta.freshness[], meta.asOf, ETag, data_stale warning
text_variant every operation with texts headline, description, instruction, textLanguage, originalLanguage, language_fallback warning

Field-level mapping: Mapping.

Home

About

Design

Data Model

Technical specification

Testing

Engineering

Technology choice

Reference

Clone this wiki locally