Skip to content

Proposal: optional retrieved_at / verified_at and a source-liveness convention — the container outlives the source #211

Description

@jpavley

Summary

§4.1 gives a concept one clock: timestamp, "last meaningful change." For bundles whose concepts point at external resource URLs, that conflates two things a consumer needs to distinguish — when the content changed vs. when a producer last fetched or verified it — and the spec has no home at all for a third signal: whether the resource still resolves.

This is adjacent to #97 (elevate timestamp) and the maintenance-signals discussion in #158, but scoped to the producer-side fields those threads don't cover: retrieval/verification dates and source liveness.

Evidence (civic public records, where sources rot fast)

We maintain an OKF v0.1 bundle of local-government records for a large county in the US Northeast. Within the bundle's first year we hit, in production:

  • Sources vanishing under live atoms: a town's public-hearing notices disappeared from its news feed before the year ended. The atom is fine; its resource is gone. Nothing in the format records or reveals that.
  • Verification as a distinct event: libraries post board minutes months late; a fact can be re-verified unchanged long after its content timestamp. A consumer deciding whether to trust a figure needs "verified last week" vs. "verified last year," which timestamp cannot express without destroying its content-change meaning.
  • Liveness is checkable but nontrivial: we run a sweep that GETs every resource in the bundle (~160 URLs). Getting truthful answers required a browser user-agent, per-host politeness delays, and 429 backoff — civic-portal hosts throttle and UA-block. It's exactly the kind of shared problem a documented convention (plus a reference tool) would keep every producer from re-solving.

Proposal (all optional, all additive under §4.1 / §11)

  1. retrieved_at — when the producer last fetched the resource.
  2. verified_at — when a human or pipeline last confirmed the concept still matches its source.
  3. A source_status convention — last-checked date + last observed HTTP state for the resource (e.g. {checked: 2026-07-02, state: ok | dead | blocked}).

Consumers that don't know the fields ignore them; nothing about §9's permissive-consumption contract changes. A v0.1 bundle stays conformant.

We have a working liveness checker (Python, no dependencies beyond the standard library + requests) built for this bundle and would be glad to contribute it as a reference implementation if there's interest.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions