Skip to content

feat(distribution-health): new leaf for validity verdicts and usability rollup#472

Merged
ddeboer merged 2 commits into
mainfrom
feat/distribution-health
Jun 15, 2026
Merged

feat(distribution-health): new leaf for validity verdicts and usability rollup#472
ddeboer merged 2 commits into
mainfrom
feat/distribution-health

Conversation

@ddeboer

@ddeboer ddeboer commented Jun 15, 2026

Copy link
Copy Markdown
Member

Closes #468. Part of the distribution-health feature: netwerk-digitaal-erfgoed/dataset-register#2103.

What

A new pure leaf package @lde/distribution-health that derives a distribution’s usability from two separately-produced signals — reachability (HTTP/SPARQL) and validity (does the fetched content parse as RDF?). It interprets the raw results of @lde/distribution-probe and @lde/sparql-importer; it does no I/O and is deliberately acyclic.

It returns plain TypeScript only — no RDF, no vocabulary. Converting a verdict to RDF is the consumer’s job (see below), mirroring @lde/iiif-validator.

Two modules, built test-first (red→green→refactor, one behaviour per cycle):

  • verdict — a ValidityVerdict type (typed failure reason parse-error / empty, best-effort parser message, the validatedFingerprint judged against, producer depth shallow / deep) plus two mappers:
    • probeResultToVerdict (shallow): interprets the body validation the probe already performs for small RDF dumps; returns null when the probe carries no validity signal (network/HTTP failure, SPARQL, or a large HEAD-only body).
    • importOutcomeToVerdict (deep): a failed import is invalid (parse-error); a successful one is valid, or empty when it produced zero triples.
  • usability — the one canonical rollup (reachability, verdicts) → { usable | unusable | unknown, cause }:
    • reachability dominates — unreachable ⇒ unusable;
    • staleness gate — a verdict applies only if its validatedFingerprint equals the currently-observed fingerprint (a null fingerprint never matches, mirroring sourceFingerprint), else it decays to unknown. This is what stops a since-fixed dump showing red;
    • depth beats breadth — a fresh deep verdict wins over a shallow one; a usability resting on a shallow verdict is flagged shallow.

RDF / vocabulary is the consumer’s responsibility

This package emits no quads and coins no predicates, so consumers never have to reconcile two namespaces for the same data. A consumer turns a ValidityVerdict into its own DQV/PROV quads under its own namespace — e.g. the NDE knowledge-graph pipeline / register write a dqv:QualityMeasurement on the distribution under def.nde.nl, alongside the existing quality measurements (qualityMeasurementsStage / failureUsage). The ValidityFailureReason local names drop straight into a SKOS failure scheme (<scheme>#${reason}).

This lands as epic tasks 4 (dataset-register) and 5 (dataset-knowledge-graph).

Tests

20 tests, 100% coverage. The usability rollup gets an exhaustive case matrix (reachability × validity × fingerprint fresh/stale × depth).

Notes

  • The shallow mapper duplicates the probe’s parseable-content-type list and 10 KiB parse threshold (commented). A structured validity signal on the probe result would remove that coupling — noted in verdict.ts.
  • Package version is 0.0.0 so the first nx release lands it at 0.1.0; npm Trusted Publisher bootstrap is a one-time manual step per AGENTS.md.

ddeboer added 2 commits June 15, 2026 16:12
…ty rollup

Adds @lde/distribution-health, a pure leaf that interprets the raw results
of @lde/distribution-probe and @lde/sparql-importer into a distribution
health verdict. Three modules:

- verdict: shallow (probe-result) and deep (import-outcome) mappers to a
  ValidityVerdict carrying a typed failure reason (parse-error / empty), a
  best-effort parser message, the validatedFingerprint judged against, and
  producer depth.
- usability: the canonical rollup over reachability x validity x fingerprint
  freshness x depth. Reachability dominates; a verdict whose fingerprint no
  longer matches decays to unknown (staleness gate); a deep verdict beats a
  shallow one.
- quads: verdict to DQV/PROV quads.

Vocabulary is minted under the LDElements namespace (https://w3id.org/lde/),
the same family @lde/pipeline already uses for sourceFingerprint; consumers
such as NDE reuse these URIs rather than re-minting their own.

Part of netwerk-digitaal-erfgoed/dataset-register#2103. Closes #468.
The verdict-to-quads mapping coined LDE vocabulary (and earlier NDE
vocabulary) inside a generic leaf, which would force consumers to handle
both def.nde.nl and w3id.org/lde predicates for the same data. It also
diverged from the established pattern: analysis leaves such as
@lde/iiif-validator return a plain TypeScript verdict and let the consumer
(the NDE knowledge-graph pipeline) map it to RDF under its own namespace.

Drop quads.ts / validityToQuads / ValidityProvenance and the n3 dependency.
The package now exports only the verdict model + mappers and the usability
rollup. Turning a verdict into DQV/PROV quads moves to the dataset-register
and dataset-knowledge-graph consumers (epic tasks 4 and 5), under def.nde.nl.
@ddeboer ddeboer merged commit a6eec4f into main Jun 15, 2026
2 checks passed
@ddeboer ddeboer deleted the feat/distribution-health branch June 15, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New @lde/distribution-health leaf: validity verdicts + usability rollup

1 participant