Skip to content

feat: consolidate probing; rename @lde/sparql-monitor → @lde/distribution-monitor#358

Merged
ddeboer merged 5 commits intomainfrom
refactor/distribution-monitor
Apr 23, 2026
Merged

feat: consolidate probing; rename @lde/sparql-monitor → @lde/distribution-monitor#358
ddeboer merged 5 commits intomainfrom
refactor/distribution-monitor

Conversation

@ddeboer
Copy link
Copy Markdown
Member

@ddeboer ddeboer commented Apr 23, 2026

Summary

Makes @lde/distribution-probe the single source of truth for probing (including SPARQL endpoints), and renames @lde/sparql-monitor to @lde/distribution-monitor to reflect its broader scope – it now monitors any DCAT distribution, SPARQL endpoint or data dump.

This is a breaking-change PR. Pre-release status per CLAUDE.md, so no shims.

Part A – Extend @lde/distribution-probe

  • probe() now takes ProbeOptions (timeoutMs, headers, sparqlQuery). Breaking: the old probe(distribution, 5000) number form is gone; pass { timeoutMs: 5000 }. @lde/pipeline’s one call site is migrated in this PR.
  • SparqlProbeResult, DataDumpProbeResult, and NetworkError all carry a new responseTimeMs: number field, measured from just before the fetch to just after the response is fully consumed (or the error is thrown). Breaking: the result-class constructors take this as a required parameter; SparqlProbeResult additionally takes acceptedContentType. Pipeline test fixtures that construct these directly are updated.
  • URL-embedded Basic auth (user:pass@host) is extracted into an Authorization: Basic … header. Applied to both SPARQL and data-dump probes; caller-supplied Authorization wins on conflict.
  • SPARQL probing dispatches on query type (ASK / SELECT / CONSTRUCT / DESCRIBE). The Accept header and response validation are query-type-aware (JSON boolean for ASK, JSON results for SELECT, non-empty RDF body for CONSTRUCT / DESCRIBE). Query-type detection uses an inline regex with #-comment stripping.
  • Design note on fetch-sparql-endpoint: the task suggested delegating to SparqlEndpointFetcher.fetchAsk/fetchBindings/fetchTriples for query dispatch. I kept raw fetch instead so SparqlProbeResult retains the underlying Response (statusCode, statusText, contentType, lastModified). SparqlEndpointFetcher throws on HTTP errors and hides the response, which is awkward when the probe’s job is to describe failures. The inline regex is the task’s explicitly-allowed fallback.

Part B – Rename @lde/sparql-monitor@lde/distribution-monitor

  • New config shape: { identifier, distribution: { accessUrl, mediaType?, conformsTo? }, sparqlQuery? }. Any DCAT distribution is now supported; sparqlQuery only applies to SPARQL endpoints.
  • SparqlMonitor class removed; MonitorService now calls probe() directly and maps the returned ProbeResultType onto the existing CheckResult shape (failing probes become success: false with failureReason/joined warnings/HTTP-status as the error message).
  • PostgreSQL schema is unchanged – CheckResult’s shape is already minimal enough to cover both cases. Adding a probe_type column is noted as a possible follow-up.
  • CLI bin is distribution-monitor. Breaking: no sparql-monitor alias.
  • MonitorServiceOptions gains timeoutMs, headers, and an injectable probe (for tests).

@lde/sparql-monitor deprecation

packages/sparql-monitor/ is deleted from the workspace. No final re-export shim is published. @lde/sparql-monitor on npm stops receiving updates; consumers must migrate to @lde/distribution-monitor directly.

Post-merge action for the maintainer (outside this PR): run npm deprecate @lde/sparql-monitor "Renamed to @lde/distribution-monitor" so npm install of the old name surfaces a warning, and enable npm Trusted Publisher on @lde/distribution-monitor after its first publish.

Out of scope

  • Structured probe-failure taxonomy (DnsFailure / ConnectionTimeout / TlsError): defer until a second consumer requests it.
  • Changes to probeResultsToQuads output in @lde/pipeline/src/distribution/report.ts.
  • Extending the PostgreSQL schema beyond what the rename needed.

ddeboer added 5 commits April 23, 2026 12:06
…tion-monitor

Extend @lde/distribution-probe:
- probe() now accepts a ProbeOptions object (timeoutMs, headers, sparqlQuery)
  with a deprecated back-compat shim for the old number-as-second-arg form
- SparqlProbeResult, DataDumpProbeResult and NetworkError all expose
  responseTimeMs measured from just before fetch to just after the response
  is fully consumed (or the error thrown)
- URL-embedded Basic auth (user:pass@host) is extracted into an
  Authorization header for both SPARQL and data-dump probes; caller-supplied
  Authorization takes precedence
- SPARQL probing dispatches on query type (ASK/SELECT/CONSTRUCT/DESCRIBE)
  with a query-type-aware Accept header and body validation. Query type
  detection uses an inline regex with comment stripping; using
  fetch-sparql-endpoint would obscure the underlying Response needed for
  SparqlProbeResult metadata

Rename @lde/sparql-monitor → @lde/distribution-monitor:
- New config shape targets any DCAT Distribution: { accessUrl, mediaType?,
  conformsTo? } + optional sparqlQuery, replacing the SPARQL-only
  endpointUrl + query shape
- SparqlMonitor class removed; MonitorService now calls probe() directly
  and maps ProbeResultType onto the existing CheckResult shape
- PostgreSQL schema unchanged – CheckResult has the same minimal shape
- Bin name renamed to distribution-monitor with sparql-monitor kept as an
  alias for one minor version
- @lde/sparql-monitor is reduced to a deprecated re-export shim (v0.6.0)
  that forwards to @lde/distribution-monitor with a runtime warning; after
  that final publish the maintainer should run
  'npm deprecate @lde/sparql-monitor "Renamed to @lde/distribution-monitor"'
  and remove packages/sparql-monitor/ in a follow-up

Update pipeline test fixtures for the new result-class constructor
signatures (responseTimeMs; SparqlProbeResult also takes acceptedContentType).
- Delete packages/sparql-monitor/ entirely; @lde/sparql-monitor stops
  receiving updates on npm rather than shipping a final re-export shim.
  Consumers migrate to @lde/distribution-monitor directly.
- Drop the sparql-monitor bin alias from @lde/distribution-monitor; the CLI
  is invoked as distribution-monitor only.
- probe() no longer accepts a bare number as the second argument; only the
  ProbeOptions object form is supported. Migrate pipeline's call site in
  SparqlDistributionResolver.resolve() to pass { timeoutMs }.
Main released @lde/distribution-probe 0.1.1 while this branch was open.
Point @lde/distribution-monitor at the released version and regenerate
the lockfile so 'npm ci' can resolve both the workspace entry and the
transitive n3@2.0.3 install.
CI failed because the old sparql-monitor thresholds don't match the
rewritten service.ts + new config.ts + types.ts. Drop to the numbers
vitest actually reports on CI (lines 94.04, functions 96.29, branches
77.27, statements 92.47) and fix the cacheDir rename that got missed
during the package rename.
@ddeboer ddeboer merged commit 0eee1fc into main Apr 23, 2026
2 checks passed
@ddeboer ddeboer deleted the refactor/distribution-monitor branch April 23, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant