feat(plugin): schema-driven config with live-reload scopes + reorganized layout; v0.28.0

[harper-joseph]

What

Introduces a declarative config schema (src/configSchema.js) as the single source of truth for every plugin option — default, operator-facing description, reload scope (live vs restart), and validation/display hints — and makes the background schedulers genuinely live-reloadable. Groundwork for managing config from the admin UI.

Schema (src/configSchema.js)

Each option/group declares:

• description — served by the management API; written for operators, will back the admin-UI config editor
• scope — 'live' (default) or 'restart'; groups can set a scope their children inherit
• secret — drives redaction (replaces the hardcoded SECRET_PATHS in util/redact.js)
• enum, min/max, nonEmpty — enforced at apply time (violation warns + keeps the default; generalizes the old hand-rolled empty-delimiter rejection)
• unit, itemType — display hints for the future editor
• movedFrom — legacy path alias (see reorg below)

defaultConfig(), merge validation, redaction, and describeConfigSchema() (JSON-serializable, served on GET /prerender_admin/config as schema) all derive from the one tree.

Live reload reaches the schedulers

Previously the five background schedulers captured their gates/intervals at boot, so options like sitemap.node silently required a restart (bitten by this during the sitemap-daily rollout). They now subscribe to onConfigApplied and re-arm on change:

• sitemap refresh: node/workerIndex pinning, refreshTime, timezone — pin/unpin/re-time without restart
• reconciler: enabled (runtime kill-switch), interval
• backlog snapshotter: management.enabled, backlogSnapshotInterval (0 = manual-only)
• unrouted reporter: enabled, interval
• queue status sync: statusSyncInterval

(Re)enabling a per-node sweep re-arms boot-shaped (delay + per-node stagger), since a config change reaches all nodes at the same moment — same herd concern as a rolling restart.

The only restart-scoped options left are render.reconcile.startDelay/startJitter. Changing one live logs a warning and is reported as pendingRestart on GET /prerender_admin/config (cleared if flapped back to the boot value).

Config reorganization (21 → 14 top-level keys)

All moves keep a movedFrom alias — old paths still apply, with a deprecation warning; when both old and new are set, the new path wins:

old	new
botPathPrefix, excludePathPatterns	ingress.*
securityToken, staging, userAgents, ignoredHeaders	origin.* (everything about fetching from the origin)
url.queryParams	cacheKey.queryParams (one-field url group dissolved)
sitemapUserAgent	sitemap.userAgent

Removed: debugHeader.value — dead config; both consumers presence-check debugHeader.key only.

The deployed consumer config sets none of the moved keys except the origin.* ones, which the aliases cover — verified against the consumer repo's config.yaml.

Admin API

GET /prerender_admin/config now returns schema and pendingRestart alongside the redacted config + warnings. The console's config view renders generically, so the new layout displays as-is; the schema-driven editor is deliberately not wired yet (stubbed-panel rule — awaiting design).

Testing

• 358/358 plugin tests pass; lint + format clean
• New test/configReactivity.test.js: scheduler re-arm state transitions under mock timers (arm/disarm/interval swap/pin move), plus a real flush-cadence test for the unrouted reporter
• config.test.js: schema completeness (every option/group described), alias coverage, enum/bounds/nonEmpty enforcement, pending-restart tracking, listener semantics
• redact.test.js: redaction list derived from schema

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a schema-driven configuration system for the prerender plugin, reorganizing configuration options under namespaces like ingress and origin while maintaining backward compatibility for legacy paths. It also enables background schedulers to dynamically re-arm themselves on configuration changes without requiring a worker restart. The review feedback is highly constructive and focuses on strengthening the configuration validation engine: specifically, ensuring that nonEmpty checks explicitly reject null and undefined values, and marking critical fields like refreshTime, timezone, securityToken.header, and debugHeader.key as nonEmpty to prevent potential runtime crashes or invalid HTTP requests.

[gemini-code-assist]

If a nonEmpty option (such as deviceTypes.supported or cacheKey.attributes) is overridden with null or undefined, the current check value === '' || (Array.isArray(value) && value.length === 0) will evaluate to false and fail to reject the invalid value. This can lead to runtime crashes when downstream code attempts to access properties or methods on the null/undefined value.

We should explicitly check for null and undefined in the nonEmpty validation.

Suggested change

	if (node.nonEmpty && (value === '' || (Array.isArray(value) && value.length === 0))) {
	if (node.nonEmpty && (value === '' || value === null || value === undefined || (Array.isArray(value) && value.length === 0))) {

References

1. Validate and reject unusable or structural configuration overrides at the entry point (where configuration is parsed or applied) by restoring them to safe defaults, rather than adding defensive guards in downstream utility functions.

[harper-joseph]

Applied in dc6ab4a, with one caveat noted in a comment: null/undefined can't actually reach this check through applyOptions — mergeInto skips null/undefined overrides and every schema path exists in the defaults — but the validator shouldn't depend on the merge layer's behavior to be safe, so the explicit checks are worth having.

[gemini-code-assist]

The refreshTime and timezone options are critical for scheduling the daily sitemap refresh. If either is configured as an empty string, parsing will fail or throw an error at runtime. Marking them as nonEmpty: true ensures that any empty overrides are rejected and reverted to their safe defaults at apply time.

Suggested change

	refreshTime: option('12:00', 'Local time-of-day ("HH:MM") for the daily sitemap refresh.'),
	timezone: option('America/New_York', 'IANA timezone `refreshTime` is interpreted in.'),
	refreshTime: option('12:00', 'Local time-of-day ("HH:MM") for the daily sitemap refresh.', { nonEmpty: true }),
	timezone: option('America/New_York', 'IANA timezone refreshTime is interpreted in.', { nonEmpty: true }),

References

1. Validate and reject unusable or structural configuration overrides at the entry point (where configuration is parsed or applied) by restoring them to safe defaults, rather than adding defensive guards in downstream utility functions.

[harper-joseph]

Done in dc6ab4a — both marked nonEmpty. (A malformed-but-non-empty value like 25:99 or a bad IANA name still passes; if that ever bites, a pattern field def on the schema is the natural extension.)

[gemini-code-assist]

The securityToken.header option specifies the header name used to authenticate prerender requests. If configured as an empty string, it will result in an invalid header name when constructing fetch requests, potentially causing runtime errors. Marking it as nonEmpty: true ensures that empty overrides are safely rejected.

Suggested change

	header: option('x-harper-renderer-bypass', 'Header name carrying the token.'),
	header: option('x-harper-renderer-bypass', 'Header name carrying the token.', { nonEmpty: true }),

References

1. Validate and reject unusable or structural configuration overrides at the entry point (where configuration is parsed or applied) by restoring them to safe defaults, rather than adding defensive guards in downstream utility functions.

[harper-joseph]

Done in dc6ab4a — origin.securityToken.header is now nonEmpty.

[gemini-code-assist]

The debugHeader.key option specifies the request header name that enables debug response headers. If configured as an empty string, it will result in an invalid header name. Marking it as nonEmpty: true ensures that empty overrides are safely rejected.

Suggested change

	key: option('x-harper-prerender-debug', 'Request header name that turns on debug response headers.'),
	key: option('x-harper-prerender-debug', 'Request header name that turns on debug response headers.', { nonEmpty: true }),

References

1. Validate and reject unusable or structural configuration overrides at the entry point (where configuration is parsed or applied) by restoring them to safe defaults, rather than adding defensive guards in downstream utility functions.

[harper-joseph]

Done in dc6ab4a — debugHeader.key is now nonEmpty.

[harper-joseph]

Addressed all four review findings in dc6ab4a:

• nonEmpty vs null/undefined (high): validator now rejects them explicitly. Note it was unreachable through applyOptions (mergeInto skips null/undefined overrides and defaults populate every path), but the validator is now self-contained rather than relying on that.
• sitemap.refreshTime / sitemap.timezone, origin.securityToken.header, debugHeader.key: all marked nonEmpty.

Added a test covering the four new nonEmpty rejections. 359/359 tests pass, lint + format clean.

[harper-joseph]

Rebased onto main (v0.26.0 bot_serve/page_age and v0.27.0 crawl-breadth landed underneath) and re-stamped v0.25.0 → v0.28.0:

• crawlStats options ported into the schema with descriptions + bounds
• the crawl-stats lazy flush timer now follows crawlStats.enabled/flushInterval live, matching the other timers in this PR (disabling flushes pending sketches rather than holding them until re-enable)

377/377 tests, lint + format clean.