Skip to content

v1.1.0

Latest

Choose a tag to compare

@github-actions github-actions released this 01 Aug 07:02
8fac0c7

Added

  • Search as you type: a suggestions dropdown that populates while the visitor types, with the full pipeline still firing only on commit (assets/js/scolta.js, assets/css/scolta.css, src/Config/ScoltaConfig.php, docs/SAYT.md). Typing did exactly two things before this: toggle the clear button and call schedulePreload(), which warms an index chunk behind a 150 ms debounce. A search ran only from the Enter keydown handler, the search button, or a programmatic doSearch(). Nothing suggested, nothing completed, and the visitor got no feedback at all until they committed. Now a debounced suggest cycle runs pagefind.search(prefix, {}), loads at most sayt_max_suggestions fragments, scores them through the same path the result list uses, dedupes by title, merges matching recent searches ahead of them, and renders a dropdown. The suggest path and doSearch() are separate machines and share nothing but the input element and the index. The suggest path never writes #scolta-results, the results header, the facet panel or history, and never triggers a summary; doSearch() cancels any pending or in-flight suggest work, closes the dropdown, and holds the suggest path off until its primary paint lands. Staleness is a version counter, not an AbortController: the cycle takes a number at the top and re-checks it after every await, so a cycle superseded by newer input, by a cleared input or by a committed search returns having performed zero DOM writes and loaded zero further fragments — pinned per await point in tests/js/sayt.test.js. pagefind.debouncedSearch() was rejected again here, for the same reason it was rejected for schedulePreload(): it is a plain input debounce and adopting it would bypass the staleness guards the multi-phase pipeline depends on. No suggest search ever carries a filters object. Naming a dimension makes Pagefind fetch that dimension's filter chunk, and on an index without the scolta.facets artifact a loaded chunk taxes every later search with a per-matched-result scan that nothing can unload short of destroy(); a keystroke-rate path must never be what triggers that, so suggestions run against the whole index and ignore active facets. Multi-word prefixes reuse the result path's AND-to-OR fallback at reduced weight, so a half-typed second word still suggests. Ten new top-level config properties (sayt_enabled, sayt_min_chars, sayt_debounce_ms, sayt_max_suggestions, sayt_recent_searches, sayt_max_recent, sayt_expand, sayt_expand_per_minute, sayt_expansion_delay_ms, sayt_suggestion_action; defaults true, 2, 150, 6, true, 3, true, 6, 500, navigate), emitted top-level by toBrowserConfig() rather than as scoring keys — the hideEmptyFacets pattern — so toJsScoringConfig() stays at exactly 40 keys. sayt_min_chars counts graphemes, not UTF-16 code units (Intl.Segmenter with a spread fallback), because "🇮🇹" is one character to the person typing it and four to .length; CJK sites generally want 1. sayt_suggestion_action governs title suggestions only: navigate renders each as a real anchor carrying the same sanitized URL the result card uses, so middle-click and ctrl-click work; search puts the title in the box and runs the full search. A recent-search suggestion always runs the search — navigating to a stored query string is meaningless. An unknown value clamps to navigate in both PHP (normalizedSaytSuggestionAction()) and the browser, the latter with one logged warning. sayt_expand_per_minute is a client-side sliding window over AI enrichment calls, and it is not an optimization: SAYT expansions share the platform's AI flood budget with committed searches (Drupal's default is 60 requests per IP per minute across expansion, summarize and follow-up), so an unbudgeted suggest path would spend a visitor's whole allowance on prefixes they never submitted and leave the search they actually ran with no expansion and no summary. Over the cap, and on every other enrichment failure — network error, abort, degraded response, no new terms, superseded cycle — the dropdown degrades silently to the keyword suggestions already on screen, with one debugLog line and no user-visible error. Enrichment fires once per settled prefix after sayt_expansion_delay_ms of idle, never per keystroke. Recent searches live in localStorage under one key, scolta:recent-searches; five are stored, sayt_max_recent that prefix- or substring-match are shown, and every access is wrapped because Safari private browsing throws on write and some enterprise policies throw on read — a storage failure degrades to "no recent searches" and never touches the search box. Stored values are strings the visitor typed and are escaped on render exactly like index metadata. ARIA is the combobox pattern: role="combobox" plus aria-autocomplete, aria-expanded and aria-controls on the input, role="listbox" on the dropdown with role="option" children carrying stable ids, and the active option tracked through aria-activedescendant and aria-selected while DOM focus never leaves the input. Arrows wrap at both ends; Escape closes without clearing and a second Escape is not consumed; Enter with no active option runs doSearch() exactly as it always has. The dropdown is absolutely positioned inside the input wrapper so opening, updating and closing cause zero layout shift, and it is styled through seven --scolta-sayt-* custom properties with no selector more specific than one class. Two new render-seam events, scolta:before-suggestions-render ({container, query}) and scolta:suggestions-rendered ({container, suggestions, query}), dispatched on the dropdown element, bubbling, non-cancellable, listener exceptions caught and logged — matching the four existing lifecycle events exactly and documented alongside them in docs/RENDER_SEAM.md. No suggestion-renderer registration API: a suggestion row is a title and an excerpt, and CSS is the seam for it. sayt_enabled: false restores the pre-1.1.0 widget byte-for-byte — no dropdown node in the scaffold, no combobox roles on the input, no storage access on any path, and an input listener that toggles the clear button and warms the chunk and nothing else; existing indexes need no rebuild either way. Coverage: a new tests/js/sayt.test.js (45 cases — debounce collapse, the grapheme floor, the fragment-load cap, an assertion that no suggest search carries a filters option, staleness at each await, the full keyboard and pointer contract, both suggestion actions plus the recent-always-searches rule and the unknown-value clamp, recent searches on and off with the escaping and storage-throws paths, doSearch suppression, the expansion budget stopping and recovering across the window roll, expansion staleness, and the off switch); seven hostile-input cases added to tests/js/security-render.test.js covering fragment titles, excerpts, expansion terms and stored values; a suggest-cycle document-load tripwire in tests/js/result-count-baseline.test.js asserting the cap is sayt_max_suggestions and not MAX_PAGEFIND_RESULTS; tests/E2E/sayt.spec.js driving the whole feature in a real browser against an index built by the PHP indexer; and PHP cases in tests/ScoltaConfigTest.php for every property including one that reads the browser bundle and asserts each PHP default matches the fallback literal, since a page rendered by an adapter that omits a key must behave identically to one that emits it. No existing test fixture needed saytEnabled: false: every suite that drives a search calls doSearch() directly rather than firing input events, and the one suite that does fire them (preload.test.js) runs against a mock returning no results. Three robustness details worth naming because each was a real failure mode. The window in which doSearch() suppresses the suggest path is owned by a search version, not flagged by a boolean, and the whole of it sits inside a try/finally. Both halves matter: a boolean lets the FIRST of two overlapping doSearch() cycles unsuppress the suggest path in the middle of the second cycle's paint, and guarding only the awaited region leaves the synchronous setup before it — the URL sync, the scaffold writes, the lifecycle emits — able to escape without releasing the window, which kills suggestions silently and permanently for the rest of the page's life. Only the cycle that opened the window releases it, and it releases it on every exit path. Both failure modes have their own regression test, each verified to fail against the unguarded shape; a suggest search that genuinely fails takes the dropdown down rather than leaving the previous prefix's suggestions on screen claiming to describe the new one; and fragment loading uses Promise.allSettled, so one fragment that 404s costs the user that one suggestion rather than the whole dropdown, with recent searches surviving even a total fragment-load failure since they do not depend on the index.

  • A platform render seam: lifecycle events plus a result-renderer registration API, so a host can own result markup instead of rewriting Scolta's (assets/js/scolta.js, docs/RENDER_SEAM.md). renderResults() built every card from one template string over five indexed fields, and the bundle dispatched no DOM events at all (verified across the whole file), so a platform that renders results as something richer — a thumbnail, taxonomy badges, a rating, an author with an avatar, a bookmark control — had nowhere to hook and nothing to replace. Two additions close that. Four events, dispatched on the element being written, bubbling, non-cancellable: scolta:before-results-render ({container, reason}) and scolta:results-rendered ({container, results, rendered, reused, appended, query}) around every write to #scolta-results, and scolta:before-filters-render / scolta:filters-rendered ({container}) around the #scolta-filters write. container always equals event.target. reason is loading (the "Searching..." placeholder, written from doSearch() and from renderResults()'s expansion-in-flight branch), search (the primary paint), expansion (the repaint after AI query expansion resolves), or append (the "show more" write). results is every scored result in the DOM after the write in DOM order, rendered is the slice this write produced, appended is true only on the additive path, and reused names the results whose DOM node was carried over rather than rebuilt. They are deliberately not cancellable: a render a consumer could veto makes every downstream state assumption conditional, for a use case nobody has yet. A listener that throws is caught and logged rather than taking the render down. Scolta.setResultRenderer(fn), plus a per-instance instance.setResultRenderer(fn) that overrides it, replaces the built-in card with fn(data, ctx) -> string | null. data is the raw fragment; ctx carries index, query, highlightTerms, excerptHtml, titleHtml, titleAttr, safeUrl, urlText, siteHtml, dateHtml and score. Returning anything other than a string falls back to the built-in card for that result alone, which is the correct behaviour for a platform that can render some entity types and not others; a renderer that throws falls back the same way with a console warning. Every ctx value ending in Html/Attr/Text, plus safeUrl, is pre-escaped exactly as the built-in card escapes it, so the easy path is the safe one — in particular excerptHtml is the already-escaped, <mark>-wrapped excerpt, so a platform whose own view mode has no excerpt region can stash it on a placeholder and restore it after a swap without redoing the escaping. query and highlightTerms are raw by design and documented as such: they exist for building a request, not for pasting into markup. The built-in card's own escaping posture is unchanged; the renderer path is opt in and its owner accepts responsibility from the returned string onward. The delegated data-scolta-* click and change handlers are bound once on the mount point, so platform markup carrying those attributes keeps working with no re-binding after any render — documented as part of the contract. This is deliberately a registration function and not a browser config key: a function cannot travel through ScoltaConfig::toBrowserConfig() and a platform's settings JSON, and a browser key PHP never emits would need a BrowserConfigParityTest::REVERSE_ALLOWLIST entry with a written justification. The config surface is untouched and the parity test is unmodified. Do not convert it into a config key later. Covered by tests/js/render-seam.test.js (35 JSDOM tests driving the real bundle).

  • Scolta emits and owns its own facet index, so the browser never loads a Pagefind filter chunk (src/Index/FacetIndexWriter.php, src/Index/StreamingFormatWriter.php, src/Index/PagefindFormatWriter.php, assets/js/scolta.js). Adopting this requires a full index rebuild; until then the browser detects the artifact's absence and falls back to the previous behaviour (see the Fixed entry below). Every built index now carries scolta.facets alongside pagefind-entry.json: a gzipped file holding a JSON header (format, version, the pf_meta hash it was built against, the page count, and every dimension's values with their corpus-wide totals), the page-index-to-fragment-hash table one hash per line, and one self-delimiting posting body per value — tag 0x00 for a varint count followed by that many varint deltas of ascending page indices, tag 0x01 for a ceil(pageCount / 8) bitmap, whichever is smaller for that value. It is keyed on the fragment hash because that is exactly what pagefind.search().results[n].id already returns, so the browser needs no CBOR decoder, no gzip helper beyond the platform's DecompressionStream, and no pf_meta parsing; scolta.js still contains none of those. The header alone answers the facet panel's value list and totals, which are byte-identical to what pagefind.filters() returned because Pagefind reports a value's unfiltered total as its posting-list length. On a 109,308-page corpus carrying 3,208,134 postings across ten dimensions the artifact is 4,060,954 bytes raw and 1,535,451 gzipped, against the 6,142,564 gzipped bytes of .pf_filter chunks it replaces — 25.0%, and Pagefind's own encoding is the reason: it writes each page id as a plain CBOR uint, spending about 2.2 gzipped bytes per posting, where delta-varint and bitmap encoding spend 0.48. Emitting it costs 0.3 s to build plus 0.6 s to gzip on that corpus. In the browser the whole file fetches, gunzips, and decodes in 42 ms (3.1 fetch, 6.8 gunzip, 0.6 header, 21.6 id table, 10.2 postings), and counting a 7,789-result matched set across all 3,970 values takes 2.9–8.5 ms. Written after pf_meta in both writers so it can be stamped with that file's hash: scolta.js compares the stamp against the hash in the cache-busted pagefind-entry.json and refuses a stale cached artifact rather than counting against ids that have moved. Covered by tests/Index/FacetIndexWriterTest.php (header, totals-are-posting-lengths, both posting encodings chosen by size, the contiguous-page-table guard, the gzip envelope, the empty corpus, and a byte-for-byte fixture assertion) and tests/js/facet-index.test.js (13 JSDOM tests reading the very fixture the PHP test pins, so the encoder and the decoder cannot drift apart silently).

  • The three co-occurrence ranking tunables are now settable from config instead of only from the browser defaults (src/Config/ScoltaConfig.php, docs/CONFIG_REFERENCE.md). specificityCooccurrence (float, default 0.9), specificityAgreementGate (float, default 0.45), and specificityAgreementDecay (float, default 1.0) join the existing specificityWeighting / specificityFloor / specificityStrongMatch trio and are emitted from toJsScoringConfig() as SPECIFICITY_COOCCURRENCE / SPECIFICITY_AGREEMENT_GATE / SPECIFICITY_AGREEMENT_DECAY, taking toJsScoringConfig() from 37 keys to the 40 scolta.js reads. The co-occurrence ranking work introduced all three as browser-side reads with ?? fallbacks but added no config property for any of them, so a site could not retune the agreement bonus at all: the only way to change the co-occurrence weighting was to edit the bundle. The defaults are deliberately byte-equal to the JS fallbacks, so this is a pure reachability change with no ranking movement (tests/js/cooccurrence-ranking.test.js pins a ranked order while leaving the keys unset and is unaffected). fromArray() needs no change: it converts snake_case to camelCase generically and casts by declared property type, so specificity_cooccurrence and friends work the moment the property exists, including the string values CMS config layers store. specificityCooccurrence: 0 reproduces the prior maximum-only merge exactly. Covered by tests/ScoltaConfigTest.php (defaults pinned against the JS fallbacks, snake_case mapping, the zero case, string-to-float coercion, and the toJsScoringConfig() key-count assertion raised from 37 to 40).

  • A browser-config parity guard, so a key the bundle reads can no longer be settable from nowhere (tests/Config/BrowserConfigParityTest.php). Root cause of the three missing tunables above, and of the same class of gap in the adapters: nothing anywhere asserted that the config a platform emits actually covers what scolta.js reads off it, so a browser-side read could be added with no corresponding config property and CI stayed green. The gap is invisible by construction, because an unsettable key silently falls back to its hardcoded JS default and the feature merely appears not to work. The new test parses the committed assets/js/scolta.js for every key the browser consumes (top-level instanceConfig.<key> reads, the scoring sub-keys in the config return literals, and the endpoints sub-keys), then diffs that set against toBrowserConfig() in both directions, recursing one level so a missing scoring or endpoints sub-key cannot hide behind a passing top-level check. Four forward-allowlisted keys (currentLanguage, allowedLinkDomains, disclaimer, priority_pages) are documented in the test with the reason each is caller-supplied rather than config-derived; the reverse allowlist is empty, as this package emits nothing the browser does not read. Three tripwire assertions on the extracted counts run before any diff, so a reformat of the bundle that breaks the extraction fails loudly instead of passing while asserting nothing. Comments are deliberately not stripped before matching, and the reverse direction uses strict set membership rather than a substring search of the bundle; both choices are justified in the test docblock.

  • hideEmptyFacets opt-out to restore the old show-disabled facet behavior (assets/js/scolta.js, src/Config/ScoltaConfig.php, docs/CONFIG_REFERENCE.md). New top-level ScoltaConfig property (bool, default true, @stability experimental), emitted top-level into window.scolta by toBrowserConfig() (not nested under scoring) and read by renderFilters() as instanceConfig.hideEmptyFacets. With the default, a facet value whose count is zero for the current query is hidden and a dimension whose values are all zero drops its whole group (an active value stays visible and uncheckable even at zero). Set it to false to restore the prior policy: every taxonomy value renders and a zero-count one is shown as a disabled (0) row, keeping the value list positionally fixed. The zero-count rows are only ever emitted disabled under the opt-out; the default no longer emits them at all. Covered by tests/ScoltaConfigTest.php (default, snake_case mapping, string coercion, top-level browser-config emission) and tests/js/faceting.test.js (opt-out renders disabled (0) rows, opt-out keeps all-zero groups, default drops all-zero groups).

  • Co-occurrence ranking: a document that agrees with several query and expansion terms now outranks one matching a single strong term (assets/js/scolta.js). The previous merge kept only each URL's single highest-scoring sub-query (merge_results dedups by max), so a document matching one discriminating term scored the same whether it agreed with the whole intent or just one word — a lone off-topic rare-word match (crisisThe Cuban Missile Crisis) or a lone common-word match (moment → a solemn post) could take the top slot over documents agreeing across the query. searchAndLoadParallel() now accumulates per document: the strongest sub-query sets the base score (identical to the old max-merge, so recall and the exact / rare-single-term paths are untouched), and each ADDITIONAL discriminating term adds SPECIFICITY_COOCCURRENCE of that term's own specificity-scaled score, taken strongest-first and geometrically decayed by SPECIFICITY_AGREEMENT_DECAY so a long enumerative page cannot out-accumulate a focused one through breadth alone. A term counts as a second agreement axis only if its specificity clears SPECIFICITY_AGREEMENT_GATE, so a near-ubiquitous word earns no agreement bonus. The user's typed words and the discriminating sub-words of an expansion phrase that the admission guard rejected (e.g. grissom out of the AND-phrase Gus Grissom) join purely as agreement axes: they lend co-occurrence credit to documents the real search already found but never introduce a document of their own, so recall is byte-identical and only the ordering changes. The bonus rides alongside the base score as agreementBonus and is applied after the primary/OR merge (applyAgreementBonus), so agreement earned across sub-queries survives the max-merge and can lift a document over a higher lone single-term score from the other path. New browser-side tunables (read from scolta.scoring with defaults): SPECIFICITY_COOCCURRENCE (default 0.9), SPECIFICITY_AGREEMENT_GATE (default 0.45), SPECIFICITY_AGREEMENT_DECAY (default 1), CROSS_LIST_BONUS (default 0.05); SPECIFICITY_COOCCURRENCE: 0 reproduces the prior max-only merge exactly.

  • Specificity-weighted ranking of partial matches, and three tunables to control it (assets/js/scolta.js, src/Config/ScoltaConfig.php). New specificityWeighting (bool, default true), specificityFloor (float, default 0.15), and specificityStrongMatch (float, default 0.55) ScoltaConfig properties, threaded to the browser as SPECIFICITY_WEIGHTING / SPECIFICITY_FLOOR / SPECIFICITY_STRONG_MATCH. Each partial-match sub-query (OR fallback, expansion terms, and the sub-words decomposed from an expansion phrase) is now weighted by how rare its term is in the corpus, so a match on a rare intent-bearing term outranks a match on a ubiquitous one instead of counting the same. The weight is clamp(ln(total/df) / ln(total+1), floor, 1), where df is the term's own Pagefind match count (scoped to the active filters) and total is the corpus size — the exact frequency signal the sub-word guard already computes, so no extra index traffic. A rare term keeps weight ~1 (so corpora that already match on rare terms are unchanged), a ubiquitous term is damped to specificityFloor (never zero, so recall is preserved). Set specificityWeighting: false to restore the previous flat/positional weighting. Documented in docs/CONFIG_REFERENCE.md; covered by tests/ScoltaConfigTest.php.

  • Pagefind index chunks are preloaded while the user types (assets/js/scolta.js, #191). Scolta runs no search until Enter or the search button, so every submitted search also paid for fetching the alphabetical index chunk(s) for the typed term, on top of the expand → merge → summarize pipeline. The input handler now calls a new schedulePreload(), which hands the term to pagefind.preload() — the chunk-resolution half of a search, which bails out before scoring. Chunks are memoized by Pagefind, so the search that fires on submit finds them already resolved. Guards: a 150 ms trailing debounce (PRELOAD_DEBOUNCE_MS) keeps the per-keystroke WASM chunk lookup off the typing path while still firing long before a human reaches Enter; a 2-character floor (PRELOAD_MIN_CHARS) avoids fetching a chunk for a stray character; a last-term check skips repeat work; preload is feature-detected, so index builds from Pagefind releases that predate it are unaffected; and both synchronous throws and rejected promises are swallowed to debugLog, because a cache warm must never break the search box. clearSearch() cancels any pending preload (the clear button does not fire an input event). No filters are passed to preload()initPagefind() already calls pagefind.filters(), which loads every filter chunk. pagefind.debouncedSearch() was deliberately not adopted: it is a plain input debounce, and routing searches through it would bypass the abortController + searchVersion staleness guards that protect the multi-phase pipeline. Covered by tests/js/preload.test.js (9 JSDOM tests driving the real input handler: debounce collapse, trimming, the 1-character floor, the repeat-term skip, cancel-on-clear, the missing-preload path, and both failure modes).

  • Optional $temperature parameter on AiClient::message() and AiClient::conversation() (src/AiClient.php). A trailing ?float $temperature = null is threaded through sendRequest() into both the Anthropic and OpenAI-compatible request builders. When null (the default) no temperature field is included in the request body and the provider default applies; when non-null it is sent to both provider shapes (the Amazee path proxies through the OpenAI-compatible endpoint and inherits that handling). The parameter is optional and trailing, so every existing caller is unaffected.

  • One canonical resolver for the AI API key and its source (src/Config/ApiKeyResolver.php, src/Config/ResolvedApiKey.php, src/Config/ApiKeySource.php, src/Config/AmazeeCredentials.php, src/Health/HealthChecker.php, src/SetupCheck.php, docs/API_KEY_PRECEDENCE.md, scolta-php#252). Root cause, and it is a structural one rather than a wrong value anywhere: the same fact was derived twice, with opposite precedence, and nothing forced the two derivations to agree. scolta-php offered no resolver at all — ScoltaConfig::$aiApiKey is a bare string, and HealthChecker and SetupCheck consumed it as an opaque value — so each adapter invented both halves itself. The effective-config path gave an explicit env or settings key priority and fell through to stored Amazee.ai credentials only when it was empty; the separate getApiKeySource() / get_api_key_source() that every reporting surface called checked Amazee first. A site with a perfectly valid SCOLTA_API_KEY therefore sent every request with its own key while the settings form, the health payload and the CLI all announced "Connected to Amazee.ai (auto-provisioned free trial)" in success green, with nothing anywhere revealing which key was actually in use. That is a diagnostic surface lying in exactly the situation where somebody reaches for it, and it has already cost a misdiagnosis: on the Athenaeum Drupal demo the Amazee message was read as evidence that the environment variable was missing, when a 109-character sk-ant-api03 value was present in the container the whole time and the real defect was elsewhere. The fix is that the resolver returns the source alongside the key, so no caller can re-derive it. ApiKeyResolver::resolve() takes the platform's explicit key candidates in precedence order plus any stored Amazee credentials and returns a ResolvedApiKey carrying the key, its ApiKeySource, the effective provider, the base URL, whether Amazee credentials are stored at all, and whether an Amazee install is still awaiting model resolution. Adapters take the provider from that object rather than setting it beside the key, and answer isAmazeeActive() from ResolvedApiKey::isAmazee() rather than asking the credential store — stored credentials that lost to an explicit key are not active, which is the precise conflation the old predicates encoded. source distinguishes amazee:operator from amazee:auto, because a provider somebody chose and a free trial that provisioned itself mean different things to an operator reading a status line. A stored-but-overridden credential is now stated rather than hidden (amazeeOverridden(), and describe() renders it as "Amazee.ai credentials stored but overridden by the SCOLTA_API_KEY environment variable"), and severity() returns warning for it: rendering that state in success green is what let the status line be read as proof that Amazee was serving traffic. HealthChecker gains an optional $resolvedKey and reports ai_key_source and ai_amazee_overridden, takes ai_provider from the resolution, and stops calling a half-provisioned Amazee install usable; SetupCheck::run() gains the same optional argument and prints describe() verbatim, so check-setup cannot word the key differently from the settings UI. Both parameters are optional and trailing, so adapters that have not moved over are unaffected. The precedence is documented, not merely implemented, in docs/API_KEY_PRECEDENCE.md: explicit keys in platform order, then eligible Amazee credentials, then nothing, with the per-platform candidate lists and the rules adapters follow. Two scoping decisions are deliberate. Amazee eligibility is an input, not an inference: Drupal's drupal_ai provider manages its own key and model, so the adapter passes amazeeEligible: false and the resolution reports none while still reporting the credentials as stored, rather than pretending they do not exist. A half-provisioned Amazee install reports its source but returns an empty key, preserving the existing behaviour for a reason worth restating: the gateway rejects the shipped dated default with HTTP 400, breaking AI permanently and silently, whereas a key-less client degrades to an unexpanded, unsummarized HTTP 200 — the same path as an unconfigured site — and the state self-heals when model resolution next succeeds. Covered by tests/Config/ApiKeyResolverTest.php, whose four-by-two matrix runs every source (env, settings, Amazee, none) with and without stored Amazee credentials and asserts the resolver, the health payload and the CLI setup check agree in every cell, and by tests/Config/ApiKeySourceSingleDerivationTest.php, which asserts the structural property rather than the symptom: no file outside the three resolver classes names an ApiKeySource case, and nothing outside the Amazee subsystem reads the credential store. Making the two derivations agree would have left them free to drift again; there is now only one.

Changed

  • The shared enrollment helper no longer establishes a managed gateway connection on its own; establishing one is an explicit caller action (src/AiProvider/Amazee/AutoProvisioner.php). AutoProvisioner::ensureAiAvailable() now does exactly one thing: when credentials are already stored and the caller's hasResolvedModels predicate reports that model names are not resolved, it re-resolves them against the stored key and fires onModelsResolved. With no stored credentials it is a no-op that returns false and sends nothing, and it never returns true. A managed gateway connection is established only by a caller that explicitly calls AmazeeTrialProvisioner::provision() from an operator-initiated enable path; those building blocks (AmazeeTrialProvisioner::provision(), AmazeeClient::provisionTrial()) are unchanged, and there is no configuration flag that routes back through the helper. Detection stays where it was: KeyExpiryRecovery is untouched, so a connection that stops being accepted still degrades /health and sets the persistent marker that prompts an admin to re-authenticate — detect and flag, never replace. The ensureAiAvailable() signature is unchanged for callers, and its return type stays bool (now always false). Covered by tests/AiProvider/Amazee/AutoProvisionerTest.php: two cases assert that with nothing stored a spy client records zero outbound requests and the storage is never written, including when the full adapter callback set is supplied; the stored-key model self-heal keeps its own cases, one of which asserts the only endpoint the heal reaches is the model list on the stored URL.
  • init() no longer destroys what is already inside the mount point (assets/js/scolta.js). It ran root.innerHTML = scaffold unconditionally. autoInit() guarded on !container.hasChildNodes(), but a platform bridge calling Scolta.init() directly bypassed that guard entirely, so any platform attempting server-side rendering into the container silently lost its markup — a trap for exactly the integration the render seam exists to support, and one that has already cost an investigation. The scaffold is now built in a <template> and inserted at the top of the container; existing children are left in place with their node identity intact, so listeners, bindings and swapped-in markup all survive. Every top-level scaffold node carries data-scolta-scaffold, which gives a re-init a precise way to remove its own previous UI (duplicate ids would be worse than useless) and lets destroy() remove exactly what init() inserted instead of clearing the whole mount point. autoInit()'s guard changes accordingly, from "the container has any children" to "the container already holds a Scolta scaffold", so a mount point holding server-rendered content now auto-initialises where it previously refused. The old clear-everything behaviour is available per mount point with data-scolta-replace on the container element — a DOM attribute rather than a config key, because markup decisions belong on the platform side of the seam and this keeps the browser config surface untouched. The createInstance() success check moves from root.hasChildNodes() (which no longer distinguishes success from failure) to the presence of a cached els.results.

Fixed

  • /health reported an authentication failure that was over, and sometimes one that never happened (src/AiProvider/Amazee/KeyExpiryRecovery.php, src/Service/AiServiceAdapter.php, src/Health/HealthChecker.php, docs/HEALTH_REFERENCE.md, scolta-php#250). Root cause, and it is one sentence: a health marker with a write path and no clear path, reported under a name broader than the condition it detects. KeyExpiryRecovery::recordAuthFailure() writes a cache marker when an AI call is rejected for authentication, and HealthChecker reads it as an opaque boolean to report ai_auth_failing, ai_usable: false and status: degraded. Nothing in the repository ever unset it. Its own docblock promised health would report AI unusable "until calls succeed again or the marker ages out" and only the second half existed — there was a clearUpgradeNeeded() for the sibling marker and no clearAuthFailure() anywhere — so a site whose credentials were fixed kept reporting a credential failure for up to the full hour of AUTH_FAILURE_TTL, with no documented way to clear it and no TTL or timestamp in the payload to suggest the report might be stale. The name was wrong as well as the state. isAuthFailure() is narrow by design (it excludes budget exhaustion and matches only ApiKeyInvalidException or four message markers) but it walks the exception chain, and a ModelProviderMismatchException carries the provider's original 4xx as its previous exception with a summary of the response body in the message — so a gateway that words an unknown-model rejection as an authentication error set the marker for a failure that had nothing to do with credentials. On the Athenaeum Drupal demo that produced ai_configured: true, ai_usable: false, ai_auth_failing: true on a site whose real fault was a configured model name, in the one place an operator looks when something is broken. Three changes. The marker clears on the first successful call: noteCallSucceeded() clears it (reading before writing, so a healthy site pays a cache read and not a cache write per call) and it is invoked from a single guardedCall() wrapper in AiServiceAdapter that every AI call path now funnels through — the failure hooks were already attached per method and the success side was simply missing from all three, so the fix is one funnel rather than three call sites, and a call path added later cannot record failures while never reporting a recovery. The marker is reported with its age: ai_auth_failing_since (the Unix timestamp recordAuthFailure() has always stored and both readers threw away by casting it to bool) and ai_auth_failing_ttl (the 3600s window) join the payload, null when no failure is recorded, so a marker that outlived its cause is visibly stale rather than looking identical to a failure from a second ago. A truthy marker with no usable timestamp reports null rather than inventing "now". The name is made true: isAuthFailure() returns false for a ModelProviderMismatchException by exception type, checked before the message chain is walked, so a mismatch routes to the specific ai_model_provider_mismatch degraded reason it already has rather than being reported as a credential problem, and it no longer flags the site for an admin re-authentication that would send them to fix the wrong thing. Transport failures and other non-authentication provider rejections keep routing to the existing generic ai_error reason and set no marker; no new health field was added for them, because a field that means "something else went wrong" tells an operator nothing the log does not. The marker list itself is unchanged and the detection is not weakened — a genuine 401 or expired_key still records the marker, still turns AI off, and still flags the site for re-authentication; the persistent upgrade-needed marker is untouched and clearing the auth failure deliberately leaves it standing, because a site serving AI again on a replacement key still has dead Amazee credentials. The clearing paths, the TTL, what sets the marker and what does not, and the per-platform commands for a site that cannot make a successful call at all are documented in the new docs/HEALTH_REFERENCE.md. Covered by tests/Health/AuthFailureMarkerLifecycleTest.php, which drives the real adapter call path and asserts against the health payload rather than the cache — clear-on-success across all three call paths, the age fields set and null, a stale marker being computably stale, the mismatch and transport and generic-rejection cases leaving ai_auth_failing false, a genuine auth rejection still setting it, and the upgrade prompt surviving the clear.
  • A gateway-resolved model name left behind by a provider switch degraded AI permanently behind a generic ai_error, with nothing in it for an operator to act on (src/AiProvider/ModelIdentity.php, src/Exception/ModelProviderMismatchException.php, src/AiClient.php, src/Http/AiEndpointHandler.php, src/AiProvider/Amazee/AutoProvisioner.php, scolta-php#251). Root cause, and it is a namespace collision rather than a bad value: Amazee.ai auto-provisioning resolves models through the Amazee LiteLLM gateway, which addresses them by its own aliases (claude-4-5-sonnet), and $onModelsResolved handed those names to adapters without saying where they may be stored. Two adapters wrote them into the same config key an operator uses to name a provider-native model ID (claude-sonnet-4-5-20250929). Gateway-scoped and operator-scoped identifiers were therefore sharing one namespace, and nothing invalidated the stored value when the effective provider changed away from the one that resolved it. The transition that exposes it: the trial expires, or an operator configures a direct provider key, so ai_provider becomes anthropic while ai_model still holds a gateway alias Anthropic's API does not recognise. Every AI request then fails, forever, and the only user-visible signal was {"degraded":true,"degraded_reason":"ai_error"} — the same payload a transient outage produces. Observed live on the Athenaeum Drupal demo on 2026-07-29, where three sibling demos on the shipped default worked with the same API key. The namespace split itself lands in the adapters that own the config keys (scolta-drupal#187, scolta-wp#185, per the ADR recorded on #251); this package contributes the two halves that belong upstream. First, the contract: the $onModelsResolved docblock now states that the resolved name is gateway-scoped and must be persisted to a gateway-scoped key, never to the operator-facing model key — the omission that let two of five binding adapters get it wrong was in the contract, not in either adapter. Second, the tripwire, for sites that carry a poisoned value through migration anyway: a new ModelIdentity::looksNativeFor() recognises whether a model name belongs to a direct provider's namespace, and AiClient uses it to convert an otherwise-anonymous provider rejection into a ModelProviderMismatchException naming both the model and the provider. AiEndpointHandler reports it as degraded_reason: "ai_model_provider_mismatch" (AiEndpointHandler::REASON_MODEL_MISMATCH) rather than ai_error, logs it at error level with ai_model and ai_provider as structured context fields, and surfaces the operator-facing message directly on the follow-up endpoint, which reports rather than degrades. Three scoping decisions are load-bearing and deliberate. The check classifies a failure that already happened; it is never a pre-flight gate. It is a heuristic over naming conventions, not a lookup against a live provider catalogue, so a model released after this code was written can be misjudged — as a classifier that costs a differently-worded error on an already-broken request, as a gate it would take a working site down, which is the exact failure mode this change set exists to remove rather than reintroduce from the other side. A configured base_url disables it entirely, because that means a gateway with its own model namespace (Amazee's proxy, a self-hosted OpenAI-compatible server), where a non-vendor model name is correct rather than wrong — without that scoping the tripwire would fire on a healthy Amazee trial on any unrelated 4xx. The two providers are handled asymmetrically: Anthropic's namespace is small and strictly conventional, so it is matched positively against the three grammars it has ever used (family-first claude-opus-5, dated generation-first claude-3-5-sonnet-20241022, pre-3 claude-2.1), and the required date on the generation-first form is what separates a real ID from the gateway-alias shape, which is otherwise indistinguishable; OpenAI's is open-ended (gpt-*, o1, ft: fine-tunes, operator-named models on compatible servers), so only names that positively belong to a different vendor are rejected there. 401 and 429 keep their own exceptions, so key-expiry recovery and rate-limit handling are unaffected on a poisoned site. Covered by tests/AiProvider/ModelIdentityTest.php (every Anthropic naming scheme recognised, the observed claude-4-5-sonnet alias and the undated generation-first form rejected, the OpenAI namespace not second-guessed, unknown providers and empty models never flagged) and tests/Http/ModelProviderMismatchTest.php (the specific reason on expand and summarize, the operator-facing message on follow-up, the structured log fields, and an ordinary provider failure still reported as ai_error), plus five cases in tests/AiClientTest.php covering the client boundary including the gateway-endpoint exemption and the per-call model override.
  • ContentItem::$metadata was dropped on the timestamp-cache path, so every unchanged entity lost its per-item meta keys (src/Index/CachedContentReference.php, src/Index/TimestampManifest.php). A build keeps two caches: PageWordCache holds tokenized text keyed by content hash, and TimestampManifest holds, per entity, a changed timestamp plus a record with everything needed to rebuild that entity's index entry without loading the entity. On an incremental build a gatherer compares timestamps and, on a match, yields a CachedContentReference built from the stored record instead of a full ContentItem. IndexBuildOrchestrator::makeSlimProxy() reads eight fields off whatever object it is handed, and CachedContentReference could supply only seven: it had no metadata property. filters and sortable round-trip because both are in the manifest record and both are constructor properties; metadata joined ContentItem and the slim proxy later and neither the record nor the reference was updated to match. The loss was completely silent. makeSlimProxy() reads the field as $page->metadata ?? [], and ?? suppresses the undefined-property warning, so the field resolved to an empty array with no notice, no log line and no failing test, and InvertedIndexBuilder then folded that empty array into the fragment's meta map. Because "unchanged" is the normal case for an entity, the loss covered the whole corpus on any build that took the cached path: a consumer putting a per-item meta key in metadata, for example the entity identity keys a Drupal site needs to render a result card, got a corpus of fragments with the key missing and nothing anywhere saying why. An off-by-one is what hid it. A forced build writes no manifest entries and calls no markSeen(), so pruneAndSave() empties the manifest; the next plain build therefore treats every entity as changed, produces a correct index, and repopulates the manifest without metadata. The build after that one is where the keys disappear, so the bug survives one intervening build and a rebuild looks like it fixed the problem. CachedContentReference now carries a trailing, defaulted metadata property, so every existing caller keeps working and an adapter that stores and replays the field gets it through to the fragment. TimestampManifest's two docblocks describing the shape of an items entry, on the class and on put(), now list sortable and metadata; the class stores whatever array it is handed, and the docblocks omitting the field is part of what let the gap go unnoticed. Covered by testCachedContentReferenceCarriesMetadataIntoIndex() in tests/Index/IndexBuildOrchestratorTest.php, a two-build test beside the existing sortable twin that reads the metadata keys back out of the gzipped fragment, plus the new tests/Index/SlimProxyFieldParityTest.php. The parity test exists because a passthrough test per field never proves the set is complete, which is the actual root cause: filters and sortable each had coverage, metadata had none, and the same hole reopens the next time a field joins ContentItem and the slim proxy. It parses the key list out of the makeSlimProxy() array literal, collects the reference's promoted constructor properties by reflection, and diffs in both directions, with a tripwire on the extracted count so a reformat of the method fails loudly instead of passing while asserting nothing. The reverse direction is allowlisted down to entityKey and contentHash, the two cache-lookup keys the orchestrator consumes before it builds the proxy. No index rebuild is required: the manifest is repopulated by the ordinary build cycle.
  • An empty HMAC secret killed the index build with a ValueError instead of simply skipping integrity tagging (src/Index/ChunkWriter.php, src/Index/ChunkReader.php, src/Index/BuildState.php, new src/Index/HmacSecret.php). Chunk integrity tagging has always been optional, and "no secret" was spelled null and only null. ChunkWriter::write() guarded with $hmacSecret !== null, which an empty string passes, so hash_init('sha256', HASH_HMAC, '') threw ValueError: Argument #3 ($key) must not be empty when HMAC is requested and the build died with a stack trace naming neither the setting nor the remedy. This was not an exotic input: any adapter that forwards framework configuration straight through gets '' rather than null when the operator has set no key, so a fresh install hit it on the first build it ever ran (reproduced downstream as tag1consulting/scolta-laravel#110, where config('app.key') on an app that has not run key:generate is string(0) ""). Empty now means unset, integrity tagging is skipped, and CRC32 is still computed exactly as before, so an untagged chunk keeps its corruption detection and remains a valid, readable chunk. Whitespace-only secrets normalise to unset too: " " is never a deliberate key, and accepting it would write a tag only a caller who reproduced the same accidental whitespace could verify, which is worse than no tag because it looks like integrity coverage and is not. A secret that has real content beside whitespace is a real secret and is used verbatim rather than trimmed, so no chunk already written under a padded key stops verifying. The fix had to land at four sites at once, and fixing the writer alone would have replaced one failure with another. With $hmacSecret = '', ChunkReader::verifyFooterDigests() still evaluated '' !== null as true at both its guards and BuildState::readChunk() still demanded hmac === true from a chunk that now deliberately carried no tag, so the build would have failed a second way, on its own freshly written chunks, reporting an HMAC mismatch that was really a missing tag. The predicate is therefore factored into one place, HmacSecret::normalize(), rather than repeated as a ternary in four, so the writer's notion of "tagged" and the reader's notion of "expects a tag" cannot drift apart in a later edit. Normalisation is idempotent, so a value that has already been through it can pass through again safely. ChunkReader::verifyHmac() keeps its non-nullable string signature, since it is @stability stable and cannot change within 1.x; an unset secret participates by returning false, because that method collapses a three-state verdict into a boolean and "not applicable" must never surface as "verified" to a caller using it as an integrity gate. Callers that need to tell a missing tag from a genuine mismatch use verifyFooterDigests(), which keeps null distinct from false. An empty secret is deliberately not promoted to a hard error: callers pass framework configuration that is legitimately empty on a fresh install, and refusing to build would block the evaluation path for no security gain, since the operator who set no key was never getting a tag either way. write()'s @param docblock now states the empty-means-unset rule instead of documenting null as the only spelling. Covered by the new tests/Index/EmptyHmacSecretTest.php (27 cases: round trips across null, '', a single space, mixed whitespace and a real secret; an explicit guard that '' raises no ValueError; that an untagged chunk verifies against no secret at all, so absence of a tag is never mistaken for integrity; that a real secret still tags, still verifies, and still fails against a different secret; that a padded secret is not treated as unset and is not trimmed; that verifyHmac('') returns false without throwing on both tagged and untagged chunks; that BuildState::readChunk() no longer demands a tag for any unset spelling while still rejecting a missing tag when a secret is configured; and that '0', which is falsy in PHP, survives normalisation as real key material). Reported as #249.
  • The facet panel counted the typed query while the result list counted the expansion too, so the two numbers described different sets (assets/js/scolta.js). doSearch() computed queryFacetCounts once from the typed query; mergeExpandedSearchResults() then ran every expansion query through searchAndLoadParallel(), merged the documents into allScoredResults, and called renderFilters() again without recomputing the counts — a comment above that call asserted the panel was deliberately byte-identical before and after expansion, on the argument that an LLM-driven expansion is nondeterministic while the typed query is not. But renderResults() prints allScoredResults.length as the header count, so on every query whose expansion seeded a document the header and the sum of the panel's counts were two different numbers, and the panel was the wrong one. Two symptoms, the second worse: every count read low, so filtering by a value returned more results than the count promised; and under the default hideEmptyFacets policy a value whose only matches came from expansion counted 0 and was hidden entirely, taking its whole dimension group with it when every one of that dimension's values was expansion-derived — the user could not filter on content sitting in front of them in the list. The nondeterminism argument does not survive contact with the fact that the LIST is equally nondeterministic: a panel that agrees with the list beside it is worth more than a panel that repeats run to run while disagreeing with it. Counts are now folded exactly once per typed query, when expansion lands, as countsOf(typed ids) + countsOf(expansion ids \ typed ids) — subtracting the typed set is what keeps a document from being counted twice, and both terms are computed under structural filters only (SKIP_FILTER_DIMENSIONS, typically the auto-language default), never under the user's selection. Everything load-bearing about the old design is preserved and asserted: counts still do not move on a facet click, a sort or a load-more (all preserveFilters cycles, which skip the recompute and reuse the stored counts); counting over allScoredResults was rejected because a page loaded with ?f_difficulty=Beginner would then report 0 for every other value, hideEmptyFacets would hide them, and the user could never change facet value; no filters object reaches searchOpts when the artifact is present, since every search the new pass issues goes through pagefindSearch(); results paint before the count pass in the expansion half of the cycle exactly as they do in the primary half; and the pass re-checks version === searchVersion after its awaits, so a superseded cycle's counts are neither stored nor rendered. Only SEEDING queries count. searchAndLoadParallel() emits no document for a typed term or an agreement-only phrase sub-word — they lend co-occurrence score to documents another query already found — so counting them would put documents in the panel that are not in the list; the query list is passed through from mergeExpandedSearchResults() rather than rebuilt in the count path, because it is assembled behind an async sub-word admission guard and a second copy of that logic would drift. The cost is close to zero. The typed term is unchanged and still exact and uncapped (search.filters over the full matched set, no fragment loads). The delta re-searches each seeding term under structural filters, which is a per-cycle memo hit whenever the user has applied no non-structural facet — the common case — so the pass adds zero additional pagefind.search calls there; with scolta.facets present it then counts by id through facetCountsFor(), which reads nothing but r.id, for zero additional fragment loads; without the artifact it loads the delta's fragments, capped at MAX_PAGEFIND_RESULTS per term exactly as the OR-fallback union already caps, and Pagefind serves them from cache since the result path has just loaded the same ones. Four decisions, stated because they are judgement calls rather than derivations. (1) Dedup identity is the fragment id, which is free on both paths; where fragments are loaded anyway, documents are additionally collapsed by the normalized URL mergeResults() dedups the list by, so the panel and the list agree when one page is indexed under two fragment ids. That normalization is now a single shared helper, and the JS fallback merge — which keyed on the raw URL while the Rust merge normalized — was moved onto it, so /foo and /foo.html are one page under both merges instead of one under WASM and two without it. In mode 1 the typed side loads no fragments and therefore contributes no URLs, so a cross-boundary URL duplicate still counts twice there, exactly as Pagefind's own native counts already do within it. (2) deduplicateByTitle() is not mirrored in the count path: it runs after scoring, on score order, and the count path has no scores. The header can therefore be smaller than a dimension's sum; the relationship is tested (sum equals the header when no title duplicates collapsed) rather than papered over. (3) The native sort branch is covered too — its sortOverride replaces allScoredResults with a URL-deduped union over the whole term set, so the counts cover that same term set, still under structural filters rather than the branch's mergedFilters; when the sort field is absent from every result the union never replaces the list and the typed counts are correctly left alone. (4) An LLM filter_hint can narrow the list mid-cycle without the user clicking anything, and counts deliberately do not follow it — they stay selection-independent, so the panel will over-report relative to the visible list in that case. That is a known and deliberate gap: making counts follow an applied hint reintroduces exactly the "every other value reads 0 and disappears" failure. No config key was added; the old behaviour was a defect, not a policy. Covered by the new tests/js/facet-count-expansion.test.js (the bug pinned as counts and header agreeing, no double counting across queries or within a multi-value document, URL identity collapse, the hideEmptyFacets recovery of both a value and a whole dimension group, agreement-only terms contributing nothing, structural scoping surviving an applied facet, counts frozen across a facet toggle, a superseded cycle's counts never landing, the native sort branch, and the artifact and fallback paths producing identical numbers with zero extra fragment loads on the artifact path — 13 of its 14 cases fail against the unfixed bundle), plus extensions to tests/js/search-memo.test.js (the pass adds zero pagefind.search calls per query string), tests/js/render-order.test.js (the merged list is painted and announced while a gated count pass is still blocked), and tests/js/result-count-baseline.test.js (a tripwire on the delta's per-term fragment cap, so a future change cannot quietly start fetching a whole match set).
  • AI query expansion repainted the entire result list even when nothing about it had changed (assets/js/scolta.js). mergeExpandedSearchResults() set displayedCount = 0 and called renderResults(true), which ran container.innerHTML = html over the whole list. On the common case — the expansion pass returning the same results in the same order — that teardown accomplished nothing and cost every platform enrichment layer a second round of work, one to two seconds after the first, destroying anything lazily loaded into the first paint. renderResults() now reconciles the container by result identity (the URL the card links to; deduplicateByTitle() has already collapsed near-duplicates, so it is unique within a painted list) instead of replacing it. The container is walked in place: a node already sitting where it belongs is left completely alone, a node that belongs elsewhere is moved with insertBefore() (which, on a node already in the document, moves rather than clones or re-parses it, so listeners and swapped-in markup ride along), only genuinely new results are built, and leftovers are removed at the end. An expansion pass that reorders nothing therefore performs zero DOM mutations on the container, verified with a MutationObserver in the test suite — leaving an unmoved node untouched rather than detaching and re-attaching it matters, because a detach and re-attach restarts CSS transitions, reloads an iframe and fires disconnectedCallback / connectedCallback on a custom element. Under a registered result renderer an unchanged order therefore moves no node at all, and a partially changed one rebuilds only the results that are genuinely new. Two deliberate limits, both documented in docs/RENDER_SEAM.md: built-in cards are only carried over while the highlight-term signature is unchanged, because expansion adds highlight terms and a card painted before that carries <mark> spans that are now stale — a naive "skip the repaint when the order is unchanged" optimisation would ship exactly that bug; and reuse is scoped to one search cycle, since every doSearch() opens with the "Searching..." placeholder, which is a genuine teardown and correctly so. Container writes per search go from three (loading, primary, expansion) to two, with the expansion pass still announcing itself so a consumer knows the list was re-evaluated.
  • ContentItem::$metadata was silently dead on the PHP indexer path (src/Index/IndexBuildOrchestrator.php, src/Index/PhpIndexer.php, src/Index/InvertedIndexBuilder.php). makeSlimProxy() and tokenizeItems() each built a seven-field proxy that omitted the field, and InvertedIndexBuilder composed fragment meta from title, date and sortable only, so anything a platform put in metadata never reached the index. The only remaining route to an arbitrary per-item meta key was sortable, which additionally writes a corpus-wide entry into the eagerly loaded pf_meta sorts table; on a 109,308-page corpus that measured as roughly a 25-35% increase in cold bootstrap payload to carry one identifier. Both proxies now carry metadata, and all three build entry points merge it into fragment meta with title/date highest precedence, then sortable, then metadata — sortable wins a key collision because a sortable key also has to line up with the pf_meta sorts table. metadata defaults to [], so no existing corpus sees a byte of difference. Proxies predating the field still build ($item->metadata ?? []). Covered by tests/Index/MetadataPassthroughTest.php, which pins the passthrough at the builder and at both stream entry points and pins both precedence rules, because a metadata key quietly overwriting title would be worse than the bug being fixed.
  • A fragment that failed to JSON-encode was written to disk as twelve bytes of delimiter (src/Index/StreamingFormatWriter.php, src/Index/PagefindFormatWriter.php). Both writers concatenated json_encode()'s return value without checking it. On invalid UTF-8 anywhere in the page that return is false, false concatenates as the empty string, and the fragment file ends up holding nothing but the delimiter — which fails JSON.parse() in the browser, on one result, at search time, with nothing logged anywhere upstream. Both now throw a RuntimeException naming the page number, the URL and json_last_error_msg(), matching the guard FacetIndexWriter::build() already had. Covered by tests/Index/FragmentEncodingFailureTest.php, which asserts on both writers and, for the streaming path, that no truncated fragment file reaches disk.
  • Every search paid Pagefind's filter counter, at a cost proportional to matched results times filter postings, and the first facet click made it permanent (assets/js/scolta.js). This is the underlying per-search cost the duplicate-search fix below explicitly left as "a separate problem". Root cause is upstream, in Pagefind 1.5's pagefind_web/src/filter.rs: SearchIndex::get_filters takes intersect_pages as a Vec<usize> and counts with intersection.contains(&p), a linear scan of the matched-result set, for every (value, page) posting in every loaded filter chunk — and lib.rs calls it twice per search(), once for the filtered counts and once for the totals. bit_set::BitSet is imported at the top of that same file and used by every other function in it; only get_filters takes a Vec. So the cost is matched results x loaded postings, it applies to every search once any chunk is loaded, and there is no unload path short of pagefind.destroy(). Two triggers loaded chunks: initPagefind() called pagefind.filters(), which loads all of them, and passing a dimension in searchOpts.filters makes Pagefind's client lazily fetch that dimension's chunk, so the first facet click reintroduced the whole cost for the life of the page. The cost tracks postings, not distinct values. Measured on a 109,308-page Drupal corpus (3,208,134 postings, 3,970 distinct values across ten dimensions), pagefind.search('math') with 7,789 matched results, medians of three runs each with a paired in-page baseline: no chunk loaded 155 ms; standard_authority (55 values, 6,468 postings) 259 ms; audience (5 values, 10,004 postings) 291 ms; subject (440 values, 389,545 postings) 2,478 ms; grade (19 values, 491,074 postings) 3,014 ms; curriculum (3,421 values, 1,764,954 postings) 10,312 ms; all ten 18,589 ms. A 440-value dimension is cheaper than a 19-value one, and rewriting the served curriculum chunk to collapse 3,421 values into 4 while keeping every posting moved it 10,151 ms to 10,178 ms — 0.3%, where a cost model based on distinct values predicts near-total elimination. Cost is equally linear in matched results: the same all-ten state costs 374 ms for a 166-result query. Fixed by moving both the counting and the filter application out of Pagefind and onto the new scolta.facets artifact. initPagefind() no longer calls pagefind.filters(), pagefindSearch() never populates searchOpts.filters, and the user's selection is applied against the artifact's posting lists (OR within a dimension, AND across dimensions, Pagefind's own semantics) before the result list is sliced and fragments are loaded. Counts are computed over the full matched set, not the ~75 fragments Scolta loads, so no facet count moves: validated against Pagefind's own output on the production corpus with zero mismatches across all 3,970 values in all ten dimensions, for both query-scoped counts and unfiltered totals. A side effect is that the same query under different facet selections is now one Pagefind search shared through the per-cycle memo, with only the cheap post-filter differing. An index built before this change has no artifact, so scolta.js falls back to pagefind.filters() and to Pagefind-side filter application, with a console.warn that names the cost and tells the operator to rebuild the search index; facets keep working, slowly, rather than breaking. The same fallback covers a browser without DecompressionStream, a payload that is not a Scolta facet index, a stale cached artifact whose stamp disagrees with pagefind-entry.json, and an index whose secondary languages were merged in with pagefind.mergeIndex() (one artifact describes one built index, and counting a merged corpus against it would be visibly wrong). Covered by tests/js/facet-index.test.js: pagefind.filters() is never called when the artifact is present, no search ever carries a filters option including after facet clicks in two different dimensions, counts are exact for a 150-result query against a MAX_PAGEFIND_RESULTS of 75, a zero-count value is reported at zero rather than omitted, filters AND across dimensions and OR within one, and each of the five fallback paths warns and still renders facets. The pre-existing suite passes unmodified because it exercises the fallback, so tests/js/result-count-baseline.test.js and tests/js/search-memo.test.js are untouched.
  • PagefindFormatWriter::buildFilterIndex() crashed the build on multi-value filter arrays (src/Index/PagefindFormatWriter.php). A multi-value filter field, which is the normal shape for a taxonomy facet, arrives as an array, and the method used $filterValue directly as an array key. PHP 8 raises TypeError: Cannot access offset of type array for both the isset() probe and the assignment, so the build died outright rather than coercing or skipping. This is the identical defect fixed in StreamingFormatWriter::writePageEntry() in 1.0.0-rc5, which normalized with is_array($v) ? $v : [$v] to match PagefindHtmlBuilder; PagefindFormatWriter was left behind. Not reachable from any shipped pipeline — nothing in src/ instantiates the class, IndexBuildOrchestrator and PhpIndexer both use StreamingFormatWriter, and no adapter constructs it — but it is @stability stable public API, and the facet-index work touches the same method. Extracted to collectFilterData() with the same normalization both other emitters already use, and covered by tests/Index/PagefindFormatWriterTest.php.
  • A filter dimension with one distinct value is no longer emitted as a Pagefind filter chunk (src/Index/StreamingFormatWriter.php, src/Index/PagefindFormatWriter.php). InvertedIndexBuilder injects a site filter whenever ContentItem::$siteName is set and a language filter whenever $language is set (in the Drupal adapter siteName falls back to system.site name, so it is populated on essentially every install), and PagefindHtmlBuilder emits the same two on the binary path. On a single-site single-language index that produces two dimensions with exactly one distinct value each, and such a dimension cannot filter anything — selecting its only value matches every page — nor show a useful count. There was no distinct-value guard anywhere in the index build; renderFilters() in the browser has always had one (Object.keys(taxonomy[dim]).length > 1) and SKIP_FILTER_DIMENSIONS skips site and language outright, so these two dimensions were never rendered as facets and this changes nothing a user sees. What they cost was real: on the 109,308-page corpus they were 485,100 of the 6,142,564 served filter bytes and 218,616 of the 3,208,134 postings, every posting of which Pagefind scanned against the matched-result set on every search. The guard applies only to Pagefind's chunks and their pf_meta references; scolta.facets deliberately still carries every dimension, because Scolta can be asked to apply one (AUTO_LANGUAGE_FILTER applies language) and applying a filter needs its posting list even when no facet renders it — and a bitmap covering every page gzips to about a hundred bytes, so carrying them costs 176 bytes. Covered by tests/Index/PagefindFormatWriterTest.php (a one-value dimension is absent from the emitted chunks while a three-value one is present) and tests/js/facet-index.test.js (the artifact carries it; the panel does not render it).
  • Every query ran its Pagefind search twice, and the results were held behind the second one (assets/js/scolta.js). On a production-size index (a Drupal site with 109,308 indexed pages and 3,970 distinct filter values across 10 dimensions) the query math took 35.9 seconds to paint: 24,516 ms in the primary search, then 11,053 ms in a second, byte-identical search, and only then render — the results were finished in memory at 24,558 ms and did not appear until 35,626 ms. Two independent defects, both fixed here. (1) The duplicate search. computeQueryFacetCounts() derives structuralFilters by keeping only the SKIP_FILTER_DIMENSIONS entries out of the filters it is handed, so whenever the user has applied no user-facing facet — the common case — its search is identical to the primary one; on the OR-fallback path computeUnionFacetCounts() likewise re-ran a per-term search for every term searchAndLoadParallel() had just searched. pagefindSearch() now memoizes in-flight searches for the duration of one search cycle, keyed on the query plus a stable serialization of the resolved Pagefind options (dimension keys and value arrays sorted, sort hint included) rather than the caller's argument, so equivalent scopes share a key and genuinely different ones do not. The memo holds the promise, not the awaited value, so concurrent callers share one in-flight search instead of racing, and it is reset alongside the cycle's other per-cycle state at the top of doSearch(), so nothing is reused across cycles. A user-applied facet makes activeFilters and structuralFilters differ and both searches still run — that is the correctness boundary, since collapsing them would silently change the facet counts, and it is asserted in both directions. (2) First paint blocked on the counts. doSearch() awaited computeQueryFacetCounts() before calling renderResults(), so the user waited for a full second search before seeing the list they asked for. Results now render as soon as they are ready and the count pass runs afterwards, re-rendering the filter panel when it lands, behind a version === searchVersion re-check so a late count pass from a superseded query can neither overwrite queryFacetCounts nor repaint the panel. The panel is deliberately not repainted during the gap: renderFilters() is count-driven (a zero-count value is hidden entirely under the default hideEmptyFacets policy), so painting it against not-yet-updated counts would show the previous query's visible value set and then reshuffle — it instead holds its last painted state, which introduces no new visual state and never flashes empty. Neither change alters what the counts contain or how many results are returned; the underlying per-search cost (Pagefind computes per-value counts across every distinct filter value once filters() has loaded the filter chunks, roughly 1.45 ms per matched result) is untouched and remains a separate problem. pagefind.debouncedSearch() was again deliberately not adopted: it would bypass the abortController + searchVersion guards. Measured on the same corpus, keystroke to first results: fractions 6,584 → 4,571 ms (31% faster), math 35,870 → 24,516 ms (32%), a six-word query 58,844 → 29,339 ms (50%). Covered by tests/js/search-memo.test.js (per-query pagefind.search call counts with no facet, with a user facet, with a structural-only filter, across the OR-fallback path, and across cycle boundaries) and tests/js/render-order.test.js (results in the DOM while a gated count search is still pending; a superseded cycle's late counts never repaint the panel).
  • Quality / experience queries now surface the content that embodies them (src/Prompt/DefaultPrompts.php). A query that names a feeling, reaction, or judgment rather than a topic ("scary moment", "inspiring story") was expanded into synonyms of the adjective ("frightening experience", "terrifying incident"), but authors narrate a tense episode by describing the concrete thing that went wrong (the malfunction, the alarm, the aborted attempt) and almost never label it "scary". The synonym expansion therefore matched the wrong pages and missed the genuinely tense ones, which shared no vocabulary with the expansion at all. This is the same expansion-restates-the-query failure that rules 13-14 (category/context) and rule 16 (named entity) fix, in the variant those rules do not reach: an abstract quality. The mirrored expand_query template picks up rule 17 (QUALITY / EXPERIENCE → CONCRETE INSTANCES), which expands such a query into the concrete events, systems, or situations that embody the quality in prose, forbids restating the query adjective as a synonym, and reconciles the term cap ("up to 6 concrete instances"); rule 15 still bounds it, so it falls back to neutral topic phrasings when unsure. Examples are drawn from two unrelated domains (aviation history, software) and from no Scolta demo corpus. The rule covers every valence, not only things that went wrong: the funny and inspiring cases are named explicitly and their vocabulary ("amusing story", "uplifting narrative") banned, after the model was observed restating the adjective on exactly those queries; its examples are marked as illustrations rather than a term bank, after it was observed emitting them verbatim for an unrelated corpus. The template text is byte-identical to scolta-core's EXPAND_QUERY, enforced by tests/Prompt/PromptTextIdentityTest.php.
  • Agreement-only terms no longer load full documents, so the co-occurrence path stops inflating the per-query loaded-document count (assets/js/scolta.js, #156). Root cause: searchAndLoadParallel() called data() on every sub-query — including the non-seeding terms (the user's typed words and the agreement-only phrase sub-words) — to test whether several terms agreed in the same document, then discarded the non-matching ones before ranking. Loading documents it then threw away pushed the loaded count for the git-manual reference query (stash uncommitted changes) to 135, tripping the result-count-baseline guard (#156, band max 90); the displayed result count was already correct, only the loaded count was inflated. Whether a non-seeding term counts as an agreement axis, and whether a URL is seeded, emitted, or dropped, is decided entirely from the results entry before data() (result-set size, specificity weight, term class, seeding-set membership) — no data() field decides agreement — so the non-seeding documents can be reasoned about by id and never loaded. Fix: full fragments are loaded only for seeding terms; each non-seeding term's matched document ids are taken straight from its search().results and intersected against the already-loaded seeded documents, and a surviving contribution's agreement magnitude is scored against the seeded document's already-loaded fragment instead of a fresh data() copy. The join is keyed by the Pagefind entry id (available before data()) rather than the loaded fragment url. Loaded count for the failing query drops from 135 to 62, back inside the baseline band, with no change to the guard's fixtures, expected values, or thresholds. Shrinking each non-seeding term's scored set to its survivors changes loaded.length, which feeds the positional rank prior (1 - i/(len-1)) — a ranking effect, so tests/js/cooccurrence-ranking.test.js pins the ranked order of a representative multi-term query (and asserts non-seeding terms introduce no documents of their own) to keep the load change from moving results silently.
  • A common word in the query no longer floods the result head over the rare terms that carry the intent (assets/js/scolta.js). On every corpus measured, a high-frequency word — typed, or leaked from an expansion phrase — pulled off-intent documents to the top because the ranker rewarded matching that common word as much as matching the rare discriminating terms. apollo 13 crisis led with generic Apollo posts (matching only "apollo", 131 of 204 posts) while the crisis coverage matching the rare terms ("oxygen tank explosion", "lunar module lifeboat") was buried; a vegetarian dinner returned Prime Rib Dinner (matching only "dinner"); a medical query filled its tail with unrelated conditions sharing "lunar"/"resident" (2820+ of ~3650 pages) rather than the vision terms ("papilledema", 24 pages). The OR fallback weighted every typed term at a flat 0.6 and expansion sub-queries used positional decay — neither reflected rarity. searchAndLoadParallel() now damps each sub-query by a specificity (inverse-document-frequency) weight derived from the term's own scoped Pagefind match count, so a rare term keeps full weight and a ubiquitous one is floored. This closes two guard bypasses without hurting recall: a high-frequency word the user typed (routed through the OR fallback) and a common sub-word leaked from an expansion phrase are both down-weighted at rank time rather than dropped. A naive alternative — ranking by raw count of expansion terms matched — was rejected because the expansion phrases themselves contain common words; counting without rarity weighting just moves the problem. Gated by specificityWeighting; corpora whose query already matches on rare terms (e.g. undo a mistake i committed → revert/reset/restore) are unaffected by construction.
  • The partial-match fallback no longer frames a strong result set as a failure (assets/js/scolta.js). Almost every conversational query trips the OR fallback, because the exact-match test ANDs every typed word and a natural sentence rarely satisfies it. The header then read "no exact matches found, showing partial matches" and the AI-summary context was marked "not representative of the collection" even when a rare, on-intent term had matched exactly. Both now key off whether any high-specificity term matched (hadSpecificMatch), tracked as sub-queries are scored: when a strong term hit, the banner reads "showing best matches" and the summary hedge attributes any gap to the search rather than the collection; only a genuine thin slice (no specific term matched anything) keeps the strong "not representative" marker and the decline.
  • Zero-count facet values are hidden instead of rendered disabled (assets/js/scolta.js). A facet dimension with many values (the social feed's hashtag dimension has hundreds) rendered every value for every query, all but a few at count 0, burying the useful ones. renderFilters() now skips a value whose count is 0 for the current query unless it is actively selected (an active value stays visible and uncheckable), and skips a dimension left with no visible values. Counts are fixed per typed query, so the visible set stays stable across facet clicks.
  • Identifier / proper-noun queries no longer miss the content that describes them (src/Prompt/DefaultPrompts.php, assets/js/scolta.js). A query centred on a named entity or event ("Apollo 13 crisis", "Apollo 1 fire") was expanded into terms that all kept the entity anchor — "Apollo 13 accident", "Apollo 13 explosion", "Apollo 13 oxygen tank failure" — and against a real Pagefind index every one of those returned zero results, because authors write "the mission", "Lovell", "the lifeboat" far more often than they repeat the mission number. The bare in-corpus vocabulary that does match ("oxygen tank explosion" → the crisis post; "lunar module lifeboat" → all three) was never emitted, so the search fell back to generic matches. This is neither a Pagefind bug, nor a numeric-tokenization bug (1969 → 55 docs, 24000 → 19), nor a stale index, nor scoring config: it is the same expansion-restates-the-query failure as the category/context cases fixed by rules 13-14, in the variant those rules do not reach — the restatement preserves an anchor rather than a category name. New rule 16 (NAMED ENTITY / EVENT → DEFINING DETAILS) expands such a query into the concrete details that identify it in prose (participants, components, distinctive phrases, causes, consequences), requires that at least half the terms drop the entity name entirely, and explicitly forbids appending the name to a list of near-synonyms. The term cap is reconciled ("up to 6 defining details"); rule 15 still bounds it, so an unrecognized entity falls back to neutral phrasings rather than acquiring invented participants or parts. Examples are drawn from three unrelated domains (consumer product, vehicle spec, historical event) and from no Scolta demo corpus, so the behaviour cannot be explained by the prompt naming demo content.
  • The AI summary can no longer claim the collection lacks content (src/Prompt/DefaultPrompts.php). The summarize grounding check previously instructed this failure: its CORPUS AWARENESS bullet told the model to say "[site] focuses on [scope], so it doesn't include a dedicated article on [topic]". The model only ever sees one search's slice of the corpus, so it can never support that claim — and it was observed telling a user that a full week-long article arc did not exist. The bullet is replaced by four rules: PARTIAL VIEW (the excerpts are a slice, never the collection), NEVER ASSERT ABSENCE (naming the observed phrasings as banned), WEAK RESULT SETS (attribute a thin result set to this search, not the collection, and suggest narrower terms), and the preserved no-invented-statistics guard from #33. The follow_up grounding check carried the same defect ("This collection doesn't appear to have content on [topic].") and is fixed the same way. The pinned snapshot fixture moves from tests/fixtures/corpus-awareness-bullet.txt to tests/fixtures/absence-grounding-rules.txt.
  • The summarizer is now told when it is looking at a fallback result set (assets/js/scolta.js). When the full query matched nothing, doSearch() rebuilds the list from the broadened OR fallback and the ranked list already warns the user "no exact matches found, showing partial matches" — but summarizeResults() passed that same slice to the LLM with no marker at all, which is what let a thin, off-target set be generalized into a claim about the whole corpus. The context header now carries [No result matched the full query; ...] in exactly that case, and the summarize prompt keys off it. Covered by tests/js/summary-weak-match-signal.test.js (marker present on fallback, absent on a normal search, and ordered ahead of the excerpts).
  • Query expansion now runs at temperature 0 for deterministic terms (src/Service/AiServiceAdapter.php, messageForOperation()). Expansion calls went out with the provider default temperature (1.0), so the same search query produced different expansion terms on every uncached call — flush the cache and re-search, get different terms and different results. Because the displayed result count is the per-term OR-union across the primary query plus every expansion term, this also made expanded counts LLM-volatile between runs (the 2026-07-13 regression work had to re-base its count tripwires on base-only reads to work around exactly this). messageForOperation() now passes temperature: 0.0 when $operation === 'expand_query' and null otherwise. Summarize and follow-up are creative surfaces and intentionally keep the provider default.

Known limitations

  • Facet counts do not exactly match the filtered result list once AI query expansion has run (#265). The panel now recounts after expansion, which it did not do before this release, and on every corpus measured it is strictly closer to the list than the previous behaviour was. It is not yet exact. The count pass runs one search per expansion term, each capped independently at MAX_PAGEFIND_RESULTS, and unions them, so on a large corpus with several terms the tally describes a different population from the ranked list: measured divergences run from off by one on a 204 page blog to a value counted at 1 that returns nothing when clicked on a 12,541 page social feed. Unexpanded queries are exact. Targeted for 1.1.1; the tracking issue records the three fixes considered and why none is a small change.