Added
-
Search as you type: a suggestions dropdown that populates while the visitor types, with the full pipeline still firing only on commit (
assets/js/scolta.js,assets/css/scolta.css,src/Config/ScoltaConfig.php,docs/SAYT.md). Typing did exactly two things before this: toggle the clear button and callschedulePreload(), which warms an index chunk behind a 150 ms debounce. A search ran only from the Enter keydown handler, the search button, or a programmaticdoSearch(). Nothing suggested, nothing completed, and the visitor got no feedback at all until they committed. Now a debounced suggest cycle runspagefind.search(prefix, {}), loads at mostsayt_max_suggestionsfragments, scores them through the same path the result list uses, dedupes by title, merges matching recent searches ahead of them, and renders a dropdown. The suggest path anddoSearch()are separate machines and share nothing but the input element and the index. The suggest path never writes#scolta-results, the results header, the facet panel orhistory, and never triggers a summary;doSearch()cancels any pending or in-flight suggest work, closes the dropdown, and holds the suggest path off until its primary paint lands. Staleness is a version counter, not anAbortController: the cycle takes a number at the top and re-checks it after every await, so a cycle superseded by newer input, by a cleared input or by a committed search returns having performed zero DOM writes and loaded zero further fragments — pinned per await point intests/js/sayt.test.js.pagefind.debouncedSearch()was rejected again here, for the same reason it was rejected forschedulePreload(): it is a plain input debounce and adopting it would bypass the staleness guards the multi-phase pipeline depends on. No suggest search ever carries a filters object. Naming a dimension makes Pagefind fetch that dimension's filter chunk, and on an index without thescolta.facetsartifact a loaded chunk taxes every later search with a per-matched-result scan that nothing can unload short ofdestroy(); a keystroke-rate path must never be what triggers that, so suggestions run against the whole index and ignore active facets. Multi-word prefixes reuse the result path's AND-to-OR fallback at reduced weight, so a half-typed second word still suggests. Ten new top-level config properties (sayt_enabled,sayt_min_chars,sayt_debounce_ms,sayt_max_suggestions,sayt_recent_searches,sayt_max_recent,sayt_expand,sayt_expand_per_minute,sayt_expansion_delay_ms,sayt_suggestion_action; defaultstrue,2,150,6,true,3,true,6,500,navigate), emitted top-level bytoBrowserConfig()rather than as scoring keys — thehideEmptyFacetspattern — sotoJsScoringConfig()stays at exactly 40 keys.sayt_min_charscounts graphemes, not UTF-16 code units (Intl.Segmenterwith a spread fallback), because"🇮🇹"is one character to the person typing it and four to.length; CJK sites generally want1.sayt_suggestion_actiongoverns title suggestions only:navigaterenders each as a real anchor carrying the same sanitized URL the result card uses, so middle-click and ctrl-click work;searchputs the title in the box and runs the full search. A recent-search suggestion always runs the search — navigating to a stored query string is meaningless. An unknown value clamps tonavigatein both PHP (normalizedSaytSuggestionAction()) and the browser, the latter with one logged warning.sayt_expand_per_minuteis a client-side sliding window over AI enrichment calls, and it is not an optimization: SAYT expansions share the platform's AI flood budget with committed searches (Drupal's default is 60 requests per IP per minute across expansion, summarize and follow-up), so an unbudgeted suggest path would spend a visitor's whole allowance on prefixes they never submitted and leave the search they actually ran with no expansion and no summary. Over the cap, and on every other enrichment failure — network error, abort, degraded response, no new terms, superseded cycle — the dropdown degrades silently to the keyword suggestions already on screen, with onedebugLogline and no user-visible error. Enrichment fires once per settled prefix aftersayt_expansion_delay_msof idle, never per keystroke. Recent searches live inlocalStorageunder one key,scolta:recent-searches; five are stored,sayt_max_recentthat prefix- or substring-match are shown, and every access is wrapped because Safari private browsing throws on write and some enterprise policies throw on read — a storage failure degrades to "no recent searches" and never touches the search box. Stored values are strings the visitor typed and are escaped on render exactly like index metadata. ARIA is the combobox pattern:role="combobox"plusaria-autocomplete,aria-expandedandaria-controlson the input,role="listbox"on the dropdown withrole="option"children carrying stable ids, and the active option tracked througharia-activedescendantandaria-selectedwhile DOM focus never leaves the input. Arrows wrap at both ends; Escape closes without clearing and a second Escape is not consumed; Enter with no active option runsdoSearch()exactly as it always has. The dropdown is absolutely positioned inside the input wrapper so opening, updating and closing cause zero layout shift, and it is styled through seven--scolta-sayt-*custom properties with no selector more specific than one class. Two new render-seam events,scolta:before-suggestions-render({container, query}) andscolta:suggestions-rendered({container, suggestions, query}), dispatched on the dropdown element, bubbling, non-cancellable, listener exceptions caught and logged — matching the four existing lifecycle events exactly and documented alongside them indocs/RENDER_SEAM.md. No suggestion-renderer registration API: a suggestion row is a title and an excerpt, and CSS is the seam for it.sayt_enabled: falserestores the pre-1.1.0 widget byte-for-byte — no dropdown node in the scaffold, no combobox roles on the input, no storage access on any path, and aninputlistener that toggles the clear button and warms the chunk and nothing else; existing indexes need no rebuild either way. Coverage: a newtests/js/sayt.test.js(45 cases — debounce collapse, the grapheme floor, the fragment-load cap, an assertion that no suggest search carries a filters option, staleness at each await, the full keyboard and pointer contract, both suggestion actions plus the recent-always-searches rule and the unknown-value clamp, recent searches on and off with the escaping and storage-throws paths, doSearch suppression, the expansion budget stopping and recovering across the window roll, expansion staleness, and the off switch); seven hostile-input cases added totests/js/security-render.test.jscovering fragment titles, excerpts, expansion terms and stored values; a suggest-cycle document-load tripwire intests/js/result-count-baseline.test.jsasserting the cap issayt_max_suggestionsand notMAX_PAGEFIND_RESULTS;tests/E2E/sayt.spec.jsdriving the whole feature in a real browser against an index built by the PHP indexer; and PHP cases intests/ScoltaConfigTest.phpfor every property including one that reads the browser bundle and asserts each PHP default matches the fallback literal, since a page rendered by an adapter that omits a key must behave identically to one that emits it. No existing test fixture neededsaytEnabled: false: every suite that drives a search callsdoSearch()directly rather than firing input events, and the one suite that does fire them (preload.test.js) runs against a mock returning no results. Three robustness details worth naming because each was a real failure mode. The window in whichdoSearch()suppresses the suggest path is owned by a search version, not flagged by a boolean, and the whole of it sits inside atry/finally. Both halves matter: a boolean lets the FIRST of two overlappingdoSearch()cycles unsuppress the suggest path in the middle of the second cycle's paint, and guarding only the awaited region leaves the synchronous setup before it — the URL sync, the scaffold writes, the lifecycle emits — able to escape without releasing the window, which kills suggestions silently and permanently for the rest of the page's life. Only the cycle that opened the window releases it, and it releases it on every exit path. Both failure modes have their own regression test, each verified to fail against the unguarded shape; a suggest search that genuinely fails takes the dropdown down rather than leaving the previous prefix's suggestions on screen claiming to describe the new one; and fragment loading usesPromise.allSettled, so one fragment that 404s costs the user that one suggestion rather than the whole dropdown, with recent searches surviving even a total fragment-load failure since they do not depend on the index. -
A platform render seam: lifecycle events plus a result-renderer registration API, so a host can own result markup instead of rewriting Scolta's (
assets/js/scolta.js,docs/RENDER_SEAM.md).renderResults()built every card from one template string over five indexed fields, and the bundle dispatched no DOM events at all (verified across the whole file), so a platform that renders results as something richer — a thumbnail, taxonomy badges, a rating, an author with an avatar, a bookmark control — had nowhere to hook and nothing to replace. Two additions close that. Four events, dispatched on the element being written, bubbling, non-cancellable:scolta:before-results-render({container, reason}) andscolta:results-rendered({container, results, rendered, reused, appended, query}) around every write to#scolta-results, andscolta:before-filters-render/scolta:filters-rendered({container}) around the#scolta-filterswrite.containeralways equalsevent.target.reasonisloading(the "Searching..." placeholder, written fromdoSearch()and fromrenderResults()'s expansion-in-flight branch),search(the primary paint),expansion(the repaint after AI query expansion resolves), orappend(the "show more" write).resultsis every scored result in the DOM after the write in DOM order,renderedis the slice this write produced,appendedis true only on the additive path, andreusednames the results whose DOM node was carried over rather than rebuilt. They are deliberately not cancellable: a render a consumer could veto makes every downstream state assumption conditional, for a use case nobody has yet. A listener that throws is caught and logged rather than taking the render down.Scolta.setResultRenderer(fn), plus a per-instanceinstance.setResultRenderer(fn)that overrides it, replaces the built-in card withfn(data, ctx) -> string | null.datais the raw fragment;ctxcarriesindex,query,highlightTerms,excerptHtml,titleHtml,titleAttr,safeUrl,urlText,siteHtml,dateHtmlandscore. Returning anything other than a string falls back to the built-in card for that result alone, which is the correct behaviour for a platform that can render some entity types and not others; a renderer that throws falls back the same way with a console warning. Everyctxvalue ending inHtml/Attr/Text, plussafeUrl, is pre-escaped exactly as the built-in card escapes it, so the easy path is the safe one — in particularexcerptHtmlis the already-escaped,<mark>-wrapped excerpt, so a platform whose own view mode has no excerpt region can stash it on a placeholder and restore it after a swap without redoing the escaping.queryandhighlightTermsare raw by design and documented as such: they exist for building a request, not for pasting into markup. The built-in card's own escaping posture is unchanged; the renderer path is opt in and its owner accepts responsibility from the returned string onward. The delegateddata-scolta-*click and change handlers are bound once on the mount point, so platform markup carrying those attributes keeps working with no re-binding after any render — documented as part of the contract. This is deliberately a registration function and not a browser config key: a function cannot travel throughScoltaConfig::toBrowserConfig()and a platform's settings JSON, and a browser key PHP never emits would need aBrowserConfigParityTest::REVERSE_ALLOWLISTentry with a written justification. The config surface is untouched and the parity test is unmodified. Do not convert it into a config key later. Covered bytests/js/render-seam.test.js(35 JSDOM tests driving the real bundle). -
Scolta emits and owns its own facet index, so the browser never loads a Pagefind filter chunk (
src/Index/FacetIndexWriter.php,src/Index/StreamingFormatWriter.php,src/Index/PagefindFormatWriter.php,assets/js/scolta.js). Adopting this requires a full index rebuild; until then the browser detects the artifact's absence and falls back to the previous behaviour (see the Fixed entry below). Every built index now carriesscolta.facetsalongsidepagefind-entry.json: a gzipped file holding a JSON header (format, version, the pf_meta hash it was built against, the page count, and every dimension's values with their corpus-wide totals), the page-index-to-fragment-hash table one hash per line, and one self-delimiting posting body per value — tag0x00for a varint count followed by that many varint deltas of ascending page indices, tag0x01for aceil(pageCount / 8)bitmap, whichever is smaller for that value. It is keyed on the fragment hash because that is exactly whatpagefind.search().results[n].idalready returns, so the browser needs no CBOR decoder, no gzip helper beyond the platform'sDecompressionStream, and nopf_metaparsing;scolta.jsstill contains none of those. The header alone answers the facet panel's value list and totals, which are byte-identical to whatpagefind.filters()returned because Pagefind reports a value's unfiltered total as its posting-list length. On a 109,308-page corpus carrying 3,208,134 postings across ten dimensions the artifact is 4,060,954 bytes raw and 1,535,451 gzipped, against the 6,142,564 gzipped bytes of.pf_filterchunks it replaces — 25.0%, and Pagefind's own encoding is the reason: it writes each page id as a plain CBOR uint, spending about 2.2 gzipped bytes per posting, where delta-varint and bitmap encoding spend 0.48. Emitting it costs 0.3 s to build plus 0.6 s to gzip on that corpus. In the browser the whole file fetches, gunzips, and decodes in 42 ms (3.1 fetch, 6.8 gunzip, 0.6 header, 21.6 id table, 10.2 postings), and counting a 7,789-result matched set across all 3,970 values takes 2.9–8.5 ms. Written afterpf_metain both writers so it can be stamped with that file's hash:scolta.jscompares the stamp against the hash in the cache-bustedpagefind-entry.jsonand refuses a stale cached artifact rather than counting against ids that have moved. Covered bytests/Index/FacetIndexWriterTest.php(header, totals-are-posting-lengths, both posting encodings chosen by size, the contiguous-page-table guard, the gzip envelope, the empty corpus, and a byte-for-byte fixture assertion) andtests/js/facet-index.test.js(13 JSDOM tests reading the very fixture the PHP test pins, so the encoder and the decoder cannot drift apart silently). -
The three co-occurrence ranking tunables are now settable from config instead of only from the browser defaults (
src/Config/ScoltaConfig.php,docs/CONFIG_REFERENCE.md).specificityCooccurrence(float, default0.9),specificityAgreementGate(float, default0.45), andspecificityAgreementDecay(float, default1.0) join the existingspecificityWeighting/specificityFloor/specificityStrongMatchtrio and are emitted fromtoJsScoringConfig()asSPECIFICITY_COOCCURRENCE/SPECIFICITY_AGREEMENT_GATE/SPECIFICITY_AGREEMENT_DECAY, takingtoJsScoringConfig()from 37 keys to the 40scolta.jsreads. The co-occurrence ranking work introduced all three as browser-side reads with??fallbacks but added no config property for any of them, so a site could not retune the agreement bonus at all: the only way to change the co-occurrence weighting was to edit the bundle. The defaults are deliberately byte-equal to the JS fallbacks, so this is a pure reachability change with no ranking movement (tests/js/cooccurrence-ranking.test.jspins a ranked order while leaving the keys unset and is unaffected).fromArray()needs no change: it converts snake_case to camelCase generically and casts by declared property type, sospecificity_cooccurrenceand friends work the moment the property exists, including the string values CMS config layers store.specificityCooccurrence: 0reproduces the prior maximum-only merge exactly. Covered bytests/ScoltaConfigTest.php(defaults pinned against the JS fallbacks, snake_case mapping, the zero case, string-to-float coercion, and thetoJsScoringConfig()key-count assertion raised from 37 to 40). -
A browser-config parity guard, so a key the bundle reads can no longer be settable from nowhere (
tests/Config/BrowserConfigParityTest.php). Root cause of the three missing tunables above, and of the same class of gap in the adapters: nothing anywhere asserted that the config a platform emits actually covers whatscolta.jsreads off it, so a browser-side read could be added with no corresponding config property and CI stayed green. The gap is invisible by construction, because an unsettable key silently falls back to its hardcoded JS default and the feature merely appears not to work. The new test parses the committedassets/js/scolta.jsfor every key the browser consumes (top-levelinstanceConfig.<key>reads, thescoringsub-keys in the config return literals, and theendpointssub-keys), then diffs that set againsttoBrowserConfig()in both directions, recursing one level so a missingscoringorendpointssub-key cannot hide behind a passing top-level check. Four forward-allowlisted keys (currentLanguage,allowedLinkDomains,disclaimer,priority_pages) are documented in the test with the reason each is caller-supplied rather than config-derived; the reverse allowlist is empty, as this package emits nothing the browser does not read. Three tripwire assertions on the extracted counts run before any diff, so a reformat of the bundle that breaks the extraction fails loudly instead of passing while asserting nothing. Comments are deliberately not stripped before matching, and the reverse direction uses strict set membership rather than a substring search of the bundle; both choices are justified in the test docblock. -
hideEmptyFacetsopt-out to restore the old show-disabled facet behavior (assets/js/scolta.js,src/Config/ScoltaConfig.php,docs/CONFIG_REFERENCE.md). New top-levelScoltaConfigproperty (bool, defaulttrue,@stability experimental), emitted top-level intowindow.scoltabytoBrowserConfig()(not nested underscoring) and read byrenderFilters()asinstanceConfig.hideEmptyFacets. With the default, a facet value whose count is zero for the current query is hidden and a dimension whose values are all zero drops its whole group (an active value stays visible and uncheckable even at zero). Set it tofalseto restore the prior policy: every taxonomy value renders and a zero-count one is shown as a disabled(0)row, keeping the value list positionally fixed. The zero-count rows are only ever emitted disabled under the opt-out; the default no longer emits them at all. Covered bytests/ScoltaConfigTest.php(default, snake_case mapping, string coercion, top-level browser-config emission) andtests/js/faceting.test.js(opt-out renders disabled(0)rows, opt-out keeps all-zero groups, default drops all-zero groups). -
Co-occurrence ranking: a document that agrees with several query and expansion terms now outranks one matching a single strong term (
assets/js/scolta.js). The previous merge kept only each URL's single highest-scoring sub-query (merge_resultsdedups by max), so a document matching one discriminating term scored the same whether it agreed with the whole intent or just one word — a lone off-topic rare-word match (crisis→ The Cuban Missile Crisis) or a lone common-word match (moment→ a solemn post) could take the top slot over documents agreeing across the query.searchAndLoadParallel()now accumulates per document: the strongest sub-query sets the base score (identical to the old max-merge, so recall and the exact / rare-single-term paths are untouched), and each ADDITIONAL discriminating term addsSPECIFICITY_COOCCURRENCEof that term's own specificity-scaled score, taken strongest-first and geometrically decayed bySPECIFICITY_AGREEMENT_DECAYso a long enumerative page cannot out-accumulate a focused one through breadth alone. A term counts as a second agreement axis only if its specificity clearsSPECIFICITY_AGREEMENT_GATE, so a near-ubiquitous word earns no agreement bonus. The user's typed words and the discriminating sub-words of an expansion phrase that the admission guard rejected (e.g.grissomout of the AND-phrase Gus Grissom) join purely as agreement axes: they lend co-occurrence credit to documents the real search already found but never introduce a document of their own, so recall is byte-identical and only the ordering changes. The bonus rides alongside the base score asagreementBonusand is applied after the primary/OR merge (applyAgreementBonus), so agreement earned across sub-queries survives the max-merge and can lift a document over a higher lone single-term score from the other path. New browser-side tunables (read fromscolta.scoringwith defaults):SPECIFICITY_COOCCURRENCE(default0.9),SPECIFICITY_AGREEMENT_GATE(default0.45),SPECIFICITY_AGREEMENT_DECAY(default1),CROSS_LIST_BONUS(default0.05);SPECIFICITY_COOCCURRENCE: 0reproduces the prior max-only merge exactly. -
Specificity-weighted ranking of partial matches, and three tunables to control it (
assets/js/scolta.js,src/Config/ScoltaConfig.php). NewspecificityWeighting(bool, defaulttrue),specificityFloor(float, default0.15), andspecificityStrongMatch(float, default0.55)ScoltaConfigproperties, threaded to the browser asSPECIFICITY_WEIGHTING/SPECIFICITY_FLOOR/SPECIFICITY_STRONG_MATCH. Each partial-match sub-query (OR fallback, expansion terms, and the sub-words decomposed from an expansion phrase) is now weighted by how rare its term is in the corpus, so a match on a rare intent-bearing term outranks a match on a ubiquitous one instead of counting the same. The weight isclamp(ln(total/df) / ln(total+1), floor, 1), wheredfis the term's own Pagefind match count (scoped to the active filters) andtotalis the corpus size — the exact frequency signal the sub-word guard already computes, so no extra index traffic. A rare term keeps weight ~1 (so corpora that already match on rare terms are unchanged), a ubiquitous term is damped tospecificityFloor(never zero, so recall is preserved). SetspecificityWeighting: falseto restore the previous flat/positional weighting. Documented indocs/CONFIG_REFERENCE.md; covered bytests/ScoltaConfigTest.php. -
Pagefind index chunks are preloaded while the user types (
assets/js/scolta.js, #191). Scolta runs no search until Enter or the search button, so every submitted search also paid for fetching the alphabetical index chunk(s) for the typed term, on top of the expand → merge → summarize pipeline. The input handler now calls a newschedulePreload(), which hands the term topagefind.preload()— the chunk-resolution half of a search, which bails out before scoring. Chunks are memoized by Pagefind, so the search that fires on submit finds them already resolved. Guards: a 150 ms trailing debounce (PRELOAD_DEBOUNCE_MS) keeps the per-keystroke WASM chunk lookup off the typing path while still firing long before a human reaches Enter; a 2-character floor (PRELOAD_MIN_CHARS) avoids fetching a chunk for a stray character; a last-term check skips repeat work;preloadis feature-detected, so index builds from Pagefind releases that predate it are unaffected; and both synchronous throws and rejected promises are swallowed todebugLog, because a cache warm must never break the search box.clearSearch()cancels any pending preload (the clear button does not fire aninputevent). No filters are passed topreload()—initPagefind()already callspagefind.filters(), which loads every filter chunk.pagefind.debouncedSearch()was deliberately not adopted: it is a plain input debounce, and routing searches through it would bypass theabortController+searchVersionstaleness guards that protect the multi-phase pipeline. Covered bytests/js/preload.test.js(9 JSDOM tests driving the real input handler: debounce collapse, trimming, the 1-character floor, the repeat-term skip, cancel-on-clear, the missing-preloadpath, and both failure modes). -
Optional
$temperatureparameter onAiClient::message()andAiClient::conversation()(src/AiClient.php). A trailing?float $temperature = nullis threaded throughsendRequest()into both the Anthropic and OpenAI-compatible request builders. When null (the default) notemperaturefield is included in the request body and the provider default applies; when non-null it is sent to both provider shapes (the Amazee path proxies through the OpenAI-compatible endpoint and inherits that handling). The parameter is optional and trailing, so every existing caller is unaffected. -
One canonical resolver for the AI API key and its source (
src/Config/ApiKeyResolver.php,src/Config/ResolvedApiKey.php,src/Config/ApiKeySource.php,src/Config/AmazeeCredentials.php,src/Health/HealthChecker.php,src/SetupCheck.php,docs/API_KEY_PRECEDENCE.md, scolta-php#252). Root cause, and it is a structural one rather than a wrong value anywhere: the same fact was derived twice, with opposite precedence, and nothing forced the two derivations to agree. scolta-php offered no resolver at all —ScoltaConfig::$aiApiKeyis a bare string, andHealthCheckerandSetupCheckconsumed it as an opaque value — so each adapter invented both halves itself. The effective-config path gave an explicit env or settings key priority and fell through to stored Amazee.ai credentials only when it was empty; the separategetApiKeySource()/get_api_key_source()that every reporting surface called checked Amazee first. A site with a perfectly validSCOLTA_API_KEYtherefore sent every request with its own key while the settings form, the health payload and the CLI all announced "Connected to Amazee.ai (auto-provisioned free trial)" in success green, with nothing anywhere revealing which key was actually in use. That is a diagnostic surface lying in exactly the situation where somebody reaches for it, and it has already cost a misdiagnosis: on the Athenaeum Drupal demo the Amazee message was read as evidence that the environment variable was missing, when a 109-charactersk-ant-api03value was present in the container the whole time and the real defect was elsewhere. The fix is that the resolver returns the source alongside the key, so no caller can re-derive it.ApiKeyResolver::resolve()takes the platform's explicit key candidates in precedence order plus any stored Amazee credentials and returns aResolvedApiKeycarrying the key, itsApiKeySource, the effective provider, the base URL, whether Amazee credentials are stored at all, and whether an Amazee install is still awaiting model resolution. Adapters take the provider from that object rather than setting it beside the key, and answerisAmazeeActive()fromResolvedApiKey::isAmazee()rather than asking the credential store — stored credentials that lost to an explicit key are not active, which is the precise conflation the old predicates encoded.sourcedistinguishesamazee:operatorfromamazee:auto, because a provider somebody chose and a free trial that provisioned itself mean different things to an operator reading a status line. A stored-but-overridden credential is now stated rather than hidden (amazeeOverridden(), anddescribe()renders it as "Amazee.ai credentials stored but overridden by the SCOLTA_API_KEY environment variable"), andseverity()returnswarningfor it: rendering that state in success green is what let the status line be read as proof that Amazee was serving traffic.HealthCheckergains an optional$resolvedKeyand reportsai_key_sourceandai_amazee_overridden, takesai_providerfrom the resolution, and stops calling a half-provisioned Amazee install usable;SetupCheck::run()gains the same optional argument and printsdescribe()verbatim, socheck-setupcannot word the key differently from the settings UI. Both parameters are optional and trailing, so adapters that have not moved over are unaffected. The precedence is documented, not merely implemented, indocs/API_KEY_PRECEDENCE.md: explicit keys in platform order, then eligible Amazee credentials, then nothing, with the per-platform candidate lists and the rules adapters follow. Two scoping decisions are deliberate. Amazee eligibility is an input, not an inference: Drupal'sdrupal_aiprovider manages its own key and model, so the adapter passesamazeeEligible: falseand the resolution reportsnonewhile still reporting the credentials as stored, rather than pretending they do not exist. A half-provisioned Amazee install reports its source but returns an empty key, preserving the existing behaviour for a reason worth restating: the gateway rejects the shipped dated default with HTTP 400, breaking AI permanently and silently, whereas a key-less client degrades to an unexpanded, unsummarized HTTP 200 — the same path as an unconfigured site — and the state self-heals when model resolution next succeeds. Covered bytests/Config/ApiKeyResolverTest.php, whose four-by-two matrix runs every source (env, settings, Amazee, none) with and without stored Amazee credentials and asserts the resolver, the health payload and the CLI setup check agree in every cell, and bytests/Config/ApiKeySourceSingleDerivationTest.php, which asserts the structural property rather than the symptom: no file outside the three resolver classes names anApiKeySourcecase, and nothing outside the Amazee subsystem reads the credential store. Making the two derivations agree would have left them free to drift again; there is now only one.
Changed
- The shared enrollment helper no longer establishes a managed gateway connection on its own; establishing one is an explicit caller action (
src/AiProvider/Amazee/AutoProvisioner.php).AutoProvisioner::ensureAiAvailable()now does exactly one thing: when credentials are already stored and the caller'shasResolvedModelspredicate reports that model names are not resolved, it re-resolves them against the stored key and firesonModelsResolved. With no stored credentials it is a no-op that returns false and sends nothing, and it never returns true. A managed gateway connection is established only by a caller that explicitly callsAmazeeTrialProvisioner::provision()from an operator-initiated enable path; those building blocks (AmazeeTrialProvisioner::provision(),AmazeeClient::provisionTrial()) are unchanged, and there is no configuration flag that routes back through the helper. Detection stays where it was:KeyExpiryRecoveryis untouched, so a connection that stops being accepted still degrades/healthand sets the persistent marker that prompts an admin to re-authenticate — detect and flag, never replace. TheensureAiAvailable()signature is unchanged for callers, and its return type staysbool(now always false). Covered bytests/AiProvider/Amazee/AutoProvisionerTest.php: two cases assert that with nothing stored a spy client records zero outbound requests and the storage is never written, including when the full adapter callback set is supplied; the stored-key model self-heal keeps its own cases, one of which asserts the only endpoint the heal reaches is the model list on the stored URL. init()no longer destroys what is already inside the mount point (assets/js/scolta.js). It ranroot.innerHTML = scaffoldunconditionally.autoInit()guarded on!container.hasChildNodes(), but a platform bridge callingScolta.init()directly bypassed that guard entirely, so any platform attempting server-side rendering into the container silently lost its markup — a trap for exactly the integration the render seam exists to support, and one that has already cost an investigation. The scaffold is now built in a<template>and inserted at the top of the container; existing children are left in place with their node identity intact, so listeners, bindings and swapped-in markup all survive. Every top-level scaffold node carriesdata-scolta-scaffold, which gives a re-init a precise way to remove its own previous UI (duplicate ids would be worse than useless) and letsdestroy()remove exactly whatinit()inserted instead of clearing the whole mount point.autoInit()'s guard changes accordingly, from "the container has any children" to "the container already holds a Scolta scaffold", so a mount point holding server-rendered content now auto-initialises where it previously refused. The old clear-everything behaviour is available per mount point withdata-scolta-replaceon the container element — a DOM attribute rather than a config key, because markup decisions belong on the platform side of the seam and this keeps the browser config surface untouched. ThecreateInstance()success check moves fromroot.hasChildNodes()(which no longer distinguishes success from failure) to the presence of a cachedels.results.
Fixed
/healthreported an authentication failure that was over, and sometimes one that never happened (src/AiProvider/Amazee/KeyExpiryRecovery.php,src/Service/AiServiceAdapter.php,src/Health/HealthChecker.php,docs/HEALTH_REFERENCE.md, scolta-php#250). Root cause, and it is one sentence: a health marker with a write path and no clear path, reported under a name broader than the condition it detects.KeyExpiryRecovery::recordAuthFailure()writes a cache marker when an AI call is rejected for authentication, andHealthCheckerreads it as an opaque boolean to reportai_auth_failing,ai_usable: falseandstatus: degraded. Nothing in the repository ever unset it. Its own docblock promised health would report AI unusable "until calls succeed again or the marker ages out" and only the second half existed — there was aclearUpgradeNeeded()for the sibling marker and noclearAuthFailure()anywhere — so a site whose credentials were fixed kept reporting a credential failure for up to the full hour ofAUTH_FAILURE_TTL, with no documented way to clear it and no TTL or timestamp in the payload to suggest the report might be stale. The name was wrong as well as the state.isAuthFailure()is narrow by design (it excludes budget exhaustion and matches onlyApiKeyInvalidExceptionor four message markers) but it walks the exception chain, and aModelProviderMismatchExceptioncarries the provider's original 4xx as its previous exception with a summary of the response body in the message — so a gateway that words an unknown-model rejection as an authentication error set the marker for a failure that had nothing to do with credentials. On the Athenaeum Drupal demo that producedai_configured: true,ai_usable: false,ai_auth_failing: trueon a site whose real fault was a configured model name, in the one place an operator looks when something is broken. Three changes. The marker clears on the first successful call:noteCallSucceeded()clears it (reading before writing, so a healthy site pays a cache read and not a cache write per call) and it is invoked from a singleguardedCall()wrapper inAiServiceAdapterthat every AI call path now funnels through — the failure hooks were already attached per method and the success side was simply missing from all three, so the fix is one funnel rather than three call sites, and a call path added later cannot record failures while never reporting a recovery. The marker is reported with its age:ai_auth_failing_since(the Unix timestamprecordAuthFailure()has always stored and both readers threw away by casting it to bool) andai_auth_failing_ttl(the 3600s window) join the payload,nullwhen no failure is recorded, so a marker that outlived its cause is visibly stale rather than looking identical to a failure from a second ago. A truthy marker with no usable timestamp reportsnullrather than inventing "now". The name is made true:isAuthFailure()returns false for aModelProviderMismatchExceptionby exception type, checked before the message chain is walked, so a mismatch routes to the specificai_model_provider_mismatchdegraded reason it already has rather than being reported as a credential problem, and it no longer flags the site for an admin re-authentication that would send them to fix the wrong thing. Transport failures and other non-authentication provider rejections keep routing to the existing genericai_errorreason and set no marker; no new health field was added for them, because a field that means "something else went wrong" tells an operator nothing the log does not. The marker list itself is unchanged and the detection is not weakened — a genuine 401 orexpired_keystill records the marker, still turns AI off, and still flags the site for re-authentication; the persistent upgrade-needed marker is untouched and clearing the auth failure deliberately leaves it standing, because a site serving AI again on a replacement key still has dead Amazee credentials. The clearing paths, the TTL, what sets the marker and what does not, and the per-platform commands for a site that cannot make a successful call at all are documented in the newdocs/HEALTH_REFERENCE.md. Covered bytests/Health/AuthFailureMarkerLifecycleTest.php, which drives the real adapter call path and asserts against the health payload rather than the cache — clear-on-success across all three call paths, the age fields set and null, a stale marker being computably stale, the mismatch and transport and generic-rejection cases leavingai_auth_failingfalse, a genuine auth rejection still setting it, and the upgrade prompt surviving the clear.- A gateway-resolved model name left behind by a provider switch degraded AI permanently behind a generic
ai_error, with nothing in it for an operator to act on (src/AiProvider/ModelIdentity.php,src/Exception/ModelProviderMismatchException.php,src/AiClient.php,src/Http/AiEndpointHandler.php,src/AiProvider/Amazee/AutoProvisioner.php, scolta-php#251). Root cause, and it is a namespace collision rather than a bad value: Amazee.ai auto-provisioning resolves models through the Amazee LiteLLM gateway, which addresses them by its own aliases (claude-4-5-sonnet), and$onModelsResolvedhanded those names to adapters without saying where they may be stored. Two adapters wrote them into the same config key an operator uses to name a provider-native model ID (claude-sonnet-4-5-20250929). Gateway-scoped and operator-scoped identifiers were therefore sharing one namespace, and nothing invalidated the stored value when the effective provider changed away from the one that resolved it. The transition that exposes it: the trial expires, or an operator configures a direct provider key, soai_providerbecomesanthropicwhileai_modelstill holds a gateway alias Anthropic's API does not recognise. Every AI request then fails, forever, and the only user-visible signal was{"degraded":true,"degraded_reason":"ai_error"}— the same payload a transient outage produces. Observed live on the Athenaeum Drupal demo on 2026-07-29, where three sibling demos on the shipped default worked with the same API key. The namespace split itself lands in the adapters that own the config keys (scolta-drupal#187, scolta-wp#185, per the ADR recorded on #251); this package contributes the two halves that belong upstream. First, the contract: the$onModelsResolveddocblock now states that the resolved name is gateway-scoped and must be persisted to a gateway-scoped key, never to the operator-facing model key — the omission that let two of five binding adapters get it wrong was in the contract, not in either adapter. Second, the tripwire, for sites that carry a poisoned value through migration anyway: a newModelIdentity::looksNativeFor()recognises whether a model name belongs to a direct provider's namespace, andAiClientuses it to convert an otherwise-anonymous provider rejection into aModelProviderMismatchExceptionnaming both the model and the provider.AiEndpointHandlerreports it asdegraded_reason: "ai_model_provider_mismatch"(AiEndpointHandler::REASON_MODEL_MISMATCH) rather thanai_error, logs it at error level withai_modelandai_provideras structured context fields, and surfaces the operator-facing message directly on the follow-up endpoint, which reports rather than degrades. Three scoping decisions are load-bearing and deliberate. The check classifies a failure that already happened; it is never a pre-flight gate. It is a heuristic over naming conventions, not a lookup against a live provider catalogue, so a model released after this code was written can be misjudged — as a classifier that costs a differently-worded error on an already-broken request, as a gate it would take a working site down, which is the exact failure mode this change set exists to remove rather than reintroduce from the other side. A configuredbase_urldisables it entirely, because that means a gateway with its own model namespace (Amazee's proxy, a self-hosted OpenAI-compatible server), where a non-vendor model name is correct rather than wrong — without that scoping the tripwire would fire on a healthy Amazee trial on any unrelated 4xx. The two providers are handled asymmetrically: Anthropic's namespace is small and strictly conventional, so it is matched positively against the three grammars it has ever used (family-firstclaude-opus-5, dated generation-firstclaude-3-5-sonnet-20241022, pre-3claude-2.1), and the required date on the generation-first form is what separates a real ID from the gateway-alias shape, which is otherwise indistinguishable; OpenAI's is open-ended (gpt-*,o1,ft:fine-tunes, operator-named models on compatible servers), so only names that positively belong to a different vendor are rejected there. 401 and 429 keep their own exceptions, so key-expiry recovery and rate-limit handling are unaffected on a poisoned site. Covered bytests/AiProvider/ModelIdentityTest.php(every Anthropic naming scheme recognised, the observedclaude-4-5-sonnetalias and the undated generation-first form rejected, the OpenAI namespace not second-guessed, unknown providers and empty models never flagged) andtests/Http/ModelProviderMismatchTest.php(the specific reason on expand and summarize, the operator-facing message on follow-up, the structured log fields, and an ordinary provider failure still reported asai_error), plus five cases intests/AiClientTest.phpcovering the client boundary including the gateway-endpoint exemption and the per-call model override. ContentItem::$metadatawas dropped on the timestamp-cache path, so every unchanged entity lost its per-item meta keys (src/Index/CachedContentReference.php,src/Index/TimestampManifest.php). A build keeps two caches:PageWordCacheholds tokenized text keyed by content hash, andTimestampManifestholds, per entity, a changed timestamp plus a record with everything needed to rebuild that entity's index entry without loading the entity. On an incremental build a gatherer compares timestamps and, on a match, yields aCachedContentReferencebuilt from the stored record instead of a fullContentItem.IndexBuildOrchestrator::makeSlimProxy()reads eight fields off whatever object it is handed, andCachedContentReferencecould supply only seven: it had nometadataproperty.filtersandsortableround-trip because both are in the manifest record and both are constructor properties;metadatajoinedContentItemand the slim proxy later and neither the record nor the reference was updated to match. The loss was completely silent.makeSlimProxy()reads the field as$page->metadata ?? [], and??suppresses the undefined-property warning, so the field resolved to an empty array with no notice, no log line and no failing test, andInvertedIndexBuilderthen folded that empty array into the fragment'smetamap. Because "unchanged" is the normal case for an entity, the loss covered the whole corpus on any build that took the cached path: a consumer putting a per-item meta key inmetadata, for example the entity identity keys a Drupal site needs to render a result card, got a corpus of fragments with the key missing and nothing anywhere saying why. An off-by-one is what hid it. A forced build writes no manifest entries and calls nomarkSeen(), sopruneAndSave()empties the manifest; the next plain build therefore treats every entity as changed, produces a correct index, and repopulates the manifest withoutmetadata. The build after that one is where the keys disappear, so the bug survives one intervening build and a rebuild looks like it fixed the problem.CachedContentReferencenow carries a trailing, defaultedmetadataproperty, so every existing caller keeps working and an adapter that stores and replays the field gets it through to the fragment.TimestampManifest's two docblocks describing the shape of anitemsentry, on the class and onput(), now listsortableandmetadata; the class stores whatever array it is handed, and the docblocks omitting the field is part of what let the gap go unnoticed. Covered bytestCachedContentReferenceCarriesMetadataIntoIndex()intests/Index/IndexBuildOrchestratorTest.php, a two-build test beside the existing sortable twin that reads the metadata keys back out of the gzipped fragment, plus the newtests/Index/SlimProxyFieldParityTest.php. The parity test exists because a passthrough test per field never proves the set is complete, which is the actual root cause:filtersandsortableeach had coverage,metadatahad none, and the same hole reopens the next time a field joinsContentItemand the slim proxy. It parses the key list out of themakeSlimProxy()array literal, collects the reference's promoted constructor properties by reflection, and diffs in both directions, with a tripwire on the extracted count so a reformat of the method fails loudly instead of passing while asserting nothing. The reverse direction is allowlisted down toentityKeyandcontentHash, the two cache-lookup keys the orchestrator consumes before it builds the proxy. No index rebuild is required: the manifest is repopulated by the ordinary build cycle.- An empty HMAC secret killed the index build with a
ValueErrorinstead of simply skipping integrity tagging (src/Index/ChunkWriter.php,src/Index/ChunkReader.php,src/Index/BuildState.php, newsrc/Index/HmacSecret.php). Chunk integrity tagging has always been optional, and "no secret" was spellednulland onlynull.ChunkWriter::write()guarded with$hmacSecret !== null, which an empty string passes, sohash_init('sha256', HASH_HMAC, '')threwValueError: Argument #3 ($key) must not be empty when HMAC is requestedand the build died with a stack trace naming neither the setting nor the remedy. This was not an exotic input: any adapter that forwards framework configuration straight through gets''rather thannullwhen the operator has set no key, so a fresh install hit it on the first build it ever ran (reproduced downstream as tag1consulting/scolta-laravel#110, whereconfig('app.key')on an app that has not runkey:generateisstring(0) ""). Empty now means unset, integrity tagging is skipped, and CRC32 is still computed exactly as before, so an untagged chunk keeps its corruption detection and remains a valid, readable chunk. Whitespace-only secrets normalise to unset too:" "is never a deliberate key, and accepting it would write a tag only a caller who reproduced the same accidental whitespace could verify, which is worse than no tag because it looks like integrity coverage and is not. A secret that has real content beside whitespace is a real secret and is used verbatim rather than trimmed, so no chunk already written under a padded key stops verifying. The fix had to land at four sites at once, and fixing the writer alone would have replaced one failure with another. With$hmacSecret = '',ChunkReader::verifyFooterDigests()still evaluated'' !== nullas true at both its guards andBuildState::readChunk()still demandedhmac === truefrom a chunk that now deliberately carried no tag, so the build would have failed a second way, on its own freshly written chunks, reporting an HMAC mismatch that was really a missing tag. The predicate is therefore factored into one place,HmacSecret::normalize(), rather than repeated as a ternary in four, so the writer's notion of "tagged" and the reader's notion of "expects a tag" cannot drift apart in a later edit. Normalisation is idempotent, so a value that has already been through it can pass through again safely.ChunkReader::verifyHmac()keeps its non-nullablestringsignature, since it is@stability stableand cannot change within 1.x; an unset secret participates by returningfalse, because that method collapses a three-state verdict into a boolean and "not applicable" must never surface as "verified" to a caller using it as an integrity gate. Callers that need to tell a missing tag from a genuine mismatch useverifyFooterDigests(), which keepsnulldistinct fromfalse. An empty secret is deliberately not promoted to a hard error: callers pass framework configuration that is legitimately empty on a fresh install, and refusing to build would block the evaluation path for no security gain, since the operator who set no key was never getting a tag either way.write()'s@paramdocblock now states the empty-means-unset rule instead of documentingnullas the only spelling. Covered by the newtests/Index/EmptyHmacSecretTest.php(27 cases: round trips acrossnull,'', a single space, mixed whitespace and a real secret; an explicit guard that''raises noValueError; that an untagged chunk verifies against no secret at all, so absence of a tag is never mistaken for integrity; that a real secret still tags, still verifies, and still fails against a different secret; that a padded secret is not treated as unset and is not trimmed; thatverifyHmac('')returns false without throwing on both tagged and untagged chunks; thatBuildState::readChunk()no longer demands a tag for any unset spelling while still rejecting a missing tag when a secret is configured; and that'0', which is falsy in PHP, survives normalisation as real key material). Reported as #249. - The facet panel counted the typed query while the result list counted the expansion too, so the two numbers described different sets (
assets/js/scolta.js).doSearch()computedqueryFacetCountsonce from the typed query;mergeExpandedSearchResults()then ran every expansion query throughsearchAndLoadParallel(), merged the documents intoallScoredResults, and calledrenderFilters()again without recomputing the counts — a comment above that call asserted the panel was deliberately byte-identical before and after expansion, on the argument that an LLM-driven expansion is nondeterministic while the typed query is not. ButrenderResults()printsallScoredResults.lengthas the header count, so on every query whose expansion seeded a document the header and the sum of the panel's counts were two different numbers, and the panel was the wrong one. Two symptoms, the second worse: every count read low, so filtering by a value returned more results than the count promised; and under the defaulthideEmptyFacetspolicy a value whose only matches came from expansion counted 0 and was hidden entirely, taking its whole dimension group with it when every one of that dimension's values was expansion-derived — the user could not filter on content sitting in front of them in the list. The nondeterminism argument does not survive contact with the fact that the LIST is equally nondeterministic: a panel that agrees with the list beside it is worth more than a panel that repeats run to run while disagreeing with it. Counts are now folded exactly once per typed query, when expansion lands, ascountsOf(typed ids) + countsOf(expansion ids \ typed ids)— subtracting the typed set is what keeps a document from being counted twice, and both terms are computed under structural filters only (SKIP_FILTER_DIMENSIONS, typically the auto-language default), never under the user's selection. Everything load-bearing about the old design is preserved and asserted: counts still do not move on a facet click, a sort or a load-more (allpreserveFilterscycles, which skip the recompute and reuse the stored counts); counting overallScoredResultswas rejected because a page loaded with?f_difficulty=Beginnerwould then report 0 for every other value,hideEmptyFacetswould hide them, and the user could never change facet value; nofiltersobject reachessearchOptswhen the artifact is present, since every search the new pass issues goes throughpagefindSearch(); results paint before the count pass in the expansion half of the cycle exactly as they do in the primary half; and the pass re-checksversion === searchVersionafter its awaits, so a superseded cycle's counts are neither stored nor rendered. Only SEEDING queries count.searchAndLoadParallel()emits no document for a typed term or an agreement-only phrase sub-word — they lend co-occurrence score to documents another query already found — so counting them would put documents in the panel that are not in the list; the query list is passed through frommergeExpandedSearchResults()rather than rebuilt in the count path, because it is assembled behind an async sub-word admission guard and a second copy of that logic would drift. The cost is close to zero. The typed term is unchanged and still exact and uncapped (search.filtersover the full matched set, no fragment loads). The delta re-searches each seeding term under structural filters, which is a per-cycle memo hit whenever the user has applied no non-structural facet — the common case — so the pass adds zero additionalpagefind.searchcalls there; withscolta.facetspresent it then counts by id throughfacetCountsFor(), which reads nothing butr.id, for zero additional fragment loads; without the artifact it loads the delta's fragments, capped atMAX_PAGEFIND_RESULTSper term exactly as the OR-fallback union already caps, and Pagefind serves them from cache since the result path has just loaded the same ones. Four decisions, stated because they are judgement calls rather than derivations. (1) Dedup identity is the fragment id, which is free on both paths; where fragments are loaded anyway, documents are additionally collapsed by the normalized URLmergeResults()dedups the list by, so the panel and the list agree when one page is indexed under two fragment ids. That normalization is now a single shared helper, and the JS fallback merge — which keyed on the raw URL while the Rust merge normalized — was moved onto it, so/fooand/foo.htmlare one page under both merges instead of one under WASM and two without it. In mode 1 the typed side loads no fragments and therefore contributes no URLs, so a cross-boundary URL duplicate still counts twice there, exactly as Pagefind's own native counts already do within it. (2)deduplicateByTitle()is not mirrored in the count path: it runs after scoring, on score order, and the count path has no scores. The header can therefore be smaller than a dimension's sum; the relationship is tested (sum equals the header when no title duplicates collapsed) rather than papered over. (3) The native sort branch is covered too — itssortOverridereplacesallScoredResultswith a URL-deduped union over the whole term set, so the counts cover that same term set, still under structural filters rather than the branch'smergedFilters; when the sort field is absent from every result the union never replaces the list and the typed counts are correctly left alone. (4) An LLMfilter_hintcan narrow the list mid-cycle without the user clicking anything, and counts deliberately do not follow it — they stay selection-independent, so the panel will over-report relative to the visible list in that case. That is a known and deliberate gap: making counts follow an applied hint reintroduces exactly the "every other value reads 0 and disappears" failure. No config key was added; the old behaviour was a defect, not a policy. Covered by the newtests/js/facet-count-expansion.test.js(the bug pinned as counts and header agreeing, no double counting across queries or within a multi-value document, URL identity collapse, thehideEmptyFacetsrecovery of both a value and a whole dimension group, agreement-only terms contributing nothing, structural scoping surviving an applied facet, counts frozen across a facet toggle, a superseded cycle's counts never landing, the native sort branch, and the artifact and fallback paths producing identical numbers with zero extra fragment loads on the artifact path — 13 of its 14 cases fail against the unfixed bundle), plus extensions totests/js/search-memo.test.js(the pass adds zeropagefind.searchcalls per query string),tests/js/render-order.test.js(the merged list is painted and announced while a gated count pass is still blocked), andtests/js/result-count-baseline.test.js(a tripwire on the delta's per-term fragment cap, so a future change cannot quietly start fetching a whole match set). - AI query expansion repainted the entire result list even when nothing about it had changed (
assets/js/scolta.js).mergeExpandedSearchResults()setdisplayedCount = 0and calledrenderResults(true), which rancontainer.innerHTML = htmlover the whole list. On the common case — the expansion pass returning the same results in the same order — that teardown accomplished nothing and cost every platform enrichment layer a second round of work, one to two seconds after the first, destroying anything lazily loaded into the first paint.renderResults()now reconciles the container by result identity (the URL the card links to;deduplicateByTitle()has already collapsed near-duplicates, so it is unique within a painted list) instead of replacing it. The container is walked in place: a node already sitting where it belongs is left completely alone, a node that belongs elsewhere is moved withinsertBefore()(which, on a node already in the document, moves rather than clones or re-parses it, so listeners and swapped-in markup ride along), only genuinely new results are built, and leftovers are removed at the end. An expansion pass that reorders nothing therefore performs zero DOM mutations on the container, verified with aMutationObserverin the test suite — leaving an unmoved node untouched rather than detaching and re-attaching it matters, because a detach and re-attach restarts CSS transitions, reloads an iframe and firesdisconnectedCallback/connectedCallbackon a custom element. Under a registered result renderer an unchanged order therefore moves no node at all, and a partially changed one rebuilds only the results that are genuinely new. Two deliberate limits, both documented indocs/RENDER_SEAM.md: built-in cards are only carried over while the highlight-term signature is unchanged, because expansion adds highlight terms and a card painted before that carries<mark>spans that are now stale — a naive "skip the repaint when the order is unchanged" optimisation would ship exactly that bug; and reuse is scoped to one search cycle, since everydoSearch()opens with the "Searching..." placeholder, which is a genuine teardown and correctly so. Container writes per search go from three (loading, primary, expansion) to two, with the expansion pass still announcing itself so a consumer knows the list was re-evaluated. ContentItem::$metadatawas silently dead on the PHP indexer path (src/Index/IndexBuildOrchestrator.php,src/Index/PhpIndexer.php,src/Index/InvertedIndexBuilder.php).makeSlimProxy()andtokenizeItems()each built a seven-field proxy that omitted the field, andInvertedIndexBuildercomposed fragmentmetafrom title, date and sortable only, so anything a platform put inmetadatanever reached the index. The only remaining route to an arbitrary per-item meta key wassortable, which additionally writes a corpus-wide entry into the eagerly loadedpf_metasorts table; on a 109,308-page corpus that measured as roughly a 25-35% increase in cold bootstrap payload to carry one identifier. Both proxies now carrymetadata, and all three build entry points merge it into fragmentmetawithtitle/datehighest precedence, thensortable, thenmetadata— sortable wins a key collision because a sortable key also has to line up with thepf_metasorts table.metadatadefaults to[], so no existing corpus sees a byte of difference. Proxies predating the field still build ($item->metadata ?? []). Covered bytests/Index/MetadataPassthroughTest.php, which pins the passthrough at the builder and at both stream entry points and pins both precedence rules, because a metadata key quietly overwritingtitlewould be worse than the bug being fixed.- A fragment that failed to JSON-encode was written to disk as twelve bytes of delimiter (
src/Index/StreamingFormatWriter.php,src/Index/PagefindFormatWriter.php). Both writers concatenatedjson_encode()'s return value without checking it. On invalid UTF-8 anywhere in the page that return isfalse,falseconcatenates as the empty string, and the fragment file ends up holding nothing but the delimiter — which failsJSON.parse()in the browser, on one result, at search time, with nothing logged anywhere upstream. Both now throw aRuntimeExceptionnaming the page number, the URL andjson_last_error_msg(), matching the guardFacetIndexWriter::build()already had. Covered bytests/Index/FragmentEncodingFailureTest.php, which asserts on both writers and, for the streaming path, that no truncated fragment file reaches disk. - Every search paid Pagefind's filter counter, at a cost proportional to matched results times filter postings, and the first facet click made it permanent (
assets/js/scolta.js). This is the underlying per-search cost the duplicate-search fix below explicitly left as "a separate problem". Root cause is upstream, in Pagefind 1.5'spagefind_web/src/filter.rs:SearchIndex::get_filterstakesintersect_pagesas aVec<usize>and counts withintersection.contains(&p), a linear scan of the matched-result set, for every(value, page)posting in every loaded filter chunk — andlib.rscalls it twice persearch(), once for the filtered counts and once for the totals.bit_set::BitSetis imported at the top of that same file and used by every other function in it; onlyget_filterstakes aVec. So the cost ismatched results x loaded postings, it applies to every search once any chunk is loaded, and there is no unload path short ofpagefind.destroy(). Two triggers loaded chunks:initPagefind()calledpagefind.filters(), which loads all of them, and passing a dimension insearchOpts.filtersmakes Pagefind's client lazily fetch that dimension's chunk, so the first facet click reintroduced the whole cost for the life of the page. The cost tracks postings, not distinct values. Measured on a 109,308-page Drupal corpus (3,208,134 postings, 3,970 distinct values across ten dimensions),pagefind.search('math')with 7,789 matched results, medians of three runs each with a paired in-page baseline: no chunk loaded 155 ms;standard_authority(55 values, 6,468 postings) 259 ms;audience(5 values, 10,004 postings) 291 ms;subject(440 values, 389,545 postings) 2,478 ms;grade(19 values, 491,074 postings) 3,014 ms;curriculum(3,421 values, 1,764,954 postings) 10,312 ms; all ten 18,589 ms. A 440-value dimension is cheaper than a 19-value one, and rewriting the servedcurriculumchunk to collapse 3,421 values into 4 while keeping every posting moved it 10,151 ms to 10,178 ms — 0.3%, where a cost model based on distinct values predicts near-total elimination. Cost is equally linear in matched results: the same all-ten state costs 374 ms for a 166-result query. Fixed by moving both the counting and the filter application out of Pagefind and onto the newscolta.facetsartifact.initPagefind()no longer callspagefind.filters(),pagefindSearch()never populatessearchOpts.filters, and the user's selection is applied against the artifact's posting lists (OR within a dimension, AND across dimensions, Pagefind's own semantics) before the result list is sliced and fragments are loaded. Counts are computed over the full matched set, not the ~75 fragments Scolta loads, so no facet count moves: validated against Pagefind's own output on the production corpus with zero mismatches across all 3,970 values in all ten dimensions, for both query-scoped counts and unfiltered totals. A side effect is that the same query under different facet selections is now one Pagefind search shared through the per-cycle memo, with only the cheap post-filter differing. An index built before this change has no artifact, soscolta.jsfalls back topagefind.filters()and to Pagefind-side filter application, with aconsole.warnthat names the cost and tells the operator to rebuild the search index; facets keep working, slowly, rather than breaking. The same fallback covers a browser withoutDecompressionStream, a payload that is not a Scolta facet index, a stale cached artifact whose stamp disagrees withpagefind-entry.json, and an index whose secondary languages were merged in withpagefind.mergeIndex()(one artifact describes one built index, and counting a merged corpus against it would be visibly wrong). Covered bytests/js/facet-index.test.js:pagefind.filters()is never called when the artifact is present, no search ever carries afiltersoption including after facet clicks in two different dimensions, counts are exact for a 150-result query against aMAX_PAGEFIND_RESULTSof 75, a zero-count value is reported at zero rather than omitted, filters AND across dimensions and OR within one, and each of the five fallback paths warns and still renders facets. The pre-existing suite passes unmodified because it exercises the fallback, sotests/js/result-count-baseline.test.jsandtests/js/search-memo.test.jsare untouched. PagefindFormatWriter::buildFilterIndex()crashed the build on multi-value filter arrays (src/Index/PagefindFormatWriter.php). A multi-value filter field, which is the normal shape for a taxonomy facet, arrives as an array, and the method used$filterValuedirectly as an array key. PHP 8 raisesTypeError: Cannot access offset of type arrayfor both theisset()probe and the assignment, so the build died outright rather than coercing or skipping. This is the identical defect fixed inStreamingFormatWriter::writePageEntry()in 1.0.0-rc5, which normalized withis_array($v) ? $v : [$v]to matchPagefindHtmlBuilder;PagefindFormatWriterwas left behind. Not reachable from any shipped pipeline — nothing insrc/instantiates the class,IndexBuildOrchestratorandPhpIndexerboth useStreamingFormatWriter, and no adapter constructs it — but it is@stability stablepublic API, and the facet-index work touches the same method. Extracted tocollectFilterData()with the same normalization both other emitters already use, and covered bytests/Index/PagefindFormatWriterTest.php.- A filter dimension with one distinct value is no longer emitted as a Pagefind filter chunk (
src/Index/StreamingFormatWriter.php,src/Index/PagefindFormatWriter.php).InvertedIndexBuilderinjects asitefilter wheneverContentItem::$siteNameis set and alanguagefilter whenever$languageis set (in the Drupal adaptersiteNamefalls back tosystem.sitename, so it is populated on essentially every install), andPagefindHtmlBuilderemits the same two on the binary path. On a single-site single-language index that produces two dimensions with exactly one distinct value each, and such a dimension cannot filter anything — selecting its only value matches every page — nor show a useful count. There was no distinct-value guard anywhere in the index build;renderFilters()in the browser has always had one (Object.keys(taxonomy[dim]).length > 1) andSKIP_FILTER_DIMENSIONSskipssiteandlanguageoutright, so these two dimensions were never rendered as facets and this changes nothing a user sees. What they cost was real: on the 109,308-page corpus they were 485,100 of the 6,142,564 served filter bytes and 218,616 of the 3,208,134 postings, every posting of which Pagefind scanned against the matched-result set on every search. The guard applies only to Pagefind's chunks and theirpf_metareferences;scolta.facetsdeliberately still carries every dimension, because Scolta can be asked to apply one (AUTO_LANGUAGE_FILTERapplieslanguage) and applying a filter needs its posting list even when no facet renders it — and a bitmap covering every page gzips to about a hundred bytes, so carrying them costs 176 bytes. Covered bytests/Index/PagefindFormatWriterTest.php(a one-value dimension is absent from the emitted chunks while a three-value one is present) andtests/js/facet-index.test.js(the artifact carries it; the panel does not render it). - Every query ran its Pagefind search twice, and the results were held behind the second one (
assets/js/scolta.js). On a production-size index (a Drupal site with 109,308 indexed pages and 3,970 distinct filter values across 10 dimensions) the querymathtook 35.9 seconds to paint: 24,516 ms in the primary search, then 11,053 ms in a second, byte-identical search, and only then render — the results were finished in memory at 24,558 ms and did not appear until 35,626 ms. Two independent defects, both fixed here. (1) The duplicate search.computeQueryFacetCounts()derivesstructuralFiltersby keeping only theSKIP_FILTER_DIMENSIONSentries out of the filters it is handed, so whenever the user has applied no user-facing facet — the common case — its search is identical to the primary one; on the OR-fallback pathcomputeUnionFacetCounts()likewise re-ran a per-term search for every termsearchAndLoadParallel()had just searched.pagefindSearch()now memoizes in-flight searches for the duration of one search cycle, keyed on the query plus a stable serialization of the resolved Pagefind options (dimension keys and value arrays sorted, sort hint included) rather than the caller's argument, so equivalent scopes share a key and genuinely different ones do not. The memo holds the promise, not the awaited value, so concurrent callers share one in-flight search instead of racing, and it is reset alongside the cycle's other per-cycle state at the top ofdoSearch(), so nothing is reused across cycles. A user-applied facet makesactiveFiltersandstructuralFiltersdiffer and both searches still run — that is the correctness boundary, since collapsing them would silently change the facet counts, and it is asserted in both directions. (2) First paint blocked on the counts.doSearch()awaitedcomputeQueryFacetCounts()before callingrenderResults(), so the user waited for a full second search before seeing the list they asked for. Results now render as soon as they are ready and the count pass runs afterwards, re-rendering the filter panel when it lands, behind aversion === searchVersionre-check so a late count pass from a superseded query can neither overwritequeryFacetCountsnor repaint the panel. The panel is deliberately not repainted during the gap:renderFilters()is count-driven (a zero-count value is hidden entirely under the defaulthideEmptyFacetspolicy), so painting it against not-yet-updated counts would show the previous query's visible value set and then reshuffle — it instead holds its last painted state, which introduces no new visual state and never flashes empty. Neither change alters what the counts contain or how many results are returned; the underlying per-search cost (Pagefind computes per-value counts across every distinct filter value oncefilters()has loaded the filter chunks, roughly 1.45 ms per matched result) is untouched and remains a separate problem.pagefind.debouncedSearch()was again deliberately not adopted: it would bypass theabortController+searchVersionguards. Measured on the same corpus, keystroke to first results:fractions6,584 → 4,571 ms (31% faster),math35,870 → 24,516 ms (32%), a six-word query 58,844 → 29,339 ms (50%). Covered bytests/js/search-memo.test.js(per-querypagefind.searchcall counts with no facet, with a user facet, with a structural-only filter, across the OR-fallback path, and across cycle boundaries) andtests/js/render-order.test.js(results in the DOM while a gated count search is still pending; a superseded cycle's late counts never repaint the panel). - Quality / experience queries now surface the content that embodies them (
src/Prompt/DefaultPrompts.php). A query that names a feeling, reaction, or judgment rather than a topic ("scary moment", "inspiring story") was expanded into synonyms of the adjective ("frightening experience", "terrifying incident"), but authors narrate a tense episode by describing the concrete thing that went wrong (the malfunction, the alarm, the aborted attempt) and almost never label it "scary". The synonym expansion therefore matched the wrong pages and missed the genuinely tense ones, which shared no vocabulary with the expansion at all. This is the same expansion-restates-the-query failure that rules 13-14 (category/context) and rule 16 (named entity) fix, in the variant those rules do not reach: an abstract quality. The mirroredexpand_querytemplate picks up rule 17 (QUALITY / EXPERIENCE → CONCRETE INSTANCES), which expands such a query into the concrete events, systems, or situations that embody the quality in prose, forbids restating the query adjective as a synonym, and reconciles the term cap ("up to 6 concrete instances"); rule 15 still bounds it, so it falls back to neutral topic phrasings when unsure. Examples are drawn from two unrelated domains (aviation history, software) and from no Scolta demo corpus. The rule covers every valence, not only things that went wrong: the funny and inspiring cases are named explicitly and their vocabulary ("amusing story", "uplifting narrative") banned, after the model was observed restating the adjective on exactly those queries; its examples are marked as illustrations rather than a term bank, after it was observed emitting them verbatim for an unrelated corpus. The template text is byte-identical toscolta-core'sEXPAND_QUERY, enforced bytests/Prompt/PromptTextIdentityTest.php. - Agreement-only terms no longer load full documents, so the co-occurrence path stops inflating the per-query loaded-document count (
assets/js/scolta.js, #156). Root cause:searchAndLoadParallel()calleddata()on every sub-query — including the non-seeding terms (the user's typed words and the agreement-only phrase sub-words) — to test whether several terms agreed in the same document, then discarded the non-matching ones before ranking. Loading documents it then threw away pushed the loaded count for thegit-manualreference query (stash uncommitted changes) to 135, tripping theresult-count-baselineguard (#156, band max 90); the displayed result count was already correct, only the loaded count was inflated. Whether a non-seeding term counts as an agreement axis, and whether a URL is seeded, emitted, or dropped, is decided entirely from theresultsentry beforedata()(result-set size, specificity weight, term class, seeding-set membership) — nodata()field decides agreement — so the non-seeding documents can be reasoned about by id and never loaded. Fix: full fragments are loaded only for seeding terms; each non-seeding term's matched document ids are taken straight from itssearch().resultsand intersected against the already-loaded seeded documents, and a surviving contribution's agreement magnitude is scored against the seeded document's already-loaded fragment instead of a freshdata()copy. The join is keyed by the Pagefind entryid(available beforedata()) rather than the loaded fragment url. Loaded count for the failing query drops from 135 to 62, back inside the baseline band, with no change to the guard's fixtures, expected values, or thresholds. Shrinking each non-seeding term's scored set to its survivors changesloaded.length, which feeds the positional rank prior (1 - i/(len-1)) — a ranking effect, sotests/js/cooccurrence-ranking.test.jspins the ranked order of a representative multi-term query (and asserts non-seeding terms introduce no documents of their own) to keep the load change from moving results silently. - A common word in the query no longer floods the result head over the rare terms that carry the intent (
assets/js/scolta.js). On every corpus measured, a high-frequency word — typed, or leaked from an expansion phrase — pulled off-intent documents to the top because the ranker rewarded matching that common word as much as matching the rare discriminating terms.apollo 13 crisisled with generic Apollo posts (matching only "apollo", 131 of 204 posts) while the crisis coverage matching the rare terms ("oxygen tank explosion", "lunar module lifeboat") was buried;a vegetarian dinnerreturned Prime Rib Dinner (matching only "dinner"); a medical query filled its tail with unrelated conditions sharing "lunar"/"resident" (2820+ of ~3650 pages) rather than the vision terms ("papilledema", 24 pages). The OR fallback weighted every typed term at a flat0.6and expansion sub-queries used positional decay — neither reflected rarity.searchAndLoadParallel()now damps each sub-query by a specificity (inverse-document-frequency) weight derived from the term's own scoped Pagefind match count, so a rare term keeps full weight and a ubiquitous one is floored. This closes two guard bypasses without hurting recall: a high-frequency word the user typed (routed through the OR fallback) and a common sub-word leaked from an expansion phrase are both down-weighted at rank time rather than dropped. A naive alternative — ranking by raw count of expansion terms matched — was rejected because the expansion phrases themselves contain common words; counting without rarity weighting just moves the problem. Gated byspecificityWeighting; corpora whose query already matches on rare terms (e.g.undo a mistake i committed→ revert/reset/restore) are unaffected by construction. - The partial-match fallback no longer frames a strong result set as a failure (
assets/js/scolta.js). Almost every conversational query trips the OR fallback, because the exact-match test ANDs every typed word and a natural sentence rarely satisfies it. The header then read "no exact matches found, showing partial matches" and the AI-summary context was marked "not representative of the collection" even when a rare, on-intent term had matched exactly. Both now key off whether any high-specificity term matched (hadSpecificMatch), tracked as sub-queries are scored: when a strong term hit, the banner reads "showing best matches" and the summary hedge attributes any gap to the search rather than the collection; only a genuine thin slice (no specific term matched anything) keeps the strong "not representative" marker and the decline. - Zero-count facet values are hidden instead of rendered disabled (
assets/js/scolta.js). A facet dimension with many values (the social feed's hashtag dimension has hundreds) rendered every value for every query, all but a few at count 0, burying the useful ones.renderFilters()now skips a value whose count is 0 for the current query unless it is actively selected (an active value stays visible and uncheckable), and skips a dimension left with no visible values. Counts are fixed per typed query, so the visible set stays stable across facet clicks. - Identifier / proper-noun queries no longer miss the content that describes them (
src/Prompt/DefaultPrompts.php,assets/js/scolta.js). A query centred on a named entity or event ("Apollo 13 crisis", "Apollo 1 fire") was expanded into terms that all kept the entity anchor — "Apollo 13 accident", "Apollo 13 explosion", "Apollo 13 oxygen tank failure" — and against a real Pagefind index every one of those returned zero results, because authors write "the mission", "Lovell", "the lifeboat" far more often than they repeat the mission number. The bare in-corpus vocabulary that does match ("oxygen tank explosion" → the crisis post; "lunar module lifeboat" → all three) was never emitted, so the search fell back to generic matches. This is neither a Pagefind bug, nor a numeric-tokenization bug (1969→ 55 docs,24000→ 19), nor a stale index, nor scoring config: it is the same expansion-restates-the-query failure as the category/context cases fixed by rules 13-14, in the variant those rules do not reach — the restatement preserves an anchor rather than a category name. New rule 16 (NAMED ENTITY / EVENT → DEFINING DETAILS) expands such a query into the concrete details that identify it in prose (participants, components, distinctive phrases, causes, consequences), requires that at least half the terms drop the entity name entirely, and explicitly forbids appending the name to a list of near-synonyms. The term cap is reconciled ("up to 6 defining details"); rule 15 still bounds it, so an unrecognized entity falls back to neutral phrasings rather than acquiring invented participants or parts. Examples are drawn from three unrelated domains (consumer product, vehicle spec, historical event) and from no Scolta demo corpus, so the behaviour cannot be explained by the prompt naming demo content. - The AI summary can no longer claim the collection lacks content (
src/Prompt/DefaultPrompts.php). Thesummarizegrounding check previously instructed this failure: its CORPUS AWARENESS bullet told the model to say "[site] focuses on [scope], so it doesn't include a dedicated article on [topic]". The model only ever sees one search's slice of the corpus, so it can never support that claim — and it was observed telling a user that a full week-long article arc did not exist. The bullet is replaced by four rules: PARTIAL VIEW (the excerpts are a slice, never the collection), NEVER ASSERT ABSENCE (naming the observed phrasings as banned), WEAK RESULT SETS (attribute a thin result set to this search, not the collection, and suggest narrower terms), and the preserved no-invented-statistics guard from #33. Thefollow_upgrounding check carried the same defect ("This collection doesn't appear to have content on [topic].") and is fixed the same way. The pinned snapshot fixture moves fromtests/fixtures/corpus-awareness-bullet.txttotests/fixtures/absence-grounding-rules.txt. - The summarizer is now told when it is looking at a fallback result set (
assets/js/scolta.js). When the full query matched nothing,doSearch()rebuilds the list from the broadened OR fallback and the ranked list already warns the user "no exact matches found, showing partial matches" — butsummarizeResults()passed that same slice to the LLM with no marker at all, which is what let a thin, off-target set be generalized into a claim about the whole corpus. The context header now carries[No result matched the full query; ...]in exactly that case, and the summarize prompt keys off it. Covered bytests/js/summary-weak-match-signal.test.js(marker present on fallback, absent on a normal search, and ordered ahead of the excerpts). - Query expansion now runs at temperature 0 for deterministic terms (
src/Service/AiServiceAdapter.php,messageForOperation()). Expansion calls went out with the provider default temperature (1.0), so the same search query produced different expansion terms on every uncached call — flush the cache and re-search, get different terms and different results. Because the displayed result count is the per-term OR-union across the primary query plus every expansion term, this also made expanded counts LLM-volatile between runs (the 2026-07-13 regression work had to re-base its count tripwires on base-only reads to work around exactly this).messageForOperation()now passestemperature: 0.0when$operation === 'expand_query'and null otherwise. Summarize and follow-up are creative surfaces and intentionally keep the provider default.
Known limitations
- Facet counts do not exactly match the filtered result list once AI query expansion has run (#265). The panel now recounts after expansion, which it did not do before this release, and on every corpus measured it is strictly closer to the list than the previous behaviour was. It is not yet exact. The count pass runs one search per expansion term, each capped independently at
MAX_PAGEFIND_RESULTS, and unions them, so on a large corpus with several terms the tally describes a different population from the ranked list: measured divergences run from off by one on a 204 page blog to a value counted at 1 that returns nothing when clicked on a 12,541 page social feed. Unexpanded queries are exact. Targeted for 1.1.1; the tracking issue records the three fixes considered and why none is a small change.