v4.9.0

Highlights

• @byline/search-analysis — a new package, published for the first time in this release, holding Byline's portable multilingual analysis layer. It converts original text and query intent into a deterministic logical representation that any backend can share: search-only NFKC normalization with offsets back into the original string, validated locale and script resolution (Thai, Japanese, Korean, Lao, Khmer, Myanmar, Han), protected domain identifiers (URLs, email addresses, mentions, hashtags, SKUs, versions, and terms such as C++, C#, Node.js), ICU word segmentation via Intl.Segmenter, optional exact-preserving language expanders, overlapping Han bigrams for substring recall across segmentation boundaries, and a stable analyzer fingerprint that makes rebuild decisions explicit. It is synchronous and CPU-local — no database, no I/O, no model inference.

  The package also owns encodeSqlToken(), which encodes every logical term as parser-safe lowercase ASCII before any SQL engine sees it. That single decision is what lets one matching contract hold across PostgreSQL and MySQL: engine stopword lists, minimum token lengths, and punctuation splitting stop being part of Byline's behaviour, and free-text never reaches to_tsquery or MATCH … AGAINST as syntax.

• @byline/search-mysql — a new package and Byline's second built-in search provider, storing portable terms in weighted MySQL FULLTEXT indexes with its own migration stream, analyzer-fingerprint enforcement, zone and locale scoping, and portable highlighted snippets. This closes the gap called out in the 4.8.0 notes, where a MySQL installation had no search provider at all and needed the documented no-op workaround to get past initBylineCore(). MySQL installations can now register a real provider. The wider MySQL adapter remains preliminary — the general-availability criteria in #58 still stand.

• @byline/search-postgres — replaced PostgreSQL-native regconfig analysis and websearch_to_tsquery with the shared portable analyzer and query plan, stored as one weighted GIN-indexed tsvector. The adapter now supports all-terms, any-terms, minimum-should-match, quoted phrases, protected identifiers, exact-preserving expansions, and ordered Han-bigram fallback — the same contract the MySQL driver implements. See Migrations below: this is a direct cutover, not an in-place upgrade.

• @byline/core — search queries now carry explicit, provider-neutral matching intent through SearchMatching: operator: 'all' | 'any', minimumShouldMatch, and phrase: 'auto' | 'required' | 'off'. This is product behaviour rather than a ranking hint, so a provider must either translate it faithfully or declare the gap through capabilities.fullText. MAX_SEARCH_QUERY_LENGTH (1,024 UTF-16 code units) is also exported and enforced at both the client entry points and inside the analyzer, so a direct provider implementation does not have to rely on route-level validation.

• Highlighting — both built-in providers return matched snippets built from the stored original text, marking exact, normalized, expanded, identifier, and Han-gram matches through the same portable token offsets used for retrieval. Highlighting runs in Node.js for the ranked rows of the requested page only; it never participates in corpus scanning or index writes, and it required no schema change. The returned string carries <mark>…</mark> delimiters and is not trusted HTML — consumers must split the delimiters and let their framework escape the surrounding text, as the documented example does.

Chores

• monorepo — a new private @byline/search-conformance package holds one executable contract for search providers: capability declarations, lifecycle and scoping, full-text matching, highlighting, portable-analysis survival, weighting, and analyzer-fingerprint rebuild enforcement. Both built-in SQL drivers run the complete aggregate suite against a live database. The suite deliberately does not compare raw scores across engines — it asserts positive ranking and relative order where the contract requires it, leaving each engine free to use its own relevance model.

• monorepo — docs/06-search/ is a new documentation section covering configuration, indexing and reindexing, the query API, the provider contract, portable analysis, a comparison of the two SQL providers, and the planned attachment-extraction boundary.

Migrations

• @byline/search-postgres, @byline/search-mysql — the search index is disposable derived state, and this release changes both the physical schema and the analyzer fingerprint. There is no in-place migration and no native compatibility mode. For each installation with an existing search index: drop the three provider-owned tables (byline_search_documents, byline_search_index_metadata, byline_search_migrations), apply the current 0001_init.sql, then rebuild from published content with client.collection(path).reindex() or the admin Reindex action.

  Dropping the migration ledger matters as much as the data tables. The migrator skips versions already recorded there, so removing only the two data tables leaves version 1 marked applied and the new schema never arrives — which then surfaces as a failing upsert against a column that does not exist. No CMS document tables are affected by any of this.

  Sequence the rebuild with the deploy rather than after it. Between installing this release and completing the reindex, the fingerprint guard rejects every query for the affected collection with SEARCH_INDEX_REINDEX_REQUIRED — deliberately, since returning results from a mixed-analyzer index is worse than returning none.

Breaking Changes

• @byline/core — breaking for third-party SearchProvider implementations only. SearchCapabilities gained a required fullText block (nativeAnalysis, portableAnalysis, allTerms, anyTerms, minimumShouldMatch, phrase), which replaces the previously proposed lexical field, and SearchProvider.reindex(opts) is no longer optional — every derived projection must be explicitly clearable so it stays rebuildable.

  Required action: if you ship your own SearchProvider, add a fullText block to your capabilities object and implement reindex. Declare nativeAnalysis: true if your engine analyses original text with its own language analyzers, or portableAnalysis: true if it indexes Byline's logical terms. Set the four matching flags to what your query translation genuinely does today — an engine that does not yet translate SearchMatching should declare those as false rather than inheriting an optimistic default. Applications that only consume search are unaffected and need no change.

  Why this is a minor rather than a major release: the provider seam is new and still settling — several parts of the search subsystem are yet to land, and the intent is to get schema migration and downstream compatibility resolved on a minor so the next feature bump, and an eventual major, are uneventful. Every publishable @byline/* package moves in one lockstep group, and taking all eighteen to 5.0.0 for a contract that has been public for days is not a trade worth making. Providers that need to move can do so in a few lines.

All other @byline/* packages bumped to 4.9.0 in lockstep with no behavioural changes this cycle.