v1.6.0

[pagefinder[bot]]

👋 A little overdue, but this release knocks out a healthy selection of bug reports, and includes a lot of work from the community. Thanks to everyone who contributed ❤️ - @bglw

Multilingual Segmentation

Pagefind's extended build now segments Chinese, Japanese, and Thai pages with ICU4X, replacing the previous dependency (PR #1383). Alongside producing a smaller binary, ICU4X segments a lot closer to the Intl.Segmenter used in the browser on the search query (which is ICU4C). This closes out a mixture of latent and reported issues where the different segmentation between indexing and searching makes for iffy results. It's not perfect yet, but it's a darn sight closer.

Bonuses:

• Segmentation is smarter in languages such as Thai, and doesn't break inside a grapheme cluster
• Japanese katakana has special handling, but should still be improved over 1.5.2
• Japanese voiced and semi-voiced sound marks are no longer normalized into each other

Fewer Fragment Files

Pagefind now has an option to limit the total number of fragment files produced, for hosting providers that cap the number of uploaded files. Contributed in PR #1020 — thanks @yakovsh !

This limit is set with --max-fragments / max_fragments, and applies per language. When applied, multiple fragments are bundled together and loaded as a set when required.

See 📘 Max fragments.

Explicit Sub-Result Titles

Added support for explicit sub-result titles. Any element with both an id and a data-pagefind-title attribute can be used as a sub-result (PR #1282 — thanks @samuel-williams-shopify !)

See 📘 Defining a sub result explicitly.

Component UI

• Added a sort attribute to <pagefind-config>, to order results by a sort key (PR Add a sort attribute to <pagefind-config> #1326)
• Added a max-sub-results attribute to <pagefind-searchbox> (PR Add max-sub-results attribute to <pagefind-searchbox> #1324)
• Added a placeholder setter to <pagefind-searchbox> and <pagefind-modal-trigger> (PR Add a placeholder setter to the trigger and searchbox components #1321)
• configureInstance now applies to an instance that already exists but has not loaded yet (PR Apply configureInstance options to existing unloaded instances #1323)
• Fixed the UI language being resolved after the first render, which could flash the wrong translations (PR Resolve the Component UI language before the first render #1332)
• Fixed component cleanup after being disconnected (PR Unregister Component UI elements from their instance on disconnect #1322)
• Fixed IME composition keystrokes support (PR fix(component-ui): ignore keystrokes belonging to an IME composition #1284 — thanks @oharu121 !)
• Fixed the modal staying open when clicking a result for the current page (PR Close the modal when a result is activated on the same page #1327)
• Fixed the filter panel checkbox shrinking when its label wraps (PR Prevent filter pane checkbox from shrinking #1166 — thanks @mbrambilla !)
• Hid the keyboard shortcut badge on touch-only devices (PR Hide the shortcut badge on touch-only devices #1325)

Search Results & Options

• Added an excerptEllipsis option to mark the truncated ends of excerpts (PR Add an opt-in excerptEllipsis search option #1320)
  • See 📘 Excerpt ellipsis
• Fixed phrase search across diacritic variants, so searching "cafe latte" can match a page containing café latte (PR Fix exact phrase matching across diacritic variants #1386 — thanks @sean0407 !)
• Fixed filters not matching values containing escaped characters such as quotes or backslashes (PR Use hifijson for JSON parsing in Rust #1392)
• Fixed compound words being counted twice in metadata coverage, which could overvalue a metadata match (PR Don't double-count compound words in metadata coverage #1319)
• Fixed a bug which could build excerpts from the wrong words (PR Align the excerpt tokenizer with the Rust indexer on U+FEFF #1318)
• Fixed decoding for HTML entities in the automatic image and image_alt metadata (PR Decode HTML entities in the automatic image metadata #1393 — thanks @digilist !)

Indexing & Performance

• More work making the index deterministic between builds (PR completion of deterministic output #1124 — thanks @gissimo !)
• Improved search performance on a bunch of broad queries (PR Perf: group matching words by page before scoring #1334)
• Fixed a bug where a word with many variants could produce an oversized index chunk (PR Count word variants toward index chunk sizes #1316)
• Searches no longer request index chunks for empty terms (PR Don't request index chunks for an empty search term #1317)
• More minor search performance work (PR Minor optimization: Flatten results using flatMap for better performance and memory management 🚄  #1238 — thanks @sanjaiyan-dev !)

Fixes & Tweaks

• Search terms that normalize to nothing are now dropped, rather than searching for an empty term (PR Drop search terms that normalize to nothing #1382)
• Improved some compound indexing, so sysconfig.is_python_build can be found by searching is_python_build
• Fixed a way that output files could be truncated in service-mode (PR fix: flush output files before they drop #1272 — thanks @zelinskyy !)
• Fixed --exclude-selectors hard-erroring when a single selector could not be parsed (PR fix(pagefind): skip exclude selectors that cannot be parsed #1312 — thanks @dualfroz !)
• Improved handling of percent-encoded page URLs (PR Percent-encode %, # and ? when building page URLs from filenames #1315, PR Percent-encode page URLs built from file paths #1335 — thanks @dualfroz !)
• Failed index fetches are now evicted from the cache correctly and are retried (PR Retry index chunk and fragment fetches that failed #1329)
• Per-page warnings are now only logged with --verbose (PR Only log per-page warning details when verbose #1314)

UI Translations

• Added Macedonian (mk) translations (PR Add Macedonian translation #1353 — thanks @I90rche !)
• Added Bulgarian (bg) translations (PR Add Bulgarian translation #1341 — thanks @salif !)
• Improved French (fr) translations (PR i18n(fr): Improve translation quality #1145 — thanks @maxonss !)
• Improved German (de) translations (PR i18n(de): Improve translation #1096 — thanks @randomguy-2650 !)
• Improved Persian (fa) translations (PR i18n(fa): improve and update Persian translations #1204 — thanks @Goudarz !)
• Improved Swahili (sw) translations (PR Update sw.json #1185 — thanks @gully223 !)

---

This discussion was created from the release v1.6.0.

[jaywhj]

Awesome update! 👍

mkdocs-materialx also has Pagefind search tightly integrated — you could also use it to generate the Pagefind docs.

Give it a try here: https://jaywhj.github.io/mkdocs-materialx/differences.html